CN114066857A - Infrared image quality evaluation method and device, electronic equipment and readable storage medium - Google Patents
Infrared image quality evaluation method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN114066857A CN114066857A CN202111370853.0A CN202111370853A CN114066857A CN 114066857 A CN114066857 A CN 114066857A CN 202111370853 A CN202111370853 A CN 202111370853A CN 114066857 A CN114066857 A CN 114066857A
- Authority
- CN
- China
- Prior art keywords
- image
- score
- objective
- test
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 186
- 238000000034 method Methods 0.000 title claims abstract description 89
- 238000003860 storage Methods 0.000 title claims abstract description 24
- 238000012360 testing method Methods 0.000 claims abstract description 223
- 238000011156 evaluation Methods 0.000 claims abstract description 151
- 238000012549 training Methods 0.000 claims abstract description 97
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 27
- 238000010801 machine learning Methods 0.000 claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 24
- 230000035945 sensitivity Effects 0.000 claims description 20
- 238000005259 measurement Methods 0.000 claims description 19
- 238000010606 normalization Methods 0.000 claims description 13
- 238000001931 thermography Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010276 construction Methods 0.000 claims description 8
- 238000004806 packaging method and process Methods 0.000 claims description 8
- 238000005286 illumination Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 5
- 230000008447 perception Effects 0.000 abstract description 27
- 238000004364 calculation method Methods 0.000 description 26
- 230000008569 process Effects 0.000 description 16
- 238000003646 Spearman's rank correlation coefficient Methods 0.000 description 9
- 238000003331 infrared imaging Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000012854 evaluation process Methods 0.000 description 3
- 210000000887 face Anatomy 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000001303 quality assessment method Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 125000000484 butyl group Chemical group [H]C([*])([H])C([H])([H])C([H])([H])C([H])([H])[H] 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000035622 drinking Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The application discloses an infrared image quality evaluation method and device, electronic equipment and a readable storage medium. The method comprises the steps of generating a training set and a testing set of image-carried image quality subjective scores based on an original infrared image data set; training a machine learning model by utilizing each training image of a training set and an image quality subjective score thereof to obtain an initial quality evaluation model; calculating the objective score of each objective quality index of each test image in the test set by using a non-reference evaluation algorithm, and determining the objective sub-dimension evaluation score of each test image; inputting each objective sub-dimension evaluation score into an initial quality evaluation model to obtain an image quality objective score of each test image; based on the initial quality evaluation model, a final image quality evaluation model is determined according to the image quality objective scores and the corresponding image quality subjective scores of the test images, so that the infrared image quality evaluation highly consistent with the human eye subjective perception can be efficiently and conveniently realized.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an infrared image quality evaluation method and apparatus, an electronic device, and a readable storage medium.
Background
As is well known, unlike the visible light imaging principle, infrared thermal imaging systems perform imaging by sensing the temperature difference between the thermal radiation of an object and the thermal radiation of the background. Because of its passive formation of image characteristics, infrared imaging equipment can be at dark night or the not good scene of illumination environment or the harsh environment that smoke and dust is densely covered like: the image shooting is carried out clearly in rain and fog days, haze days and the like, the defect that visible light imaging is limited by illumination conditions is overcome, and the method is widely applied to the fields of military affairs, industry, automobile auxiliary driving, security protection, medicine and the like.
Due to the fact that imaging wavelength is long, degradation phenomena such as large noise, low image contrast, low signal-to-noise ratio, unclear edges, fuzzy visual effect, narrow gray scale range and the like generally exist in an infrared image, in addition, in the processes of sensing, obtaining, storing and transmitting and image post-processing operation of the infrared detector device, some noise and fuzziness are inevitably introduced, even some information is lost, and the factors can cause image quality reduction or image distortion. The quality of the infrared image directly determines the visual experience of a user and the acquisition of information quantity, and the evaluation of the quality of the infrared image is particularly important. Although there are parameter indexes such as minimum distinguishable temperature difference and noise equivalent temperature difference in infrared imaging, none of the indexes can objectively reflect subjective feeling of a user on an image. Therefore, the general infrared image quality evaluation method which is highly consistent with the subjective feeling of the user is applied, and by the method, the user can accurately know the quality of the infrared image to be evaluated, so that the method can be used for guiding the construction and adjustment of infrared image acquisition equipment and a processing system, optimizing an image processing algorithm and parameter setting, and finally presenting the image with higher quality to the user.
In the visible light field, current image quality evaluation methods include a subjective evaluation method and an objective evaluation method. The subjective evaluation method is to subjectively score an image by an observer, and then calculate an average subjective score or an average subjective score difference value, and specifically, may be divided into an absolute evaluation method and a relative evaluation method. The objective evaluation method is that a computer obtains the quality index of an image according to a certain algorithm, and is divided into full-reference, half-reference and no-reference evaluation methods according to whether a reference image needs to be introduced in the evaluation process, wherein the no-reference evaluation method is also called blind image quality evaluation. In the field of infrared imaging, a subjective evaluation method is still applicable, and the method has the advantages of accuracy, reliability, strong subjectivity, difficult expression by a mathematical model, complex realization, time consumption and labor consumption and is influenced by factors such as professional background, psychology and motivation of an observer. As for the objective evaluation method, the full-reference evaluation model usually has the best effect because all image information can be utilized, but in the infrared imaging process, because an original image without distortion cannot be obtained, the objective evaluation of the infrared image can only be realized by a non-reference evaluation method. The no-reference evaluation method is divided into specific distortion evaluation and non-specific distortion type evaluation, the specific distortion evaluation method is mature, the most widely applied evaluation is the evaluation of image blur and noise, and in addition, the evaluation of blocking effect and JPEG (Joint Photographic Experts Group) compression distortion is also provided, but in practical application, infrared images are often damaged by various distortions, so that the evaluation algorithm aiming at specific distortion cannot reflect the overall quality level of the images. The non-specific distortion type evaluation is closer to the user evaluation mode and more valuable, and the related art is generally based on a method of a support vector machine, such as a BRISQE (No-Reference Image Quality Assessment) algorithm, a method based on a probability model, such as a NIQE (Natural Image Quality Assessment) algorithm, and in addition, a dictionary-based method and a neural network-based method, etc., for Image evaluation, but the above methods are complex in calculation process and poor in operability in a real application scene.
In view of this, how to efficiently and conveniently realize the infrared image quality evaluation highly consistent with the human eye subjective perception is a technical problem to be solved by those in the field.
Disclosure of Invention
The application provides an infrared image quality evaluation method, an infrared image quality evaluation device, electronic equipment and a readable storage medium, which can efficiently and conveniently realize infrared image quality evaluation highly consistent with human eye subjective perception.
In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:
the embodiment of the invention provides an infrared image quality evaluation method on one hand, which comprises the following steps:
generating a training set and a testing set of image-carried subjective image quality scores based on an original infrared image data set;
training a machine learning model by utilizing each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model;
calculating the objective score of each objective quality index of each test image in the test set by using a non-reference evaluation algorithm, and determining the objective sub-dimension evaluation score of each test image;
inputting each objective sub-dimension evaluation score into the initial quality evaluation model to obtain an image quality objective score of each test image;
and determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
Optionally, the generating a training set and a test set of subjective scores of image quality carried by images based on the original infrared image data set includes:
acquiring infrared images acquired by a plurality of infrared thermal imaging devices in various types of application scenes under different illumination environments to form an original infrared image data set; the infrared thermal imaging equipment is different in manufacturer, packaging mode and resolution;
acquiring image quality subjective scores of all infrared images in the original infrared image data set;
dividing each infrared image in the original infrared image data set into a plurality of training images and a plurality of testing images based on a preset division ratio;
generating a training set according to the plurality of training images and the image quality subjective scores corresponding to the training images;
and generating a test set according to the multiple test images and the image quality subjective scores corresponding to the test images.
Optionally, the obtaining of the image quality subjective score of each infrared image in the original infrared image data set includes:
acquiring subjective evaluation scores of a plurality of experts on each subjective quality index of each infrared image to obtain subjective sub-dimension evaluation scores of each infrared image;
acquiring subjective evaluation scores of a plurality of experts and a plurality of common users on the whole infrared images, and calculating the subjective overall evaluation scores of the infrared images according to preset expert weight coefficients and user weight coefficients;
and determining the image quality subjective score of each infrared image according to the subjective sub-dimension evaluation score and the subjective overall evaluation score of each infrared image.
Optionally, the training of the machine learning model by using each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model includes:
a support vector regression model is constructed in advance;
constructing a score feature vector according to the subjective sub-dimension evaluation score of each training image;
and inputting the score feature vector and the subjective overall evaluation score corresponding to each training image into the support vector regression model for training to obtain the initial quality evaluation model.
Optionally, the calculating, by using a non-reference evaluation algorithm, an objective score of each objective quality index of each test image in the test set, and determining an objective sub-dimension evaluation score of each test image includes:
calculating the image space variance of each test image and carrying out normalization processing to obtain a non-uniformity objective score;
determining an objective score of image noise by calculating a local normalized brightness coefficient of each test image;
determining an objective image definition score by calculating the structural similarity between each test image and a corresponding reference image;
calculating an objective score of image contrast of each test image based on the image spatial frequency;
calculating the objective fraction of the dynamic brightness range of each test image based on the maximum gray level and the minimum gray level of the image;
and for each test image, carrying out normalization processing on the non-uniformity objective score, the image noise objective score, the image definition objective score, the image contrast objective score and the dynamic brightness range objective score of the current test image to obtain the objective sub-dimension evaluation score of the current test image.
Optionally, the determining an objective noise score of the image by calculating a local normalized luminance coefficient of each test image includes:
calculating a local normalized brightness coefficient of the current test image for each test image, and fitting the local normalized brightness coefficient through a generalized Gaussian model to obtain a fitting parameter mean value and a fitting parameter variance;
calculating local normalized brightness coefficient neighborhood coefficients of the local normalized brightness coefficients in multiple directions, and fitting the local normalized brightness coefficient neighborhood coefficients through an asymmetric generalized Gaussian model to obtain multiple fitting parameters;
and extracting multidimensional statistical characteristics from the fitting parameter mean, the fitting parameter variance and each fitting parameter in different scales, and inputting the multidimensional statistical characteristics to the initial quality evaluation model to obtain the objective image noise score of the current test image.
Optionally, the determining the objective score of the image sharpness by calculating the structural similarity between each test image and the corresponding reference image includes:
for each test image, carrying out low-pass filtering on the current test image by using a Gaussian smoothing filter to obtain a corresponding current reference image;
respectively extracting gradient information and edge information in a target direction of the current test image and the current reference image;
generating a test gradient image according to the gradient information and the edge information of the current test image; generating a reference gradient image according to the gradient information and the edge information of the current reference image;
determining a plurality of target image blocks meeting preset gradient information conditions from the test gradient image, and determining that each target image block corresponds to a target reference image block of the reference gradient image;
calling a pre-constructed image structure similarity relational expression to calculate the structure similarity between each target image block and the corresponding target reference image block; the image structure similarity relational expression is determined according to a brightness comparison function, a contrast comparison function, a structure information comparison function and respective weight coefficients;
and determining the image definition objective fraction of the current test image according to the structural similarity of each target image block.
Optionally, the calculating an objective score of image contrast of each test image based on the image spatial frequency includes:
traversing the current test image according to the spatial frequency in the horizontal direction and the vertical direction by using a preset size template for each test image to obtain a spatial frequency matrix of the current test image;
based on the spatial frequency matrix, normalizing the image spatial frequency of each pixel point one by one to obtain a normalized image spatial frequency matrix;
determining a contrast sensitivity weight matrix of the current test image according to a pre-constructed contrast sensitivity relational expression and the normalized image space frequency matrix;
and calculating the objective score of the image contrast of the current test image according to the contrast sensitivity weight matrix, the image size of the current test image, the maximum gray level and the minimum gray level.
Optionally, the determining, based on the initial quality evaluation model, a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images includes:
respectively calculating the performance measurement indexes of the image quality objective score and the image quality subjective score of the current test image for each test image; wherein the performance measure comprises one or more of a Pearson linear correlation coefficient, a Spanish rank correlation coefficient, a Kendel rank correlation coefficient, and a root mean square error;
if the performance measurement index meets a preset performance condition, taking the initial quality evaluation model as the image quality evaluation model;
and if the performance measurement index does not meet the preset performance condition, generating an instruction for optimizing the initial quality evaluation model, and training the initial quality evaluation model again until the preset performance condition is met.
In another aspect, an embodiment of the present invention further provides an infrared image quality evaluation method, including:
an image quality evaluation model is obtained by utilizing the infrared image quality evaluation method in advance;
acquiring an infrared image to be evaluated;
calculating an objective score of each objective quality index of the infrared image to be evaluated by using a non-reference evaluation algorithm, and determining an objective sub-dimension evaluation score of the infrared image to be evaluated;
and inputting the objective sub-dimension evaluation score to the image quality evaluation model to obtain the image quality evaluation score of the infrared image to be evaluated.
In another aspect, an embodiment of the present invention further provides an infrared image quality evaluation apparatus, including:
the data set construction module is used for generating a training set and a test set of image-carried subjective scores of image quality based on an original infrared image data set;
the model training module is used for training a machine learning model by utilizing each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model;
the sub-dimension score calculating module is used for calculating the objective score of each objective quality index of each test image in the test set by utilizing a non-reference evaluation algorithm and determining the objective sub-dimension evaluation score of each test image;
the objective evaluation module is used for inputting the evaluation scores of the objective sub-dimensions into the initial quality evaluation model to obtain the image quality objective score of each test image;
and the model determining module is used for determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
Another aspect of the embodiments of the present invention provides an infrared image quality evaluation apparatus, including:
the model construction module is used for obtaining an image quality evaluation model by utilizing the infrared image quality evaluation method in advance;
the image acquisition module is used for acquiring an infrared image to be evaluated;
the objective scoring module is used for calculating an objective score of each objective quality index of the infrared image to be evaluated by utilizing a no-reference evaluation algorithm and determining an objective sub-dimension evaluation score of the infrared image to be evaluated;
and the quality evaluation module is used for inputting the objective sub-dimension evaluation score to the image quality evaluation model to obtain the image quality evaluation score of the infrared image to be evaluated.
An embodiment of the present invention further provides an electronic device, which includes a processor, and the processor is configured to implement the steps of the infrared image quality evaluation method according to any one of the foregoing methods when executing the computer program stored in the memory.
Finally, an embodiment of the present invention provides a readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the infrared image quality evaluation method are implemented.
The technical scheme provided by the application has the advantages that the subjective overall perception evaluation score of human eyes on an image is used as a characteristic vector training model, an objective evaluation method which is high in consistency with the subjective perception of the human eyes and faces to specific distortion is selected for objective evaluation on an infrared image to be tested, objective score values of the infrared image to be tested in each sub-dimension are obtained, objective score vectors of each sub-dimension are input into a trained model, objective total score of the infrared image to be tested is obtained, the objectivity of the subjective score and the objective score are effectively combined, and finally obtained objective infrared image quality score is highly consistent with the subjective perception of the human eyes. After the model training is completed, in the whole image quality evaluation process, only the objective sub-dimension evaluation score of the infrared image to be evaluated needs to be calculated, and the infrared image quality evaluation result highly consistent with the human eye subjective perception can be obtained, so that the infrared image quality evaluation highly consistent with the human eye subjective perception can be efficiently and conveniently realized, the operability is high, and the practicability is better.
In addition, the embodiment of the invention also provides a corresponding implementation device, electronic equipment and a readable storage medium for the infrared image quality evaluation method, so that the method has higher practicability, and the device, the electronic equipment and the readable storage medium have corresponding advantages.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the related art, the drawings required to be used in the description of the embodiments or the related art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flow chart of an infrared image quality evaluation method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of another infrared image quality evaluation method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of another infrared image quality evaluation method according to an embodiment of the present invention;
fig. 4 is a structural diagram of an embodiment of an infrared image quality evaluation apparatus according to an embodiment of the present invention;
fig. 5 is a structural diagram of another specific embodiment of an infrared image quality evaluation apparatus according to an embodiment of the present invention;
fig. 6 is a block diagram of an embodiment of an electronic device according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.
Having described the technical solutions of the embodiments of the present invention, various non-limiting embodiments of the present application are described in detail below.
Referring to fig. 1, fig. 1 is a schematic flow chart of an infrared image quality evaluation method according to an embodiment of the present invention, where the embodiment of the present invention includes the following:
s101: and generating a training set and a testing set of image-carried subjective scores based on the original infrared image data set.
In this embodiment, the original infrared image data set includes a plurality of infrared images, each of the infrared images is obtained by the infrared thermal imaging device acquiring a target scene, and the target scene may be a scene specified by a person skilled in the art in an indoor environment, an outdoor environment, any lighting condition, any time period, or any external environment. The image quality subjective score is the subjective opinion of the user on the infrared image quality based on the user, and is the subjective perception of human eyes. Image quality subjective scores may be first performed on each infrared image of the original infrared image dataset or on a designated part of the infrared images, and the obtained image quality subjective scores may be used as labels of the infrared images, and then, according to a certain ratio, for example, 8: and 2, dividing each infrared image of the original infrared image data set into a training set and a testing set. In order to distinguish the infrared images in the data sets, the infrared images in the original infrared image data set are referred to as infrared images, the infrared images included in the training set are referred to as training images, and the infrared images included in the test set are referred to as test images. The number of the infrared images contained in the training set and the test set can be flexibly selected according to the actual application scene, and certainly, in order to expand the original infrared image data set, various processing such as turning, cutting, denoising and the like can be performed on each infrared image in the original infrared image data set, and the processed infrared images are supplemented into the original image data set. Of course, the original infrared image data set may be divided into a training set and a test set according to a certain ratio, and then the subjective image quality scoring may be performed on each training image or designated training image in the training set, each test image or designated test image in the test set, which does not affect the implementation of the present application.
S102: and training a machine learning model by utilizing each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model.
The machine learning model in this embodiment may be any existing machine learning model, such as a support vector machine model, a support vector regression model, a convolutional neural network model, a forward multilayer perceptron, and the like, and a person skilled in the art may flexibly select a corresponding machine learning model according to actual needs, train the machine learning model using each training image in a training set and its corresponding image quality subjective score as sample data, and obtain an initial quality evaluation model for image quality evaluation.
S103: and calculating the objective score of each objective quality index of each test image in the test set by using a non-reference evaluation algorithm, and determining the objective sub-dimension evaluation score of each test image.
In the step, a no-reference evaluation algorithm aiming at specific dimensionality distortion is introduced to each test image in a test set to calculate the score of the objective sub-dimensionality. For each test image, determining an objective sub-dimension evaluation score according to the objective score corresponding to each objective quality index, wherein the objective sub-dimension evaluation score can be a data set or a matrix, each element is the objective score of each objective quality index, the objective sub-dimension evaluation score can also be a score, the score is obtained by adding the objective scores corresponding to the objective quality indexes and can also be obtained by carrying out weighted summation on the objective scores corresponding to the objective quality indexes, the weighting factor of each objective quality index can be set in advance according to an actual application scene, and each test image corresponds to a group of objective sub-dimension evaluation scores.
S104: and inputting the evaluation scores of the objective sub-dimensions into the initial quality evaluation model to obtain the image quality objective score of each test image.
After the objective scores of the images in the sub-dimensions are calculated and obtained by the objective evaluation algorithm aiming at the specific dimension distortion in the last step, the objective scores are input into an initial quality evaluation model trained in S102, image quality scores are output after the initial quality model is calculated and processed, and the output result is the objective total score of the test image.
S105: and determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
It can be understood that the technical problem to be solved by the present application is to obtain an infrared image quality evaluation result highly consistent with human eye subjective perception, so that the finally obtained infrared image quality evaluation result is closer to the human evaluation result, but the evaluation process does not depend on human subjective factors, and the image quality evaluation result highly consistent with human eye subjective perception is output by a quality evaluation model. Based on this, the performance of the initial quality evaluation model obtained by training in the step S102 is determined whether to meet the requirement by judging whether the image quality result output by the initial quality evaluation model is highly consistent with the subjective perception of human eyes, if so, the initial quality evaluation model in the step S102 can be directly used as the quality evaluation model used in the actual infrared image evaluation process, and if not, the initial quality evaluation model needs to be optimized until the requirement is met. As for the performance of the initial quality evaluation model, the difference between the subjective score of the image quality carried in step S101 and the objective score of the image quality output in step S104 is determined by comparing them, and the smaller the difference between the two is, it is proved that the higher the consistency between the objective score of the image quality output by the initial quality evaluation model and the subjective perception of human eyes is, the higher the performance of the initial quality evaluation model is.
In the technical scheme provided by the embodiment of the invention, the subjective overall perception evaluation score of human eyes on an image is used as a characteristic vector training model, an objective evaluation method which has high consistency with the subjective perception of the human eyes and faces specific distortion is selected for objective evaluation on an infrared image to be tested to obtain objective score values of the infrared image to be tested in each sub-dimension, the objective score vectors of each sub-dimension are input into the trained model to obtain objective total score of the infrared image to be tested, the objectivity of the subjective score and the objective score are effectively combined, and finally obtained objective infrared image quality score is highly consistent with the subjective perception of the human eyes. After the model training is completed, in the whole image quality evaluation process, only the objective sub-dimension evaluation score of the infrared image to be evaluated needs to be calculated, and the infrared image quality evaluation result highly consistent with the human eye subjective perception can be obtained, so that the infrared image quality evaluation highly consistent with the human eye subjective perception can be efficiently and conveniently realized, the operability is high, and the practicability is better.
In the above embodiment, how to execute step S101 is not limited, and a generation manner of the training set and the test set is provided in this embodiment, that is, an implementation process of generating the training set and the test set of the subjective score of image quality carried by the image based on the original infrared image data set may include the following steps:
a1: acquiring infrared images acquired by a plurality of infrared thermal imaging devices in various types of application scenes under different illumination environments to form an original infrared image data set; the infrared thermal imaging devices are different in manufacturers, packaging modes and resolutions.
The step is to describe the selection and the construction of the infrared test scene, and because a complete quality evaluation database is not provided for the infrared image at present, an infrared image database can be established for evaluating the quality of the infrared image. On the basis of fully learning and understanding infrared imaging principles and characteristics, 25 groups of indoor scenes and 25 groups of outdoor scenes can be selected and constructed, wherein the indoor scenes comprise the following typical scenes: electric light, computer, display, drinking cup, power, air conditioner, table chair, green planting, personage etc. outdoor typical scene: sky, ground, and natural scene objects of various temperatures such as pedestrians, vehicles, trees, buildings, etc. Considering that different detectors have great influence on imaging effect, in order to improve robustness of different detectors, the infrared imaging device for acquiring infrared image data can select a plurality of uncooled infrared thermal imaging devices which are produced by different manufacturers, have different packaging modes and different resolutions, and the packaging mode can be ceramic packaging or metal packaging. By the steps, a plurality of groups of indoor and outdoor scenes under different weather and different time periods can be constructed, and the total amount of data covering different distortion conditions such as noise, motion blur, defocus, non-uniformity and the like is about 1000+ pieces.
A2: and acquiring the image quality subjective score of each infrared image in the original infrared image data set.
And after a typical infrared test scene is set up in the last step, and an infrared image data set is constructed by using image data collected by an infrared thermal imaging device, subjective scoring can be carried out on the infrared image data set, wherein the subjective scoring comprises but is not limited to expert sub-dimension scoring aiming at uniformity, noise, definition, contrast and dynamic range and expert and common user combined total scoring aiming at the whole image, and a subjective scoring result of the image is obtained.
A3: and dividing each infrared image in the original infrared image data set into a plurality of training images and a plurality of testing images based on a preset division ratio.
A4: and generating a training set according to the plurality of training images and the image quality subjective scores corresponding to the training images.
A5: and generating a test set according to the multiple test images and the image quality subjective scores corresponding to the test images.
The preset division ratio of the steps can be flexibly selected according to actual requirements, for example, the preset division ratio can be selected according to 8: and 2, dividing the original infrared image data set into a training set and a testing set. For example, 1000 pieces of image data including various distortions of different degrees acquired in the above steps may be processed according to 8: and 2, dividing the training set and the test set according to the proportion, wherein no intersection exists between the image data of the training set and the image data of the test set.
The above embodiment of subjective scoring on an infrared image data set is not limited in any way, and this application also provides an alternative embodiment, which may include the following:
acquiring subjective evaluation scores of a plurality of experts on each subjective quality index of each infrared image in an original infrared image data set to obtain subjective sub-dimension evaluation scores of each infrared image;
acquiring subjective evaluation scores of a plurality of experts and a plurality of common users on the whole infrared images in the original infrared image data set, and calculating the subjective overall evaluation scores of the infrared images according to preset expert weight coefficients and user weight coefficients;
and determining the image quality subjective score of each infrared image in the original infrared image data set according to the subjective sub-dimension evaluation score and the subjective overall evaluation score of each infrared image.
In this embodiment, the subjective scores of the infrared images of the raw infrared image dataset include expert sub-dimension scores and expert and general user joint weighted total scores. The method can combine the human eye visual perception characteristic and the infrared imaging characteristic to select several items which affect the infrared image quality, and comprises the following steps: uniformity, noise, sharpness/blur, contrast, dynamic brightness range as subjective evaluation dimensions. Correspondingly, the subjective sub-dimension evaluation score may include a non-uniformity subjective score, an image noise subjective score, an image sharpness subjective score, an image contrast subjective score, and a dynamic brightness range subjective score, that is, the subjective quality index of the above embodiment. As shown in fig. 2, for each training image of the training set, the subjective score sub-dimension of each training image may include a non-uniformity subjective score, an image noise subjective score, an image sharpness subjective score, an image contrast subjective score, and a dynamic luminance range subjective score. One sub-dimension is a subjective quality index, the subjective sub-dimension evaluation score can be a data set or a matrix, each element is a subjective quality score corresponding to each subjective quality index, the subjective sub-dimension evaluation score can also be a score, the score is obtained by adding the subjective quality scores corresponding to the subjective quality indexes and can also be obtained by weighting and summing the subjective quality scores corresponding to the subjective quality indexes, the weighting factor of each objective quality index can be set in advance according to the actual application scene, and each test image corresponds to one group of subjective sub-dimension evaluation scores. The subjective evaluation and evaluation of each infrared image in the original infrared image data set, that is, the subjective evaluation score of each subjective quality index can be determined by adopting a single-stimulus continuous quality grading method, wherein an observer only observes the current to-be-observed infrared image in a certain continuous timeAnd measuring the image, continuously grading the image to be measured according to the grading table, and obtaining the quality evaluation of the image to be measured according to the grading and the grading time. The subjective evaluation score of the subjective quality index adopts expert scoring, the professional knowledge of the expert in the infrared image processing and evaluation aspect is fully utilized to score each sub-dimension of the images in the data set or the performance of each subjective quality index, in the method, 24 experts are selected as scorers, and the scoring results of all the scorers are subjected to mean normalization processing, so that a subjective score vector table of all the images in the data set in each sub-dimension, such as a subjective score vector table
The process of performing overall subjective evaluation on the infrared image of the original infrared image data set may be as follows: the overall subjective evaluation score comprises an expert score and a common user score, for example, 24 experts and 50 common users can be selected, after scoring results of all observers are obtained, weighted mean normalization processing is carried out, the expert scoring weight can be 0.6, the common user scoring weight can be 0.4, and therefore a subjective total score table { z ] of all images in the data set is obtained1,z2,z3...zn}. The subjective sub-dimension score result vector and the subjective total score value of each infrared image in the final original infrared image data set, namely the image quality subjective score of A2 can be expressed as
Therefore, the embodiment aims at the situation that no public image data set is available in the current long-wave infrared image evaluation field, and the embodiment constructs an infrared image data set with subjective evaluation on the basis of fully learning and understanding the infrared imaging principle and characteristics, so that the blank that no professional evaluation data set is available in the infrared imaging field is filled. In the embodiment, subjective scores of the infrared image data set are divided into sub-dimension scores and total scores, the sub-dimension scores are graded by using experts, and professional knowledge of the experts in the aspects of infrared image processing and evaluation is fully utilized to grade the performance of each sub-dimension of the images in the data set. The total scoring adopts expert scoring and common user combined weighted scoring, so that the cognitive perception of the public on the image is effectively combined by utilizing expert experience knowledge, and the subjective scoring result of the infrared image is comprehensive, delicate, professional and reliable.
Based on the above embodiment, the present application further provides an alternative implementation manner for S102, that is, the implementation process of training the machine learning model by using the training set to obtain the initial quality evaluation model may include:
a support vector regression model is constructed in advance, a grid optimization algorithm packaged by an LIBSVM tool kit is used for calculating a hyper-parameter gamma, and a hyper-parameter epsilon and a hyper-parameter C input by a user are obtained at the same time;
constructing a score feature vector according to the subjective sub-dimension evaluation score of each training image;
and inputting the score feature vectors and the subjective overall evaluation scores corresponding to the training images into a support vector regression model for training to obtain an initial quality evaluation model.
In this embodiment, the machine learning model adopts an SVR (Support Vector Regression) model, and the SVR can map a nonlinear indivisible feature in a low dimension to a linear separable feature in a high dimension by using a kernel function, so as to solve the problem of small sample training. In the embodiment, mapping from the feature vector to the quality score is completed by using an epsilon-insensitive nonlinear regression epsilon-SVR, and the specific implementation can refer to an LIBSVM package, wherein hyper-parameters needing to be manually set are C and epsilon, and gamma can be obtained by using a grid optimization algorithm packaged in the package.
In the above embodiment, how to execute step S103 is not limited, and an optional calculation manner of the evaluation score of the objective sub-dimension in this embodiment is shown in fig. 2, that is, an implementation process of determining the evaluation score of the objective sub-dimension of each test image by calculating the objective score of each objective quality indicator of each test image in the test set by using a non-reference evaluation algorithm may include the following steps:
b1: and calculating the image space variance of each test image and carrying out normalization processing to obtain the non-uniformity objective score.
In the present embodiment, the infrared image non-uniformity may include fixed pattern noise, dark signal non-uniformity, and photo response non-uniformity; the fixed pattern noise means that characteristic parameters for different pixels in a pixel array are different; dark signal nonuniformity is called because dark signals of different pixels are different; the different sensitivity of different image elements is called photo-response nonuniformity. Variance is a measure of the degree of dispersion of a set of data in probability theory and statistics, and can be described by variance for all types of non-uniformity. The larger the variance, the more uneven the image; the smaller the variance, the better the uniformity of the image. Therefore, for the image of the test set, the spatial variance of the image can be calculated and the result of the normalization processing can be used as the objective value of the nonuniformity of the image. For an infrared image with M rows and N columns, the non-uniformity calculation formula may be:
wherein rho is a self-defined parameter value and is used for representing the nonuniformity of the infrared image, yijIs the pixel value at pixel element (i, j).
B2: and determining the objective fraction of the image noise by calculating the local normalized brightness coefficient of each test image.
It is understood that the noise of the infrared image may include thermal noise, shot noise, 1/f noise, fixed pattern noise, and streak noise. The thermal noise and shot noise can be regarded as white noise, the intensity of the 1/f noise is inversely proportional to the frequency, the 1/f noise is fractal noise, the fixed pattern noise and the stripe noise belong to non-uniform noise, and the non-uniform factors are already processed in the step B1, so that the step aims at the thermal noise, the shot noise and the 1/f noise. In the BRISQUE algorithm, MSCN (Mean filtered Contrast Normalized, local Normalized luminance of an image) coefficients have statistical properties sensitive to various degradations, and the visual quality affecting image distortion can be predicted by quantifying changes in the statistical properties.
Optionally, an alternative calculation method of the objective noise score of the image may include:
for each test image, calculating a local normalized brightness coefficient of the current test image, and fitting the local normalized brightness coefficient through a generalized Gaussian model to obtain a fitting parameter mean value and a fitting parameter variance; calculating local normalized brightness coefficient neighborhood coefficients of the local normalized brightness coefficients in multiple directions, and fitting each local normalized brightness coefficient neighborhood coefficient through an asymmetric generalized Gaussian model to obtain multiple fitting parameters; and extracting multidimensional statistical characteristics from the fitting parameter mean value, the fitting parameter variance and each fitting parameter in different scales, and inputting the multidimensional statistical characteristics to the initial quality evaluation model to obtain the image noise objective score of the current test image.
In this embodiment, the MSCN coefficient histogram exhibits gaussian distribution characteristics for natural images, and the histogram is smoother for noisy images. MSCN coefficient
The calculation is as follows:
in the formula, c is a constant, the prevention denominator is 0, I belongs to 1,2.. M, j belongs to 1,2.. N and represents the spatial position of the pixel, I (I, j) represents the intensity of the central pixel, μ (I, j) represents the mean value of the current local area, and σ (I, j) represents the variance of the current local area. W ═ wk,lI | (K) — K. -, K; l ═ L., L } is a two-dimensional circularly symmetric gaussian weighting function, and K ═ L ═ 3 denotes the image patch size.
Utilizing the existing Generalized Gaussian model (Generalized Gaussian Distri)butyl, GGD) can be fitted with the normalized brightness information calculated above to obtain fitting parameter mean z and variance phi2. In addition, in order to reflect the statistical characteristics of adjacent coefficients, a horizontal H, a vertical V and a main diagonal D are constructed on the basis of MSCN1Minor diagonal D2MSCN neighborhood coefficients for four directions:
the Asymmetric Generalized Gaussian Distribution model (AGGD) in the prior art can be adopted to fit the neighborhood MSCN coefficient in four directions to obtain fitting parameters
Considering that human vision has multi-scale property, features are respectively extracted on an original scale and a 2-time down-sampling scale, 36 statistical features can be extracted, and then mapping from the 36 features to image quality scores is established through an initial quality evaluation model.
B3: and determining the objective image definition score by calculating the structural similarity between each test image and the corresponding reference image.
It is understood that the sharpness of the image may be expressed by using the structural similarity between the target image x and the reference image y, which may include a brightness comparison function, a contrast comparison function, and a structural information comparison function.
Wherein, the brightness comparison function can be expressed as:
the contrast comparison function can be expressed as:
the structure information comparison function can be expressed as:
in the formula, C1、C2And C3Is constant, avoids denominator being 0, muxIs the mean value of the gray levels of the image x, muyIs the mean value of the gray levels, σ, of the image yxIs the gray scale standard deviation, σ, of the image xyIs the gray scale standard deviation, σ, of the image yxyIs the covariance of the images x, y.
Based on the above, the structural similarity of the images of the present embodiment can be represented by the relational expression SSIM (x, y) ═ l (x, y)]α·[c(x,y)]β·[s(x,y)]γThe weights α, β, and γ are calculated to control the luminance, contrast, and structural information, respectively, and α ═ β ═ γ ═ 1, for example. After the structural similarity of each infrared image is determined, the greater the structural similarity is, the clearer the image to be detected is proved, and the objective image definition score of the image to be detected can be calculated based on the structural similarity.
B4: an objective score for image contrast is calculated for each test image based on the image spatial frequency.
On the basis of analyzing the contrast sensitivity characteristic of the human visual system, a contrast sensitivity function can be defined as:
wherein f is the spatial frequency of the image,
fR，fCthe spatial frequencies in the horizontal and vertical directions, respectively.
Where M is the number of rows of the image and N is the number of columns of the image.
Because the spatial frequency of the image is directly related to the contrast sensitivity characteristic of a human eye vision system, an image contrast objective score calculation mode can be determined by combining the spatial frequency of the image with an actual application scene, and the image contrast objective score of the image to be detected can be obtained through the calculation mode.
B5: and calculating the dynamic brightness range objective fraction of each test image based on the image maximum gray level and the image minimum gray level.
The dynamic brightness range of the thermal infrared imager refers to the adaptability to the temperature change of a scene in a shooting scene, and specifically refers to the change range of the infrared image brightness, namely the adjustment range representing the brightest and darkest in the image. For the test set image, a dynamic range calculation formula can be used to obtain the dynamic brightness range of the image and carry out normalization operation as the objective value of the dynamic brightness range. The dynamic range calculation formula can be expressed as:
in the formula, LmaxFor the maximum grey level of the picture obtained by statistics, LminIs the minimum gray level of the image.
B6: and for each test image, carrying out normalization processing on the non-uniformity objective score, the image noise objective score, the image definition objective score, the image contrast objective score and the dynamic brightness range objective score of the current test image to obtain the objective sub-dimension evaluation score of the current test image.
From the above, in this embodiment, based on that the subjective overall perception of the human eyes on the image is a comprehensive result of perception in each dimension, the total score of the human eyes on the infrared image quality is regarded as a mapping function of each sub-dimension score, the contribution of each sub-dimension score to the overall total score is different in size, that is, a sub-dimension evaluation score _ all (including an objective sub-dimension evaluation score and a subjective sub-dimension evaluation score) is function (score _ nuc (including an objective sub-dimension evaluation score and a subjective sub-dimension evaluation score), score _ noise (including an image noise objective score and an image noise subjective score), score _ blu (including an image sharpness objective score and an image sharpness subjective score), score _ contrast (an image contrast objective score and an image contrast subjective score), score _ dynamic (including a dynamic luminance range objective score and a dynamic luminance range subjective score)). And for the infrared image to be tested, selecting an objective evaluation method which has high consistency with human eye subjective perception and faces to specific distortion for objective evaluation to obtain objective scores of the infrared image to be tested in each sub-dimension, inputting each sub-dimension objective score vector into a trained initial quality evaluation model to obtain an objective total score of the infrared image to be tested, and effectively combining the objectivity of subjective scoring and the high efficiency of objective scoring.
The above embodiment does not set any limitation on how to perform the process of determining the objective image sharpness score by calculating the structural similarity between each test image and the corresponding reference image, and this embodiment also provides an optional implementation manner, which may include:
for each test image, carrying out low-pass filtering on the current test image by using a Gaussian smoothing filter to obtain a corresponding current reference image;
respectively extracting gradient information and edge information in a target direction of a current test image and a current reference image;
generating a test gradient image according to the gradient information and the edge information of the current test image; generating a reference gradient image according to the gradient information and the edge information of the current reference image;
determining a plurality of target image blocks meeting preset gradient information conditions from the test gradient image, and determining that each target image block corresponds to a target reference image block of the reference gradient image;
calling a pre-constructed image structure similarity relational expression to calculate the structure similarity between each target image block and the corresponding target reference image block; the image structure similarity relational expression is determined according to the brightness comparison function, the contrast comparison function, the structure information comparison function and respective weight coefficients;
and determining the image definition objective fraction of the current test image according to the structural similarity of each target image block.
In the present embodiment, the no-reference image sharpness evaluation index NRSS is introduced to calculate the image sharpness. First, a reference image is constructed for the image to be evaluated, the image to be evaluated is defined as I, and the size of the reference image can be 7 multiplied by 7, sigma2The Gaussian smoothing filter with the value of 6 carries out low-pass filtering on the image I to be evaluated to obtain a reference image I of the image I to be evaluatedrLpf (i). Extracting images I and IrUsing the characteristic that human eyes are most sensitive to edge information in the horizontal direction and the vertical direction, a Sobel operator can be used for respectively extracting the edge information in the horizontal direction and the edge information in the vertical direction, a gradient image is generated based on the edge information and the gradient information, and I are definedrRespectively G and Gr。
The determination process of the target image blocks meeting the preset gradient information condition may be: the preset gradient information of this embodiment may be that the gradient information is the most abundant, specifically, a gradient information threshold may be determined first, and the image block greater than the threshold is the gradient block with the most abundant gradient information. As an alternative embodiment, the process of selecting the N image blocks with most abundant gradient information from the gradient image G may be: the gradient image G is divided into a number of small image blocks by 8 x 8, and in order to avoid losing important edges, the inter-block step size of each image block is 4, i.e. there is 50% overlap of neighboring blocks. Calculating the variance of each image block, wherein the greater the variance is, the richer the gradient information is, and finding out the N block with the maximum variance, which is marked as { xi1,2., N }, corresponding to GrThe corresponding block in (2) is defined as yi1,2. Of course, other methods can be used to select the desired target image block from the gradient image, which does not affect the implementation of the present application. After determining the target image blocks, each target image block x may be computed firstiAnd its image block y in the reference imageiStructural similarity SSIM (x) of (1)i,yi)SSIM(xi,yi) The image structure similarity relationship SSIM (x, y) ═ l (x, y)]α·[c(x,y)]β·[s(x,y)]γThe structural similarity is calculated, and then the structural definition can be calculated based onThe NRSS calculates a relational expression to calculate and obtain an objective score of image definition, and the structural definition NRSS calculates the relational expression as follows:
therefore, in the process of calculating the objective score of the image definition, the structural similarity is calculated by selecting the representative image blocks from the test image and the reference image, so that the overall image quality evaluation efficiency can be improved, the calculation precision of the objective score of the image definition can be improved, and the improvement of the image quality evaluation accuracy is facilitated.
The above embodiment does not set any limitation on how to calculate the objective score of image contrast of each test image based on the image spatial frequency, and this embodiment also provides an alternative calculation method for the objective score of image contrast, which may include the following steps:
traversing the current test image according to the spatial frequency in the horizontal direction and the vertical direction by using a preset size template for each test image to obtain a spatial frequency matrix of the current test image;
based on the spatial frequency matrix, normalizing the image spatial frequency of each pixel point one by one to obtain a normalized image spatial frequency matrix;
determining a contrast sensitivity weight matrix of the current test image according to a pre-constructed contrast sensitivity relational expression and the normalized image space frequency matrix;
and calculating the objective score of the image contrast of the current test image according to the contrast sensitivity weight matrix, the image size of the current test image, the maximum gray level and the minimum gray level.
In this embodiment, a spatial frequency f in the horizontal direction of the 3 x 3 template may be usedRAnd spatial frequency f in the vertical directionCTraversing the image to obtain an image space frequency matrix f (i, j), normalizing the image space frequency one by one based on a normalization calculation relation, wherein the normalization calculation relation can be tabulatedShown as follows:
in the formula (f)minAnd fmaxObtaining a normalized image spatial frequency matrix f for the minimum and maximum values of the image spatial frequency f (i, j)monAfter (i, j), f can bemonAnd (i, j) substituting the contrast sensitivity function to obtain a contrast sensitivity weight matrix C (i, j) of the image. And calling a pre-constructed contrast objective score calculation relation to calculate the image contrast objective score of the test image based on the contrast sensitivity weight matrix. The objective score of contrast calculation relation can be expressed as:
wherein the content of the first and second substances,
Lmaxfor the maximum grey level of the picture obtained by statistics, LminThe minimum gray level of the image is obtained by respectively calculating the maximum pixel value and the minimum pixel value in the image. M is the number of rows of the current test image, N is the number of columns of the current test image, M is the mth row, and N is the nth column.
Therefore, the calculation accuracy of the objective contrast score of the image can be improved by reflecting the spatial frequency of the human eye sensitivity and combining the image gray scale information to calculate the objective contrast score, and the improvement of the image quality evaluation accuracy is facilitated.
The foregoing embodiment does not limit how to perform S105, and the present application also provides an implementation manner of determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model, which may include:
for each test image, specific values of performance measurement indexes of the image quality objective score and the image quality subjective score of the current test image are respectively calculated, the performance measurement indexes are any one or any combination of a Pearson linear correlation coefficient, a Spanish rank correlation coefficient, a Kendel rank correlation coefficient and a root mean square error, and correspondingly, the performance measurement indexes are the performance measurement indexes for respectively calculating the image quality objective score and the image quality subjective score of the current test image.
If the pearson linear correlation coefficient and/or the spearman rank correlation coefficient and/or the Kendall rank correlation coefficient and/or the root mean square error meet the preset performance condition, namely the performance measurement index meets the preset performance condition, taking the initial quality evaluation model as an image quality evaluation model;
and if the Pearson linear correlation coefficient and/or the spearman rank correlation coefficient and/or the Kendall rank correlation coefficient and/or the root mean square error do not meet the preset performance condition, namely the performance measurement index does not meet the preset performance condition, generating an instruction for optimizing the initial quality evaluation model, and training the initial quality evaluation model again until the preset performance condition is met.
The Pearson linear correlation coefficient PLCC of the image quality objective score and the image quality subjective score can be calculated through the Pearson linear correlation coefficient calculation relational expression, and the larger the Pearson linear correlation coefficient is, the higher the correlation between the output result of the model and the human eye score is, and the better the performance of the initial quality evaluation model is. The pearson linear correlation coefficient calculation relationship can be expressed as:
wherein x isiI ∈ {1, 2.. n } represents an array of image quality objective scores for the test set images, yiI ∈ {1, 2.. n } represents an image quality subjective score array for the images in the test set, n is the number of images in the test set,
and
are each { x1,x2,…,xnAnd { y }1,y2,…ynMean ofxAnd σyTheir standard deviations are given respectively.
In the implementation, the spearman rank correlation coefficient SROCC of the image quality objective score and the image quality subjective score can be calculated through a spearman rank correlation coefficient calculation relational expression, and the bigger the value of the spearman rank correlation coefficient is, the higher the correlation between the model output result and the human eye score is, which indicates that the performance of the initial quality evaluation model is better. The spearman rank correlation coefficient calculation relation can be expressed as:
wherein n is the number of images in the test set, and the logarithm group xiAnd array yiRespectively sorting the values in the group from small to large to obtain rxiAnd ryiRespectively, the order of the ith value in the array, rxi-ryiThe rank difference of the two sets of data.
In the present embodiment, the kender rank correlation coefficient KROCC of the image quality objective score and the image quality subjective score can be calculated by the kender rank correlation coefficient calculation relational expression. The larger the Kendall rank correlation coefficient is, the higher the correlation between the output result of the model and the human eye score is, and the better the performance of the initial quality evaluation model is. Array x for objective total scores for test set imagesiI ∈ {1, 2.. n }, and an array y of subjective scores for human eyes of the images of the test setiI ∈ {1, 2.. n }, defining the data pairs in both arrays to be consistent (x)i>xj,yi>yjOr xi<xj,yi<yj) There are P data pairs, and the data pairs in the two arrays are not consistent (x)i>xj,yi<yjOr xi<xj,yi>yj) Has Q related coefficients of Kendel rankShown as follows:
wherein n is the number of images in the test set.
In this embodiment, the root mean square error of the objective score of image quality and the subjective score of image quality may be calculated by a root mean square error calculation relational expression. The smaller the root mean square error is, the higher the correlation between the model output result and the human eye score is, indicating that the initial quality evaluation model has better performance. The root mean square error calculation relationship may be expressed as:
wherein x isiI ∈ {1, 2.. n } denotes an objective total score array for the test set image, yiAnd i belongs to {1,2,. n } represents an array of subjective scores of human eyes for the images in the test set, and n is the number of the images in the test set.
For example, table 1 shows the values of pearson linear correlation coefficient, spearman rank correlation coefficient, kender rank correlation coefficient, and root mean square error in an illustrative example.
TABLE 1 specific values of the Performance metrics
| Index (I) | PLCC | SROCC | KROCC | RMSE |
| Consistency | 0.9302 | 0.9123 | 0.9541 | 2.6517 |
In this embodiment, the preset performance condition refers to the precision of the quality evaluation model determined by a person skilled in the art according to an actual application scenario, the specification of the preset performance condition is related to a combination of adopted performance measurement indexes, namely, a pearson linear correlation coefficient, a spearman rank correlation coefficient, a kender rank correlation coefficient and a root mean square error, if the performance measurement indexes are the root mean square error, the preset performance condition may be that the root mean square error value is less than 2.7, if the root mean square error value calculated according to the above manner is greater than 2.7, the performance of the initial quality evaluation model is not up to the standard and needs to be optimized, and if the root mean square error value calculated according to the above manner is less than 2.7, the initial quality evaluation model meets the preset performance condition. If the performance measurement indexes are root mean square error and pearson linear correlation coefficient, the preset performance condition may be that the root mean square error value is smaller than 2.7 and the pearson linear correlation coefficient is larger than 0.9, if the root mean square error value calculated in the above manner is larger than 2.7 and/or the pearson linear correlation coefficient is smaller than 0.9, the performance of the initial quality evaluation model is not up to standard and needs to be optimized, and if the root mean square error value calculated in the above manner is smaller than 2.7 and the pearson linear correlation coefficient is larger than 0.9, the initial quality evaluation model meets the preset performance condition.
As can be seen from the above, in the embodiment, by comparing the Spearman order correlation coefficient SROCC, Pearson linear correlation coefficient PLCC, Kendall order correlation coefficient KROCC, and root mean square error RMSE of the subjective total score and the objective total score of the test set image, the evaluation of the consistency between the objective evaluation aspect of infrared image quality and the subjective perception of human eyes is realized, and the practicability is strong.
It can be understood that, the above embodiment describes how to obtain a model for evaluating the actual infrared image quality, so that the model outputs an image quality evaluation result highly consistent with the subjective perception of human eyes, thereby implementing the evaluation of the image quality of the infrared image, based on this, the present application also provides another embodiment, which is used for implementing the image quality evaluation of any infrared image with unknown image quality subjective score, please refer to fig. 3, and the following contents can be included:
s301: an image quality evaluation model is obtained by utilizing the steps of any one of the above infrared image quality evaluation method embodiments in advance.
The image quality evaluation model in this step is the final image quality evaluation model obtained in S105 of the above-described embodiment.
S302: and acquiring an infrared image to be evaluated.
The infrared image to be evaluated is any infrared image needing image quality evaluation.
S303: and calculating the objective score of each objective quality index of the infrared image to be evaluated by using a non-reference evaluation algorithm, and determining the objective sub-dimension evaluation score of the infrared image to be evaluated.
S304: and inputting the objective sub-dimension evaluation score into an image quality evaluation model to obtain an image quality evaluation score of the infrared image to be evaluated.
The same methods or steps as those in the above embodiments can refer to the contents described in the above embodiments, and are not described herein again.
As can be seen from the above, the non-specific distortion type-oriented non-reference infrared image quality evaluation method is constructed in the embodiment, infrared image quality evaluation results of infrared images of different infrared detection devices and different scenes can be obtained by the method, and the result is highly consistent with the subjective perception of human eyes.
It should be noted that, in the present application, there is no strict sequential execution order among the steps, and as long as a logical order is met, the steps may be executed simultaneously or according to a certain preset order, and fig. 1 to fig. 3 are only schematic manners, and do not represent only such an execution order.
The embodiment of the invention also provides a corresponding device for the infrared image quality evaluation method, so that the method has higher practicability. Wherein the means can be described separately from the functional module point of view and the hardware point of view. In the following, the infrared image quality evaluation apparatus provided by the embodiment of the present invention is introduced, and the infrared image quality evaluation apparatus described below and the infrared image quality evaluation method described above may be referred to in correspondence with each other.
Based on the angle of the functional module, referring to fig. 4, fig. 4 is a structural diagram of an infrared image quality evaluation apparatus according to an embodiment of the present invention, in a specific implementation manner, the apparatus may include:
a data set construction module 401, configured to generate a training set and a test set of image-quality subjective scores based on an original infrared image data set;
a model training module 402, configured to train a machine learning model by using each training image in the training set and a corresponding image quality subjective score to obtain an initial quality evaluation model;
a sub-dimension score calculating module 403, configured to calculate an objective score of each objective quality indicator of each test image in the test set by using a non-reference evaluation algorithm, and determine an objective sub-dimension evaluation score of each test image;
the objective evaluation module 404 is used for inputting the evaluation scores of the objective sub-dimensions into the initial quality evaluation model to obtain the image quality objective score of each test image;
and a model determining module 405, configured to determine a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
Optionally, in some embodiments of this embodiment, the data set constructing module 401 may be configured to: acquiring infrared images acquired by a plurality of infrared thermal imaging devices in various types of application scenes under different illumination environments to form an original infrared image data set; the infrared thermal imaging equipment is different in manufacturer, packaging mode and resolution; acquiring image quality subjective scores of infrared images in an original infrared image data set; dividing each infrared image in the original infrared image data set into a plurality of training images and a plurality of testing images based on a preset division ratio; generating a training set according to the plurality of training images and the image quality subjective scores corresponding to the training images; and generating a test set according to the multiple test images and the image quality subjective scores corresponding to the test images.
As an optional implementation manner of the foregoing embodiment, the data set constructing module 401 may further be configured to: acquiring subjective evaluation scores of a plurality of experts on each subjective quality index of each infrared image to obtain subjective sub-dimension evaluation scores of each infrared image; acquiring subjective evaluation scores of a plurality of experts and a plurality of common users on the whole infrared images, and calculating the subjective overall evaluation scores of the infrared images according to preset expert weight coefficients and user weight coefficients; and determining the image quality subjective score of each infrared image according to the subjective sub-dimension evaluation score and the subjective overall evaluation score of each infrared image.
As another optional implementation manner of the foregoing embodiment, the model training module 402 may be further configured to: a support vector regression model is constructed in advance; constructing a score feature vector according to the subjective sub-dimension evaluation score of each training image; and inputting the score feature vectors and the subjective overall evaluation scores corresponding to the training images into a support vector regression model for training to obtain an initial quality evaluation model.
Optionally, in other embodiments of this embodiment, the sub-dimension score calculating module 403 may include:
the uniformity calculation unit is used for calculating the image space variance of each test image and carrying out normalization processing to obtain a non-uniformity objective score;
the noise calculation unit is used for determining the objective fraction of the image noise by calculating the local normalized brightness coefficient of each test image;
the definition calculating unit is used for determining the objective image definition scores by calculating the structural similarity between each test image and the corresponding reference image;
the contrast calculation unit is used for calculating the objective scores of the image contrast of the test images based on the image spatial frequency;
the dynamic range calculating unit is used for calculating the objective scores of the dynamic brightness ranges of the test images based on the maximum gray level and the minimum gray level of the images;
and the overall sub-dimension score calculating unit is used for normalizing the objective non-uniformity score, the objective image noise score, the objective image definition score, the objective image contrast score and the objective dynamic brightness range score of the current test image for each test image to obtain the objective sub-dimension evaluation score of the current test image.
As an optional implementation manner of the foregoing embodiment, the foregoing noise calculating unit may be further configured to: for each test image, calculating a local normalized brightness coefficient of the current test image, and fitting the local normalized brightness coefficient through a generalized Gaussian model to obtain a fitting parameter mean value and a fitting parameter variance; calculating local normalized brightness coefficient neighborhood coefficients of the local normalized brightness coefficients in multiple directions, and fitting each local normalized brightness coefficient neighborhood coefficient through an asymmetric generalized Gaussian model to obtain multiple fitting parameters; and extracting multidimensional statistical characteristics from the fitting parameter mean value, the fitting parameter variance and each fitting parameter in different scales, and inputting the multidimensional statistical characteristics to the initial quality evaluation model to obtain the image noise objective score of the current test image.
As another optional implementation manner of the foregoing embodiment, the above-mentioned clarity calculating unit may be further configured to: for each test image, carrying out low-pass filtering on the current test image by using a Gaussian smoothing filter to obtain a corresponding current reference image; respectively extracting gradient information and edge information in a target direction of a current test image and a current reference image; generating a test gradient image according to the gradient information and the edge information of the current test image; generating a reference gradient image according to the gradient information and the edge information of the current reference image; determining a plurality of target image blocks meeting preset gradient information conditions from the test gradient image, and determining that each target image block corresponds to a target reference image block of the reference gradient image; calling a pre-constructed image structure similarity relational expression to calculate the structure similarity between each target image block and the corresponding target reference image block; the image structure similarity relational expression is determined according to the brightness comparison function, the contrast comparison function, the structure information comparison function and respective weight coefficients; and determining the image definition objective fraction of the current test image according to the structural similarity of each target image block.
As still another alternative implementation manner of the foregoing embodiment, the contrast calculating unit may be further configured to: traversing the current test image according to the spatial frequency in the horizontal direction and the vertical direction by using a preset size template for each test image to obtain a spatial frequency matrix of the current test image; based on the spatial frequency matrix, normalizing the image spatial frequency of each pixel point one by one to obtain a normalized image spatial frequency matrix; determining a contrast sensitivity weight matrix of the current test image according to a pre-constructed contrast sensitivity relational expression and the normalized image space frequency matrix; and calculating the objective score of the image contrast of the current test image according to the contrast sensitivity weight matrix, the image size of the current test image, the maximum gray level and the minimum gray level.
Optionally, in still other embodiments of this embodiment, the model determining module 405 may be further configured to: respectively calculating the performance measurement indexes of the image quality objective score and the image quality subjective score of the current test image for each test image; the performance measurement indexes comprise one or more of Pearson linear correlation coefficient, spearman rank correlation coefficient, Kendall rank correlation coefficient and root mean square error; if the performance measurement index meets the preset performance condition, taking the initial quality evaluation model as an image quality evaluation model; and if the performance measurement index does not meet the preset performance condition, generating an instruction for optimizing the initial quality evaluation model, and training the initial quality evaluation model again until the preset performance condition is met.
Referring to fig. 5 based on the angle of the functional module, fig. 5 is a structural diagram of an infrared image quality evaluation apparatus according to another embodiment of the present invention, where the apparatus may include:
the model construction module 501 is used for obtaining an image quality evaluation model by using any one of the above infrared image quality evaluation methods in advance;
an image acquisition module 502, configured to acquire an infrared image to be evaluated;
the objective scoring module 503 is configured to calculate an objective score of each objective quality index of the infrared image to be evaluated by using a non-reference evaluation algorithm, and determine an objective sub-dimension evaluation score of the infrared image to be evaluated;
the quality evaluation module 504 is configured to input the objective sub-dimension evaluation score to the image quality evaluation model, so as to obtain an image quality evaluation score of the infrared image to be evaluated.
The functions of the functional modules of the infrared image quality evaluation device in the embodiment of the present invention may be specifically implemented according to the method in the above method embodiment, and the specific implementation process may refer to the related description of the above method embodiment, which is not described herein again.
Therefore, the embodiment of the invention can efficiently and conveniently realize the infrared image quality evaluation which is highly consistent with the human eye subjective perception.
The above-mentioned infrared image quality evaluation apparatus is described from the perspective of a functional module, and further, the present application also provides an electronic device described from the perspective of hardware. Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 6, the electronic device includes a memory 60 for storing a computer program; a processor 61, configured to execute a computer program to implement the steps of the infrared image quality evaluation method according to any one of the above embodiments.
The processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 61 may also be a controller, a microcontroller, a microprocessor or other data processing chip, and the like. The processor 61 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 61 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
Memory 60 may include one or more computer-readable storage media, which may be non-transitory. Memory 60 may also include high speed random access memory as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. The memory 60 may in some embodiments be an internal storage unit of the electronic device, for example a hard disk of a server. The memory 60 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk provided on a server, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. Further, the memory 60 may also include both internal storage units of the electronic device and external storage devices. The memory 60 may be used for storing various data and application software installed in the electronic device, such as: the code of the program that executes the vulnerability handling method, etc. may also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 60 is at least used for storing a computer program 601, wherein the computer program is loaded and executed by the processor 61, and then the relevant steps of the infrared image quality evaluation method disclosed in any one of the foregoing embodiments can be implemented. In addition, the resources stored by the memory 60 may also include an operating system 602, data 603, and the like, and the storage may be transient storage or permanent storage. Operating system 602 may include Windows, Unix, Linux, etc., among others. The data 603 may include, but is not limited to, data corresponding to the infrared image quality evaluation result, and the like.
In some embodiments, the electronic device may further include a display 62, an input/output interface 63, a communication interface 64, otherwise known as a network interface, a power supply 65, and a communication bus 66. The display 62 and the input/output interface 63, such as a Keyboard (Keyboard), belong to a user interface, and the optional user interface may also include a standard wired interface, a wireless interface, and the like. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, as appropriate, is used for displaying information processed in the electronic device and for displaying a visualized user interface. The communication interface 64 may optionally include a wired interface and/or a wireless interface, such as a WI-FI interface, a bluetooth interface, etc., typically used to establish a communication link between an electronic device and other electronic devices. The communication bus 66 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 6, but this is not intended to represent only one bus or type of bus.
Those skilled in the art will appreciate that the configuration shown in fig. 6 is not intended to be limiting of the electronic device and may include more or fewer components than those shown, such as a sensor 67 that performs various functions.
The functions of the functional modules of the electronic device according to the embodiments of the present invention may be specifically implemented according to the method in the above method embodiments, and the specific implementation process may refer to the description related to the above method embodiments, which is not described herein again.
Therefore, the embodiment of the invention can efficiently and conveniently realize the infrared image quality evaluation which is highly consistent with the human eye subjective perception.
It is to be understood that, if the infrared image quality evaluation method in the above-described embodiment is implemented in the form of a software functional unit and sold or used as a separate product, it may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application may be substantially or partially implemented in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods of the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a multimedia card, a card type Memory (e.g., SD or DX Memory, etc.), a magnetic Memory, a removable magnetic disk, a CD-ROM, a magnetic or optical disk, and other various media capable of storing program codes.
Based on this, the embodiment of the present invention further provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor, and the steps of the infrared image quality evaluation method according to any one of the above embodiments are provided.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. For hardware including devices and electronic equipment disclosed by the embodiment, the description is relatively simple because the hardware includes the devices and the electronic equipment correspond to the method disclosed by the embodiment, and the relevant points can be obtained by referring to the description of the method.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The method, the device, the electronic device and the readable storage medium for evaluating the quality of the infrared image provided by the application are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present application.
Claims (14)
1. An infrared image quality evaluation method is characterized by comprising the following steps:
generating a training set and a testing set of image-carried subjective image quality scores based on an original infrared image data set;
training a machine learning model by utilizing each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model;
calculating the objective score of each objective quality index of each test image in the test set by using a non-reference evaluation algorithm, and determining the objective sub-dimension evaluation score of each test image;
inputting each objective sub-dimension evaluation score into the initial quality evaluation model to obtain an image quality objective score of each test image;
and determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
2. The infrared image quality evaluation method of claim 1, wherein the generating of the training set and the test set of image-bearing image quality subjective scores based on the original infrared image dataset comprises:
acquiring infrared images acquired by a plurality of infrared thermal imaging devices in various types of application scenes under different illumination environments to form an original infrared image data set; the infrared thermal imaging equipment is different in manufacturer, packaging mode and resolution;
acquiring image quality subjective scores of all infrared images in the original infrared image data set;
dividing each infrared image in the original infrared image data set into a plurality of training images and a plurality of testing images based on a preset division ratio;
generating a training set according to the plurality of training images and the image quality subjective scores corresponding to the training images;
and generating a test set according to the multiple test images and the image quality subjective scores corresponding to the test images.
3. The infrared image quality evaluation method of claim 2, wherein the obtaining of the image quality subjective score of each infrared image in the original infrared image dataset comprises:
acquiring subjective evaluation scores of a plurality of experts on each subjective quality index of each infrared image to obtain subjective sub-dimension evaluation scores of each infrared image;
acquiring subjective evaluation scores of a plurality of experts and a plurality of common users on the whole infrared images, and calculating the subjective overall evaluation scores of the infrared images according to preset expert weight coefficients and user weight coefficients;
and determining the image quality subjective score of each infrared image according to the subjective sub-dimension evaluation score and the subjective overall evaluation score of each infrared image.
4. The infrared image quality evaluation method of claim 3, wherein the training of the machine learning model by using the training images in the training set and the corresponding image quality subjective scores to obtain an initial quality evaluation model comprises:
a support vector regression model is constructed in advance;
constructing a score feature vector according to the subjective sub-dimension evaluation score of each training image;
and inputting the score feature vector and the subjective overall evaluation score corresponding to each training image into the support vector regression model for training to obtain the initial quality evaluation model.
5. The infrared image quality evaluation method according to any one of claims 1 to 4, wherein the determining the objective sub-dimension evaluation score of each test image by calculating the objective score of each objective quality index of each test image in the test set by using a non-reference evaluation algorithm comprises:
calculating the image space variance of each test image and carrying out normalization processing to obtain a non-uniformity objective score;
determining an objective score of image noise by calculating a local normalized brightness coefficient of each test image;
determining an objective image definition score by calculating the structural similarity between each test image and a corresponding reference image;
calculating an objective score of image contrast of each test image based on the image spatial frequency;
calculating the objective fraction of the dynamic brightness range of each test image based on the maximum gray level and the minimum gray level of the image;
and for each test image, carrying out normalization processing on the non-uniformity objective score, the image noise objective score, the image definition objective score, the image contrast objective score and the dynamic brightness range objective score of the current test image to obtain the objective sub-dimension evaluation score of the current test image.
6. The infrared image quality evaluation method of claim 5, wherein the determining an image noise objective score by calculating a local normalized luminance coefficient of each test image comprises:
calculating a local normalized brightness coefficient of the current test image for each test image, and fitting the local normalized brightness coefficient through a generalized Gaussian model to obtain a fitting parameter mean value and a fitting parameter variance;
calculating local normalized brightness coefficient neighborhood coefficients of the local normalized brightness coefficients in multiple directions, and fitting the local normalized brightness coefficient neighborhood coefficients through an asymmetric generalized Gaussian model to obtain multiple fitting parameters;
and extracting multidimensional statistical characteristics from the fitting parameter mean, the fitting parameter variance and each fitting parameter in different scales, and inputting the multidimensional statistical characteristics to the initial quality evaluation model to obtain the objective image noise score of the current test image.
7. The infrared image quality evaluation method of claim 5, wherein the determining an objective score of image sharpness by calculating structural similarity between each test image and a corresponding reference image comprises:
for each test image, carrying out low-pass filtering on the current test image by using a Gaussian smoothing filter to obtain a corresponding current reference image;
respectively extracting gradient information and edge information in a target direction of the current test image and the current reference image;
generating a test gradient image according to the gradient information and the edge information of the current test image; generating a reference gradient image according to the gradient information and the edge information of the current reference image;
determining a plurality of target image blocks meeting preset gradient information conditions from the test gradient image, and determining that each target image block corresponds to a target reference image block of the reference gradient image;
calling a pre-constructed image structure similarity relational expression to calculate the structure similarity between each target image block and the corresponding target reference image block; the image structure similarity relational expression is determined according to a brightness comparison function, a contrast comparison function, a structure information comparison function and respective weight coefficients;
and determining the image definition objective fraction of the current test image according to the structural similarity of each target image block.
8. The infrared image quality evaluation method of claim 5, wherein the calculating an objective score for image contrast for each test image based on image spatial frequency comprises:
traversing the current test image according to the spatial frequency in the horizontal direction and the vertical direction by using a preset size template for each test image to obtain a spatial frequency matrix of the current test image;
based on the spatial frequency matrix, normalizing the image spatial frequency of each pixel point one by one to obtain a normalized image spatial frequency matrix;
determining a contrast sensitivity weight matrix of the current test image according to a pre-constructed contrast sensitivity relational expression and the normalized image space frequency matrix;
and calculating the objective score of the image contrast of the current test image according to the contrast sensitivity weight matrix, the image size of the current test image, the maximum gray level and the minimum gray level.
9. The infrared image quality evaluation method according to any one of claims 1 to 4, wherein the determining a final image quality evaluation model based on the initial quality evaluation model from the image quality objective scores and the corresponding image quality subjective scores of the respective test images comprises:
respectively calculating the performance measurement indexes of the image quality objective score and the image quality subjective score of the current test image for each test image; wherein the performance measure comprises one or more of a Pearson linear correlation coefficient, a Spanish rank correlation coefficient, a Kendel rank correlation coefficient, and a root mean square error;
if the performance measurement index meets a preset performance condition, taking the initial quality evaluation model as the image quality evaluation model;
and if the performance measurement index does not meet the preset performance condition, generating an instruction for optimizing the initial quality evaluation model, and training the initial quality evaluation model again until the preset performance condition is met.
10. An infrared image quality evaluation method is characterized by comprising the following steps:
obtaining an image quality evaluation model in advance by using the infrared image quality evaluation method according to any one of claims 1 to 9;
acquiring an infrared image to be evaluated;
calculating an objective score of each objective quality index of the infrared image to be evaluated by using a non-reference evaluation algorithm, and determining an objective sub-dimension evaluation score of the infrared image to be evaluated;
and inputting the objective sub-dimension evaluation score to the image quality evaluation model to obtain the image quality evaluation score of the infrared image to be evaluated.
11. An infrared image quality evaluation device characterized by comprising:
the data set construction module is used for generating a training set and a test set of image-carried subjective scores of image quality based on an original infrared image data set;
the model training module is used for training a machine learning model by utilizing each training image in the training set and the corresponding image quality subjective score to obtain an initial quality evaluation model;
the sub-dimension score calculating module is used for calculating the objective score of each objective quality index of each test image in the test set by utilizing a non-reference evaluation algorithm and determining the objective sub-dimension evaluation score of each test image;
the objective evaluation module is used for inputting the evaluation scores of the objective sub-dimensions into the initial quality evaluation model to obtain the image quality objective score of each test image;
and the model determining module is used for determining a final image quality evaluation model according to the image quality objective scores and the corresponding image quality subjective scores of the test images based on the initial quality evaluation model.
12. An infrared image quality evaluation device characterized by comprising:
a model construction module, configured to obtain an image quality evaluation model by using the infrared image quality evaluation method according to any one of claims 1 to 9 in advance;
the image acquisition module is used for acquiring an infrared image to be evaluated;
the objective scoring module is used for calculating an objective score of each objective quality index of the infrared image to be evaluated by utilizing a no-reference evaluation algorithm and determining an objective sub-dimension evaluation score of the infrared image to be evaluated;
and the quality evaluation module is used for inputting the objective sub-dimension evaluation score to the image quality evaluation model to obtain the image quality evaluation score of the infrared image to be evaluated.
13. An electronic device, characterized in that it comprises a processor and a memory, said processor being adapted to carry out the steps of the infrared image quality evaluation method according to any one of claims 1 to 9 and/or according to claim 10 when executing a computer program stored in said memory.
14. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the infrared image quality evaluation method according to any one of claims 1 to 9 and/or according to claim 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111370853.0A CN114066857A (en) | 2021-11-18 | 2021-11-18 | Infrared image quality evaluation method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111370853.0A CN114066857A (en) | 2021-11-18 | 2021-11-18 | Infrared image quality evaluation method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114066857A true CN114066857A (en) | 2022-02-18 |
Family
ID=80277916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111370853.0A Pending CN114066857A (en) | 2021-11-18 | 2021-11-18 | Infrared image quality evaluation method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114066857A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114897885A (en) * | 2022-06-17 | 2022-08-12 | 北京东宇宏达科技有限公司 | Infrared image quality comprehensive evaluation system and method |
| CN115147701A (en) * | 2022-07-01 | 2022-10-04 | 北京东宇宏达科技有限公司 | Infrared polarization image quality evaluation method and acquisition system |
| CN115760822A (en) * | 2022-11-28 | 2023-03-07 | 深圳市捷易科技有限公司 | Image quality detection model establishing method and system |
| CN116843683A (en) * | 2023-08-30 | 2023-10-03 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN117952962A (en) * | 2024-03-25 | 2024-04-30 | 南京科进实业有限公司 | Bone mineral density detection image processing method and system |
| CN117952968A (en) * | 2024-03-26 | 2024-04-30 | 沐曦集成电路(上海)有限公司 | Image quality evaluation method based on deep learning |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107396095A (en) * | 2017-08-28 | 2017-11-24 | 方玉明 | One kind is without with reference to three-dimensional image quality evaluation method |
| CN107481236A (en) * | 2017-08-28 | 2017-12-15 | 方玉明 | A kind of quality evaluating method of screen picture |
| CN108428227A (en) * | 2018-02-27 | 2018-08-21 | 浙江科技学院 | Non-reference picture quality appraisement method based on full convolutional neural networks |
| CN108460756A (en) * | 2018-01-30 | 2018-08-28 | 西安电子科技大学 | Based on statistical nature without reference ir image quality evaluating method |
| CN108615231A (en) * | 2018-03-22 | 2018-10-02 | 浙江科技学院 | A kind of full reference picture assessment method for encoding quality based on neural network learning fusion |
| CN109003265A (en) * | 2018-07-09 | 2018-12-14 | 嘉兴学院 | A kind of non-reference picture assessment method for encoding quality based on Bayes's compressed sensing |
| CN109754390A (en) * | 2018-12-11 | 2019-05-14 | 西北大学 | A kind of non-reference picture quality appraisement method based on mixing visual signature |
| CN110728656A (en) * | 2019-09-06 | 2020-01-24 | 西安电子科技大学 | Meta-learning-based no-reference image quality data processing method and intelligent terminal |
| CN112184672A (en) * | 2020-09-30 | 2021-01-05 | 上海大学 | No-reference image quality evaluation method and system |
-
2021
- 2021-11-18 CN CN202111370853.0A patent/CN114066857A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107396095A (en) * | 2017-08-28 | 2017-11-24 | 方玉明 | One kind is without with reference to three-dimensional image quality evaluation method |
| CN107481236A (en) * | 2017-08-28 | 2017-12-15 | 方玉明 | A kind of quality evaluating method of screen picture |
| CN108460756A (en) * | 2018-01-30 | 2018-08-28 | 西安电子科技大学 | Based on statistical nature without reference ir image quality evaluating method |
| CN108428227A (en) * | 2018-02-27 | 2018-08-21 | 浙江科技学院 | Non-reference picture quality appraisement method based on full convolutional neural networks |
| CN108615231A (en) * | 2018-03-22 | 2018-10-02 | 浙江科技学院 | A kind of full reference picture assessment method for encoding quality based on neural network learning fusion |
| CN109003265A (en) * | 2018-07-09 | 2018-12-14 | 嘉兴学院 | A kind of non-reference picture assessment method for encoding quality based on Bayes's compressed sensing |
| CN109754390A (en) * | 2018-12-11 | 2019-05-14 | 西北大学 | A kind of non-reference picture quality appraisement method based on mixing visual signature |
| CN110728656A (en) * | 2019-09-06 | 2020-01-24 | 西安电子科技大学 | Meta-learning-based no-reference image quality data processing method and intelligent terminal |
| CN112184672A (en) * | 2020-09-30 | 2021-01-05 | 上海大学 | No-reference image quality evaluation method and system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114897885A (en) * | 2022-06-17 | 2022-08-12 | 北京东宇宏达科技有限公司 | Infrared image quality comprehensive evaluation system and method |
| CN115147701A (en) * | 2022-07-01 | 2022-10-04 | 北京东宇宏达科技有限公司 | Infrared polarization image quality evaluation method and acquisition system |
| CN115760822A (en) * | 2022-11-28 | 2023-03-07 | 深圳市捷易科技有限公司 | Image quality detection model establishing method and system |
| CN115760822B (en) * | 2022-11-28 | 2024-03-19 | 深圳市捷易科技有限公司 | Image quality detection model building method and system |
| CN116843683A (en) * | 2023-08-30 | 2023-10-03 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN116843683B (en) * | 2023-08-30 | 2024-03-05 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN117952962A (en) * | 2024-03-25 | 2024-04-30 | 南京科进实业有限公司 | Bone mineral density detection image processing method and system |
| CN117952968A (en) * | 2024-03-26 | 2024-04-30 | 沐曦集成电路(上海)有限公司 | Image quality evaluation method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114066857A (en) | Infrared image quality evaluation method and device, electronic equipment and readable storage medium | |
| CN110930353B (en) | Method and device for detecting state of hole site protection door, computer equipment and storage medium | |
| CN110046673B (en) | No-reference tone mapping image quality evaluation method based on multi-feature fusion | |
| US7936377B2 (en) | Method and system for optimizing an image for improved analysis of material and illumination image features | |
| Qureshi et al. | Towards the design of a consistent image contrast enhancement evaluation measure | |
| CN103198467B (en) | Image processing apparatus and image processing method | |
| CN111369523B (en) | Method, system, equipment and medium for detecting cell stack in microscopic image | |
| US9679216B2 (en) | Morphological automatic triangle orientation detection | |
| CN103377468A (en) | Image processing device and image processing method | |
| CN110827269B (en) | Crop growth change condition detection method, device, equipment and medium | |
| US20130136348A1 (en) | Method and system for generating intrinsic images using a single reflectance technique | |
| CN109948566A (en) | A kind of anti-fraud detection method of double-current face based on weight fusion and feature selecting | |
| Pan et al. | No-reference assessment on haze for remote-sensing images | |
| CN116228780B (en) | Silicon wafer defect detection method and system based on computer vision | |
| van Zwanenberg et al. | Edge detection techniques for quantifying spatial imaging system performance and image quality | |
| CN108447058B (en) | Image quality evaluation method and system | |
| CN112419270A (en) | No-reference image quality evaluation method and device under meta learning and computer equipment | |
| Fang et al. | Image quality assessment on image haze removal | |
| US8913829B2 (en) | Automatic processing scale estimation for use in an image process | |
| TWI735967B (en) | Information processing method and computer program | |
| CN112651945A (en) | Multi-feature-based multi-exposure image perception quality evaluation method | |
| CN114897885B (en) | Infrared image quality comprehensive evaluation system and method | |
| US9232215B1 (en) | Measuring video acuity | |
| CN115457614B (en) | Image quality evaluation method, model training method and device | |
| CN115456888A (en) | Correction method and device for electronic artistic examination works, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |