CN113191365B - Cultural semantic image reconstruction effect evaluation method - Google Patents
Cultural semantic image reconstruction effect evaluation method Download PDFInfo
- Publication number
- CN113191365B CN113191365B CN202110515388.9A CN202110515388A CN113191365B CN 113191365 B CN113191365 B CN 113191365B CN 202110515388 A CN202110515388 A CN 202110515388A CN 113191365 B CN113191365 B CN 113191365B
- Authority
- CN
- China
- Prior art keywords
- image
- semantic
- cultural
- mask
- evaluation method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a cultural semantic image reconstruction effect evaluation method, which comprises the following steps: collecting a traditional pattern image, marking various semantic objects in the pattern image by using an image semantic marking tool, generating a semantic mask image according to the marking, forming a data set, and dividing the data set into a training set and a verification set; using a training set training generator, stopping when the loss function converges, and inputting the semantic mask image in the verification set into the generator to obtain a semantic image; comparing the image output by the generator with the real image in the verification set, and measuring the accuracy, the average merging ratio, the edge consistency, the PSNR, the SSIM and the like to obtain objective indexes; and integrating the objective indexes, and obtaining a single index capable of comprehensively evaluating the reconstruction effect of the cultural semantic image by setting index weights. The invention can realize the evaluation of the reconstruction of the cultural semantic image and can obtain an effective evaluation result.
Description
Technical Field
The invention relates to the technical field of computer image processing, in particular to a method for quantitatively evaluating a culture semantic image reconstruction effect.
Background
China is a multi-ethnic civilization ancient country with a long history, and in the development of the history, people use various symbols and patterns to represent various objects and symbolism. Patterns representing the same content also have changes with the meaning of the specific history at that time at different times. Finally, the Chinese traditional pattern with rich and colorful content and evolution rule in period is formed. The construction of a traditional pattern material library, and the combination of traditional pattern patterns and modern fashion is an important way for protecting and speaking traditional culture.
In the process of constructing a traditional pattern material library, on one hand, the traditional pattern material library is generally transmitted from mouth to mouth, and a master transmits the technology to a hiking as a secret means, so that a common person cannot simply judge whether the reconstructed image belongs to the same class as the original pattern, and whether distortion conditions or non-conforming conditions exist; on the other hand, in the course of reconstruction, the quality of the image produced by reconstruction varies due to the reconstruction method used. Therefore, when the texture material library is constructed, screening is required to be carried out on the image in storage, and a large amount of manpower and material resources are consumed.
Disclosure of Invention
The invention aims to solve the problems encountered in the conventional pattern semantic reconstruction process, and provides a cultural semantic image reconstruction effect evaluation method which is used for completing the quality evaluation of reconstructed images and helping a user to rapidly screen high-quality reconstructed images from the aspects of structure, signal-to-noise ratio, edge consistency and the like.
The cultural semantic image reconstruction effect evaluation method provided by the invention is characterized by comprising the following steps of: the method comprises the following steps:
step 1, collecting a plurality of traditional pattern images, manually marking example objects in the traditional pattern images and obtaining corresponding artificial semantic mask images, thereby obtaining a data set consisting of the traditional pattern images and corresponding artificial semantic mask images; randomly dividing the data set into a training set and a verification set;
step 2, training the image generator by taking the artificial semantic mask image and the corresponding traditional pattern image in the training set as inputs until the energy function converges, and obtaining the trained image generator after the training is finished;
step 3, verifying the trained image generator by using the image in the verification set:
3.1, calculating the accuracy of the generated image
Training the semantic segmentation model by using the training set; the artificial semantic mask image in the verification set is used for generating a composite image G through an image generator, and the composite image G is subjected to semantic segmentation by using a trained semantic segmentation model to obtain a semantic segmentation mask image I GS The method comprises the steps of carrying out a first treatment on the surface of the Then calculate the semantic mask semantic segmentation mask image I GS And corresponding artificial semantic mask image I S Pixel consistency between, i.e. the accuracy of the generated image:
wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m*n Representing the dimensionsAn image of m x n;
3.2, calculating the average intersection ratio MIoU of the mask images
According to the semantic segmentation mask image and the corresponding artificial semantic mask image, calculating the average intersection ratio MIoU
Wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m*n An image of size m x n;
3.3, calculating edge accuracy EA of the composite image
Acquiring edge contours of the composite image G and the traditional pattern image, and calculating edge accuracy EA of the composite image G according to the following formula
Wherein p is ii Representing the same edge p as the conventional pattern image and the composite image G 00 Or other positions p 11 Is a pixel of (1); i=0, 1, j=0, 1;
3.4 calculating the peak signal to noise ratio PSNR of the composite image
Wherein MAX I Representing the maximum value of the image color of the composite image, wherein MSE represents the mean square error of the pixel values of the composite image and the traditional pattern image; the 8-bit sample point is denoted as 255,
3.5, calculating the structural similarity SSIM of the generated image
SSIM(X,Y)＝l(X,Y)*c(X,Y)*s(X,Y)
Wherein SSIM (X, Y) represents the structural similarity of image X and image Y, l (X, Y) represents the brightness comparison of image X and image Y, c (X, Y) represents the contrast comparison of image X and image Y, s (X, Y) represents the structural comparison of image X and image Y, H represents the image height, W represents the image width, X (i, j) represents the pixel value at the X coordinate (i, j) of the image, Y (i, j) represents the pixel value at the Y coordinate (i, j) of the image, μ x Sum mu y Representing the pixel mean of image X and image Y respectively,
and->
Representing pixel variance, σ, of image X and image Y, respectively XY Pixel covariance representing image X and image Y; c (C) 1 、C 2 、C 3 Is a preset constant;
3.6, carrying out normalization treatment on the five evaluation indexes, and then carrying out weighted average to obtain a final evaluation index
Wherein G represents a composite image set, m represents the number of images in the set G, I represents the images, and alpha, beta, gamma, delta and epsilon are weights of different evaluation indexes.
The invention can realize the evaluation of the reconstruction of the cultural semantic image and can obtain an effective evaluation result.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for evaluating a reconstruction effect of a cultural semantic image according to an embodiment of the invention.
Detailed Description
As shown in the figure, the method for evaluating the reconstruction effect of the cultural semantic image comprises the following steps:
the cultural semantic image reconstruction effect evaluation method is characterized by comprising the following steps of: the method comprises the following steps:
s110, collecting a plurality of traditional pattern images, manually marking example objects in the traditional pattern images and obtaining corresponding artificial semantic mask images, thereby obtaining a data set consisting of the traditional pattern images and corresponding artificial semantic mask images; the data set is randomly divided into a training set and a verification set according to the proportion of 7:3. In this embodiment, an image semantic annotation tool (for example, labelme) is used to manually annotate a traditional pattern image, so as to obtain a semantic mask image.
S120, training the image generator by taking the artificial semantic mask image and the corresponding traditional pattern image in the training set as inputs until the energy function converges, and obtaining the trained image generator after the training is finished. In this step, a specific deep learning generation method should be used, and the generator is continuously optimized according to the training set until the loss function is converged and then stopped. The artificial semantic mask image in the verification set is then input into an image generator to obtain a composite image (semantic generated image).
S130, verifying the trained image generator by using the image in the verification set:
3.1, calculating the accuracy of the generated image
Training the semantic segmentation model by using the training set; the artificial semantic mask image in the verification set is used for generating a composite image G through an image generator, and the composite image G is subjected to semantic segmentation by using a trained semantic segmentation model to obtain a semantic segmentation mask image I GS The method comprises the steps of carrying out a first treatment on the surface of the Then calculate the semantic mask semantic segmentation mask image I GS And corresponding artificial semantic mask image I S Pixel consistency between, i.e. the accuracy of the generated image:
wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m x n Representing an image of size m x n.
3.2, calculating the average intersection ratio MIoU of the mask images
The homography ratio refers to the ratio of the intersection and union of the two sets of real values and predicted values, and can be calculated by using the sum (union) of TP (intersection) ratios TP, FP, FN. MIoU is typically calculated on a class basis, and IoU for each class is calculated and accumulated and averaged. The larger the value, the more pixels that are predicted to be correct, while the fewer pixels that are predicted to be of other classes and other classes are predicted to be of the class.
According to the semantic segmentation mask image and the corresponding artificial semantic mask image, calculating the average intersection ratio MIoU
Wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m*n Representing an image of size m x n.
3.3, calculating edge accuracy EA of the composite image
Similar to the accuracy, the edge accuracy describes the pixel consistency of the generated image with the real image.
Acquiring edge contours of the composite image G and the traditional pattern image, and calculating edge accuracy EA of the composite image G according to the following formula
Wherein p is ii Representing the same edge p as the conventional pattern image and the composite image G 00 Or other positions p 11 Is a pixel of (1); i=0, 1, j=0, 1. In the step, a Canny algorithm is used for extracting the edge contour, and the convolution kernel size used by the Canny algorithm is 3.
3.4 calculating the peak signal to noise ratio PSNR of the composite image
The peak signal-to-noise ratio, which represents the ratio of the maximum possible power of a signal to the destructive noise power affecting its accuracy of representation, is often expressed in logarithmic decibels, and is an objective measure of image distortion or noise level. Obtained by calculation by the following formula:
wherein MAX I Representing the maximum value of the image color of the composite image, the MSE represents the mean square error of the pixel values of the composite image and the conventional pattern image.
3.5, calculating the structural similarity SSIM of the generated image
The index for measuring the structural similarity between images is measured by comparing three parts of brightness (brightness), contrast (contrast) and structure (structure), and can give a numerical value between 0 and 1, and the larger the index is, the smaller the difference between the output image and the undistorted image is, namely, the better the image quality is.
SSIM(X,Y)＝l(X,Y)*c(X,Y)*s(X,Y)
Wherein SSIM (X, Y) represents the structural similarity of image X and image Y, l (X, Y) represents the brightness comparison of image X and image Y, c (X, Y) represents the contrast comparison of image X and image Y, s (X, Y) represents the structural comparison of image X and image Y, H represents the image height, W represents the image width, X (i, j) represents the pixel value at the X coordinate (i, j) of the image, Y (i, j) represents the pixel value at the Y coordinate (i, j) of the image, μ x Sum mu y Representing the pixel mean of image X and image Y respectively,
and->
Representing pixel variance, σ, of image X and image Y, respectively XY Pixel covariance representing image X and image Y; c (C) 1 、C 2 、C 3 To avoid the case of a score of 0, C is usually taken as a preset constant 1 ＝(K 1 *L) 2 ,C 2 ＝(K 2 *L) 2 ,C 3 ＝C 2 /2. In this step, K 1 ＝0.01，K 2 ＝0.03，L＝255。
S140, carrying out normalization processing on the five evaluation indexes, and then carrying out weighted average to obtain a final evaluation index
Wherein G represents a composite image set, m represents the number of images in the set G, I represents the images, and alpha, beta, gamma, delta and epsilon are weights of different evaluation indexes. In this embodiment, α=0.2, β=0.2, γ=0.2, δ=0.2, and ε=0.2.
Wherein, the accuracy, edge accuracy and structure consistency are all in the range of [0,1]]In between, a higher value represents a higher quality of the generated image. The value range of the average cross ratio and the peak signal-to-noise ratio is 0, ++ infinity]Between using functions
It is normalized.
In addition to the embodiments described above, other embodiments of the invention are possible. All technical schemes formed by equivalent substitution or equivalent transformation fall within the protection scope of the invention.
Claims (9)
1. A cultural semantic image reconstruction effect evaluation method is characterized in that: the method comprises the following steps:
step 1, collecting a plurality of traditional pattern images, manually marking example objects in the traditional pattern images and obtaining corresponding artificial semantic mask images, thereby obtaining a data set consisting of the traditional pattern images and corresponding artificial semantic mask images; randomly dividing the data set into a training set and a verification set;
step 2, training the image generator by taking the artificial semantic mask image and the corresponding traditional pattern image in the training set as inputs until the energy function converges, and obtaining the trained image generator after the training is finished;
step 3, verifying the trained image generator by using the image in the verification set:
3.1, calculating the accuracy of the generated image
Training the semantic segmentation model by using the training set; the artificial semantic mask image in the verification set generates a synthetic image through an image generator, and semantic segmentation is carried out on the synthetic image by using a trained semantic segmentation model to obtain a semantic segmentation mask image I GS The method comprises the steps of carrying out a first treatment on the surface of the Then calculate the semantic mask semantic segmentation mask image I GS And corresponding artificial semantic mask image I S Pixel consistency between, i.e. the accuracy of the generated image:
wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m*n An image of size m x n; 3.2, calculating the average intersection ratio MIoU of the mask images
According to the semantic segmentation mask image and the corresponding artificial semantic mask image, calculating the average intersection ratio MIoU
Wherein p is ii Refers to the number of pixels with the true class I and predicted as I, k is the number of semantic classes, I m*n An image of size m x n;
3.3, calculating edge accuracy EA of the composite image
Acquiring edge contours of the composite image and the traditional pattern image, and calculating edge accuracy EA of the composite image according to the following formula
Wherein p is ii Referring to the number of pixels of the true class i and predicted as i, i=0, 1, j=0, 1;
3.4 calculating the peak signal to noise ratio PSNR of the composite image
Wherein MAX I Representing the maximum value of the image color of the composite image, wherein MSE represents the mean square error of the pixel values of the composite image and the traditional pattern image; the 8-bit sample point is denoted as 255,
3.5, calculating the structural similarity SSIM of the generated image
SSIM(X,Y)＝l(X,Y)*c(X,Y)*s(X,Y)
Wherein SSIM (X, Y) represents the structural similarity of image X and image Y, l (X, Y) represents the brightness comparison of image X and image Y, c (X, Y) represents the contrast comparison of image X and image Y, s (X, Y) represents the structural comparison of image X and image Y, H represents the image height, W represents the image width, X (i, j) represents the pixel value at the X coordinate (i, j) of the image, Y (i, j) represents the pixel value at the Y coordinate (i, j) of the image, μ x Sum mu y Representing the pixel mean of image X and image Y respectively,
representing pixel variance, sigma, of image X XY Pixel covariance representing image X and image Y; c (C) 1 、C 2 、C 3 Is a preset constant;
3.6, carrying out normalization treatment on the five evaluation indexes, and then carrying out weighted average to obtain a final evaluation index
Wherein G represents a composite image set, m represents the number of images in the composite image set G, I represents the images, and alpha, beta, gamma, delta and epsilon are weights of different evaluation indexes.
2. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in the step 1, the traditional pattern image is manually marked by using an image semantic marking tool, and a semantic mask image is obtained.
3. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in the step 1, the data set is randomly divided into a training set and a verification set according to the proportion of 7:3.
4. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in the step 2, a specific deep learning generation method should be used, and the generator is continuously optimized according to the training set until the loss function is converged and then stopped.
5. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in the step 3.3, the Canny algorithm is used for extracting the edge contour, and the convolution kernel size used by the Canny algorithm is 3.
6. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in step 3.5, C 1 ＝(K 1 *L) 2 ,C 2 ＝(K 2 *L) 2 ,
K 1 ＝0.01，K 2 ＝0.03，L＝255。
7. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: in step 3.6, α=0.2, β=0.2, γ=0.2, δ=0.2, and ε=0.2.
8. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: the range of values of the accuracy, the edge accuracy and the structural consistency are all between 0 and 1, and the higher the value is, the higher the quality of the generated image is.
9. The cultural semantic image reconstruction effect evaluation method according to claim 1, wherein: the value ranges of the average cross ratio and the peak signal-to-noise ratio are between [0, + -infinity ], and a function is used
It is normalized. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110515388.9A CN113191365B (en) | 2021-05-12 | 2021-05-12 | Cultural semantic image reconstruction effect evaluation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110515388.9A CN113191365B (en) | 2021-05-12 | 2021-05-12 | Cultural semantic image reconstruction effect evaluation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113191365A CN113191365A (en) | 2021-07-30 |
| CN113191365B true CN113191365B (en) | 2023-04-21 |
Family
ID=76981338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110515388.9A Active CN113191365B (en) | 2021-05-12 | 2021-05-12 | Cultural semantic image reconstruction effect evaluation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113191365B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112750129A (en) * | 2021-03-11 | 2021-05-04 | 湘潭大学 | Image semantic segmentation model based on feature enhancement position attention mechanism |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3742346A3 (en) * | 2019-05-23 | 2021-06-16 | HTC Corporation | Method for training generative adversarial network (gan), method for generating images by using gan, and computer readable storage medium |
| CN112016618A (en) * | 2020-08-28 | 2020-12-01 | 方玉明 | Measurement method for generalization capability of image semantic segmentation model |
| CN112308860B (en) * | 2020-10-28 | 2024-01-12 | 西北工业大学 | Earth observation image semantic segmentation method based on self-supervision learning |
-
2021
- 2021-05-12 CN CN202110515388.9A patent/CN113191365B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112750129A (en) * | 2021-03-11 | 2021-05-04 | 湘潭大学 | Image semantic segmentation model based on feature enhancement position attention mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113191365A (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105046277B (en) | Robust mechanism study method of the feature significance in image quality evaluation | |
| CN104134204B (en) | Image definition evaluation method and image definition evaluation device based on sparse representation | |
| CN109583474B (en) | Training sample generation method for industrial big data processing | |
| CN105844278A (en) | Multi-feature fused fabric scanning pattern recognition method | |
| CN111325687B (en) | Smooth filtering evidence obtaining method based on end-to-end deep network | |
| CN1912889A (en) | Deformed fingerprint identification method based on local triangle structure characteristic collection | |
| CN103700077B (en) | A kind of method for adaptive image enhancement based on human-eye visual characteristic | |
| CN116051382A (en) | Data enhancement method based on deep reinforcement learning generation type antagonistic neural network and super-resolution reconstruction | |
| CN107018410B (en) | A kind of non-reference picture quality appraisement method based on pre- attention mechanism and spatial dependence | |
| CN115170548A (en) | Leather defect automatic detection method and device based on unsupervised learning | |
| CN114897825A (en) | Solid wood floor sorting method and system based on computer vision | |
| CN108090914B (en) | Color image segmentation method based on statistical modeling and pixel classification | |
| CN113191365B (en) | Cultural semantic image reconstruction effect evaluation method | |
| CN110880192B (en) | Image DCT coefficient distribution fitting method based on probability density function dictionary | |
| CN114972332B (en) | Bamboo laminated wood crack detection method based on image super-resolution reconstruction network | |
| CN112200748A (en) | Image blind denoising method based on capsule generation countermeasure network noise modeling | |
| CN106373129A (en) | FCM remote sensing image segmentation method based on dual degree of membership | |
| Li et al. | Minimum standard deviation difference-based thresholding | |
| CN116563133A (en) | Low-illumination color image enhancement method based on simulated exposure and multi-scale fusion | |
| CN105513071A (en) | Topographic map schema quality evaluation method | |
| CN111104868B (en) | Cross-quality face recognition method based on convolutional neural network characteristics | |
| CN113112515B (en) | Evaluation method for pattern image segmentation algorithm | |
| CN103632358B (en) | For the method that low-resolution image is converted to high-definition picture | |
| CN113239730A (en) | Method for automatically eliminating structural false modal parameters based on computer vision | |
| CN107784662B (en) | Image target significance measurement method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |