CN117575916B - Image quality optimization method, system, equipment and medium based on deep learning - Google Patents
Image quality optimization method, system, equipment and medium based on deep learning Download PDFInfo
- Publication number
- CN117575916B CN117575916B CN202410077127.7A CN202410077127A CN117575916B CN 117575916 B CN117575916 B CN 117575916B CN 202410077127 A CN202410077127 A CN 202410077127A CN 117575916 B CN117575916 B CN 117575916B
- Authority
- CN
- China
- Prior art keywords
- image
- optimized
- region
- optimized image
- quality difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000013135 deep learning Methods 0.000 title claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 52
- 230000007547 defect Effects 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 24
- 238000005516 engineering process Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000035945 sensitivity Effects 0.000 claims description 7
- 238000013136 deep learning model Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000000691 measurement method Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000013145 classification model Methods 0.000 claims description 3
- 238000011524 similarity measure Methods 0.000 claims 1
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The invention relates to the technical field of image processing, in particular to an image quality optimization method, system, equipment and medium based on deep learning, which comprises the following steps: performing double-channel super-resolution optimization on the original image to obtain a first optimized image and a second optimized image; performing image quality difference identification on the first optimized image and the second optimized image, and extracting an image area with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image to obtain an image quality difference area; calculating the ratio between the area of the image quality difference area and the original image area to obtain the image quality difference duty ratio; comparing the image quality difference duty ratio with a preset second-order threshold value: if the image quality difference ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; the image quality can be comprehensively and accurately improved, and the influence of image distortion and defects is reduced.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image quality optimization method, system, device, and medium based on deep learning.
Background
In the field of image processing, optimization and improvement of image quality are important research subjects, and with development of technology, especially application of deep learning in image processing, image optimization and improvement can be performed by using a specific algorithm, wherein super-resolution technology is a common method, and resolution and quality of an image can be improved by learning a mapping relation between a low-resolution image and a high-resolution image.
However, the existing super-resolution image quality optimization method only adopts a single-channel super-resolution optimization result, and the situation that the optimized image has partial image area distortion is easily caused due to the singleness of the optimization method and the defects of the original image.
Disclosure of Invention
In order to solve the technical problems, the invention provides the image quality optimization method based on the deep learning, which can comprehensively and accurately improve the quality of images and reduce the influence of image distortion and defects.
In a first aspect, the present invention provides a method for optimizing image quality based on deep learning, the method comprising:
Performing double-channel super-resolution optimization on the original image to obtain a first optimized image and a second optimized image;
Performing image quality difference identification on the first optimized image and the second optimized image, and extracting an image area with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image to obtain an image quality difference area;
calculating the ratio between the area of the image quality difference area and the original image area to obtain the image quality difference duty ratio;
Comparing the image quality difference duty ratio with a preset second-order threshold value: if the image quality difference ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting an area corresponding to the image quality difference area in the original image to obtain an in-doubt area of the original image;
performing double-channel super-resolution optimization on the suspicious region of the original image to obtain a first region optimized image and a second region optimized image;
Comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, combining and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively according to the same optimizing channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed the preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed.
Further, the dual-channel super-resolution optimization method comprises the following steps:
interpolation is carried out on pixel values around known pixels in the original image to increase the resolution of the image, and a first optimized image is obtained;
And extracting information from the original image and deducing missing pixels to increase the resolution of the image by using an image processing algorithm and a signal processing technology, so as to obtain a second optimized image.
Further, the image quality difference region extraction method includes:
constructing a two-classification model by using a deep learning method;
Constructing a training data set containing the quality difference of the marked image, and manually marking the area with the quality difference on the image of the training data set;
training the model using the prepared training data set;
Presetting a first-order threshold value, and determining a significant difference;
Once model training is completed, deducing the first and second optimized images by using a trained network to obtain image difference mapping;
and comparing the pixel value in the difference map with a set first-order threshold value, extracting a region with the image quality difference exceeding the first-order threshold value, and obtaining an image quality difference region.
Further, the dual-channel super-resolution optimization method for the original image suspicious region comprises the following steps:
Processing is independently carried out for each suspicious region, and the suspicious region is cut out;
constructing a deep learning model, wherein the model takes a low-resolution suspicious region as input and a high-resolution image as output;
And inputting the suspicious region of each original image into the trained model to generate a first region optimized image and a second region optimized image.
Further, the final optimized image output method includes:
performing similarity calculation on the first region optimized image and the second region optimized image by comparing pixel values, structures and brightness differences between the first region optimized image and the second region optimized image by using a similarity measurement method;
Presetting a third threshold value as a standard for judging the similarity of the two areas;
Comparing the calculated similarity with a third threshold value, and judging the similarity of the first region optimized image and the second region optimized image; if the similarity exceeds a preset third threshold value, the optimization results of the two areas are similar, and merging and restoring are carried out; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the fact that the difference of the optimized results of the two regions is large is indicated that the original image has defects and cannot be optimized;
Combining the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively, and carrying out weighted average processing on pixel values of the two regions to obtain an image of finally integrating two region information;
and randomly selecting a restored image as a final optimized image of the original image to output.
Further, the similarity measurement method comprises a structural similarity index and a mean square error.
Further, the set influencing factors of the second order threshold include image quality standard, sensitivity of image processing application, characteristics of original image and user experience requirement.
In another aspect, the present application also provides an image quality optimization system based on deep learning, the system comprising:
The first double-channel super-resolution optimization module is used for carrying out double-channel super-resolution optimization on the original image, obtaining a first optimized image and a second optimized image, and sending the first optimized image and the second optimized image;
The image quality difference recognition and region extraction module is used for receiving the first optimized image and the second optimized image, carrying out image quality difference recognition on the first optimized image and the second optimized image, extracting an image region with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image, obtaining an image quality difference region and sending the image quality difference region;
The image quality difference analysis module is used for receiving the image quality difference area, calculating the ratio between the area of the image quality difference area and the area of the original image, obtaining the image quality difference duty ratio and sending the image quality difference duty ratio;
The decision module is used for receiving the image quality difference duty ratio, comparing the image quality difference duty ratio with a preset second order threshold value, and if the image quality difference duty ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting a region corresponding to the image quality difference region in the original image, obtaining an in-doubt region of the original image, and transmitting the in-doubt region;
The second double-channel super-resolution optimization module is used for receiving the original image in-doubt area, performing double-channel super-resolution optimization on the original image in-doubt area independently, obtaining a first area optimized image and a second area optimized image, and transmitting the first area optimized image and the second area optimized image;
The similarity analysis and image restoration module is used for receiving the first region optimized image and the second region optimized image, comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, respectively merging and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image according to the same optimization channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed the preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed.
In a third aspect, the present application provides an electronic device comprising a bus, a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the transceiver, the memory and the processor being connected by the bus, the computer program when executed by the processor implementing the steps of any of the methods described above.
In a fourth aspect, the application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of any of the methods described above.
Compared with the prior art, the invention has the beneficial effects that: according to the method, the original image is subjected to double-channel super-resolution optimization, the information of the image on two channels is considered, the resolution and quality of the image can be comprehensively improved, and details and textures in the image are effectively reserved; by carrying out image quality difference recognition on the first and second optimized images, extracting image areas with quality differences, the possible distortion or problem in the optimized images can be positioned and understood, and guidance is provided for subsequent processing; by calculating the ratio of the area of the image quality difference area to the original image area, the concept of the image quality difference ratio is introduced, so that the method can dynamically adjust the optimization strategy, and different processing modes are adopted according to different parts of the image, thereby being more flexibly suitable for different types of images;
For the areas with quality differences, independent double-channel super-resolution optimization is adopted, so that the problem areas can be processed in a targeted manner, distortion possibly introduced is reduced to the greatest extent, and the quality of the whole image is improved; the consistency of the image areas is further judged by carrying out similarity comparison on the optimized images of the first area and the second area, and the information of the two areas is fully utilized by adopting a merging and restoring strategy, so that the image defects are effectively restored;
In summary, in the image quality optimization process, the method comprehensively considers a plurality of factors including information of different channels, positioning and processing of image quality differences, dynamic adjustment optimization strategies and similarity comparison and restoration mechanisms, and can comprehensively and accurately improve the quality of images and reduce the influence of image distortion and defects.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a flow chart of a two-channel super-resolution optimization method;
Fig. 3 is a block diagram of an image quality optimization system based on deep learning.
Detailed Description
In the description of the present application, those skilled in the art will appreciate that the present application may be embodied as methods, apparatus, electronic devices, and computer-readable storage media. Accordingly, the present application may be embodied in the following forms: complete hardware, complete software (including firmware, resident software, micro-code, etc.), a combination of hardware and software. Furthermore, in some embodiments, the application may also be embodied in the form of a computer program product in one or more computer-readable storage media, which contain computer program code.
Any combination of one or more computer-readable storage media may be employed by the computer-readable storage media described above. The computer-readable storage medium includes: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer readable storage medium include the following: portable computer magnetic disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, flash memories, optical fibers, optical disk read-only memories, optical storage devices, magnetic storage devices, or any combination thereof. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, device.
The technical scheme of the application obtains, stores, uses, processes and the like the data, which all meet the relevant regulations of national laws.
The application provides a method, a device and electronic equipment through flow charts and/or block diagrams.
It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer readable program instructions may also be stored in a computer readable storage medium that can cause a computer or other programmable data processing apparatus to function in a particular manner. Thus, instructions stored in a computer-readable storage medium produce an instruction means which implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The present application will be described below with reference to the drawings in the present application.
Embodiment one: as shown in fig. 1 to 2, the image quality optimization method based on deep learning of the present invention specifically includes the following steps:
s1, performing double-channel super-resolution optimization on an original image to obtain a first optimized image and a second optimized image;
The two-channel super-resolution technology is a technology for improving the image quality by using a deep learning method, an image is generally expressed as a plurality of channels, for a color image, the common channel number is red, green and blue three channels, the traditional super-resolution technology mainly focuses on the image reconstruction of a single channel, and the two-channel super-resolution technology improves the resolution and the quality of the image by simultaneously considering the information of the plurality of channels;
The double-channel super-resolution optimization method comprises the following steps:
S11, interpolating pixel values around known pixels in an original image to increase the resolution of the image, and obtaining a first optimized image; in image processing, interpolation is a commonly used technique for estimating the values of unknown points between known points, and for super-resolution, interpolation is used to increase the number of pixels in an image so that the image appears clearer and finer;
S12, extracting information from the original image by using an image processing algorithm and a signal processing technology, and deducing missing pixels to increase the resolution of the image, so as to obtain a second optimized image; by using the spatial relationship between pixels, a neighborhood-based algorithm is used to extract information from the image, and missing pixels are estimated by block matching the image.
In the step, the interpolation method is used for interpolating the pixels around the known pixels in the original image, so that the resolution of the image can be increased, and the image is clearer and finer; the dual-channel super-resolution technology utilizes a deep learning method, improves the resolution and quality of an image by considering the information of a plurality of channels, and compared with the traditional single-channel super-resolution technology, the dual-channel super-resolution technology can fully utilize the correlation and information of different channels in the image, thereby providing more accurate reconstruction results;
The method has the advantages that the simple interpolation and the higher-level information extraction and pixel inference technology are considered, the resolution of the image can be comprehensively optimized, the artifacts possibly introduced by the simple interpolation are overcome, and the effect of improving the super resolution is achieved; the neighborhood-based algorithm and the block matching technology utilize the spatial relationship among pixels to help to extract more information from the original image, and the algorithms can infer missing pixels more accurately by considering the relevance of the pixels in the local area, so that the resolution of the image is further improved;
in summary, the two-channel super-resolution optimization method can remarkably improve the resolution and quality of the image by comprehensively applying interpolation and depth learning technologies and considering the multi-channel information and the spatial relationship.
S2, carrying out image quality difference identification on the first optimized image and the second optimized image, and extracting an image area with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image to obtain an image quality difference area;
The image quality difference region extraction method comprises the following steps:
s21, constructing a model for two classifications by using a deep learning method, so that the model learns and distinguishes the quality difference between the first and second optimized images;
s22, constructing a training data set containing quality differences of marked images, manually marking the images of the training data set, and designating areas with the quality differences;
s23, training the model by using the prepared training data set, and optimizing network parameters to minimize a loss function of the difference recognition task in the training process;
s24, presetting a first-order threshold value for determining which differences are significant;
S25, once model training is completed, deducing the first and second optimized images by using a trained network to obtain image difference mapping;
S26, comparing the pixel value in the difference mapping with a set first-order threshold value, extracting a region with the image quality difference exceeding the first-order threshold value, and obtaining an image quality difference region.
In the step, by using a deep learning model, the system can automatically learn the quality difference between the first and second optimized images without manually prescribing the characteristics or rules of the difference, thereby improving the intelligence and the adaptability of the system; the model can learn complex image characteristics, so that difference recognition can be well carried out on images outside a training data set, and the accuracy and generalization of the model are improved; constructing a training data set and manually marking the model so that the model can learn the mode of quality difference from the data, thereby better adapting to the requirement of a specific task;
The preset first-order threshold allows the system to adjust the sensitivity of the image quality difference according to specific requirements, so that the method has more flexibility when coping with different types of images or quality standards; by comparing the pixel values in the difference map with the first-order threshold, the system can accurately locate and extract the image areas with quality differences, and provide clear information for the subsequent processing steps;
in summary, the present step combines with the deep learning technique, so that the system is more adaptive and intelligent, and the efficiency and accuracy for solving the quality problem in the image processing are improved.
S3, calculating the ratio of the area of the image quality difference area to the area of the original image to obtain the image quality difference duty ratio;
Step S3, evaluating the degree of image quality loss by calculating the ratio of the image quality difference area to the original image area so as to know the distortion degree of the optimized image relative to the original image; the following is a detailed description of this step:
S31, calculating the area of a quality difference region in the first and second optimized images by using an image processing technology;
s32, dividing the area of the image quality difference area by the total area of the original image to obtain an image quality difference duty ratio;
The distortion degree of the optimized image can be known quantitatively through the calculated image quality difference duty ratio, and the higher difference duty ratio means that more image areas are subjected to quality loss after being optimized.
In the step, the distortion degree of the optimized image relative to the original image can be quantitatively measured by calculating the quality difference duty ratio, so that the quality change of the optimized image can be known; calculating the area of the quality difference region by using an image processing technology, so that distortion introduced in the optimization process can be accurately positioned; if the difference of the image quality is high, the system can prompt that the large quality loss exists, so that problem positioning is performed, defects in an optimization algorithm are conveniently analyzed deeply, corresponding improvement measures are adopted, and the image processing effect is improved;
in summary, step S3 provides an operable, quantitative image quality evaluation index for the system, so that the whole image processing flow is more automated and controllable.
S4, comparing the image quality difference duty ratio with a preset second-order threshold value: if the image quality difference ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting an area corresponding to the image quality difference area in the original image to obtain an in-doubt area of the original image;
Step S4, judging the degree of image quality difference by setting a threshold value, and carrying out finer processing under the condition of larger difference, so that the sensitivity to distortion in image processing is considered, excessive distortion in integral image optimization is avoided, and the improvement of integral image quality is ensured;
The setting influence factors of the second-order threshold value include:
A. image quality criteria, defining image quality criteria including color fidelity, sharpness, contrast; different application scenes have different requirements on image quality, so that the selection of the threshold value needs to consider the specific requirements of the application;
B. Sensitivity of image processing applications, different image processing applications having different sensitivities to image quality, the threshold value being set to account for the sensitivity of the particular application;
C. the nature of the original image, the type and extent of defects present in the original image, the nature of which can affect the threshold level at which the second order threshold is set;
D. The user experience demands that the user expects different to the image processing result, if the user pays more attention to the overall image quality, a lower threshold value needs to be set, and if the user demands extremely high detail, a higher threshold value needs to be set.
In the step, the second-order threshold value is set so that the system can select to directly output one of the first optimized image and the second optimized image without further processing under the condition that the difference of the image quality is smaller, thereby avoiding unnecessary fine processing and reducing the waste of calculation resources;
For the situation that the image quality difference is large, the system selects and extracts the suspicious region, so that excessive distortion is prevented from being introduced in the whole image optimization; by randomly selecting one of the first optimized image and the second optimized image as the final output, the system retains the quality of the overall image rather than simply discarding one of the images, improving user experience;
in summary, step S4 makes the image processing system perform selective processing more intelligently according to the degree of the difference of image quality while improving the image quality, so as to meet the dual pursuit of the image processing effect and efficiency in practical application.
S5, performing double-channel super-resolution optimization on the suspicious region of the original image to obtain a first region optimized image and a second region optimized image;
The dual-channel super-resolution optimization method for the original image suspicious region comprises the following steps:
s51, processing each suspicious region independently, and cutting out the suspicious region to form a plurality of irregularly-shaped regions;
s52, constructing a deep learning model, wherein the model takes a low-resolution suspicious region as input and a high-resolution image as output;
S53, inputting the suspicious region of each original image into the trained model, and generating two optimized images, namely a first region optimized image and a second region optimized image.
In the step, by independently processing each suspicious region, the system can perform personalized optimization according to the specific characteristics and distortion of each region, so that the optimization effect is improved; the deep learning model is utilized, the low-resolution suspicious region is used as input, the high-resolution image is used as an output target, the model can learn details and structures in the original image so as to reconstruct the suspicious region more accurately, and the double-channel information is utilized to perform more comprehensive optimization;
The method has the advantages that the independent optimization is carried out on the suspicious region, the possible defect of global optimization on the whole image is avoided, the total image optimization can cause excessive smoothness or distortion due to the quality difference of different regions, and the detail and the structure of the image are better reserved by purposefully processing the suspicious region, so that the effect of super-resolution optimization is improved; because the model is learned through training data, the model can adapt to various image quality differences and distortion types, and the robustness in practical application is improved;
In summary, the step S5 can accurately process the region with quality difference in the image, and provides a more effective means for improving the final image quality, so that the optimization result more accords with the local features and the real details of the original image.
S6, comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, combining and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively according to the same optimizing channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed;
S6, comparing the similarity of the first area optimized image and the second area optimized image to determine whether enough similarity exists for merging and restoring, wherein the step aims to comprehensively consider the information of the two optimized areas to finally determine the optimal optimization result of the original image; the final optimized image output method comprises the following steps:
S61, performing similarity calculation on the first region optimized image and the second region optimized image by comparing pixel values, structures and brightness differences between the first region optimized image and the second region optimized image by using a similarity measurement method; the similarity measurement method comprises a structural similarity index SSIM and a mean square error MSE;
s62, presetting a third threshold value, wherein the third threshold value is used as a standard for judging whether the two areas are similar enough;
S63, comparing the calculated similarity with a third threshold value, and judging whether the first area optimized image and the second area optimized image are similar enough or not; if the similarity exceeds a preset third threshold value, the optimization results of the two areas are similar, and the combination restoration is needed to be considered; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the fact that the difference of the optimized results of the two regions is large is indicated, and the defect exists in the original image and cannot be optimized is indicated;
S64, combining the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively; the pixel values of the two areas are weighted and averaged to obtain an image which finally comprehensively considers the information of the two areas;
s65, randomly selecting one restored image as a final optimized image of the original image to output.
In the step, the similarity comparison is carried out on the first area optimized image and the second area optimized image, so that the difference between different areas can be reasonably processed in the merging and restoring process; the preset third threshold value is used as a similarity standard, so that an adaptive decision mechanism is allowed, and the characteristics of different images can be better adapted;
When the similarity does not reach a preset third threshold, clearly indicating that the original image has defects and cannot be optimized; the method is beneficial to avoiding unreasonable merging and restoration under the condition of large image difference, thereby protecting the image quality; after merging and restoring, a piece of restored image is randomly selected as a final optimized image, so that certain randomness is introduced; this helps to increase the diversity of the results by not always selecting the same image in the case of higher similarity;
In summary, in the step S6, by intelligently and comprehensively considering the optimization results of different regions, the adaptive decision is performed according to the similarity threshold, so that the possible problem of differential processing in the super-resolution image quality optimization is effectively solved, and the quality and applicability of the final optimized image are improved.
Embodiment two: as shown in fig. 3, the image quality optimization system based on deep learning of the present invention specifically includes the following modules;
The first double-channel super-resolution optimization module is used for carrying out double-channel super-resolution optimization on the original image, obtaining a first optimized image and a second optimized image, and sending the first optimized image and the second optimized image;
The image quality difference recognition and region extraction module is used for receiving the first optimized image and the second optimized image, carrying out image quality difference recognition on the first optimized image and the second optimized image, extracting an image region with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image, obtaining an image quality difference region and sending the image quality difference region;
The image quality difference analysis module is used for receiving the image quality difference area, calculating the ratio between the area of the image quality difference area and the area of the original image, obtaining the image quality difference duty ratio and sending the image quality difference duty ratio;
The decision module is used for receiving the image quality difference duty ratio, comparing the image quality difference duty ratio with a preset second order threshold value, and if the image quality difference duty ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting a region corresponding to the image quality difference region in the original image, obtaining an in-doubt region of the original image, and transmitting the in-doubt region;
The second double-channel super-resolution optimization module is used for receiving the original image in-doubt area, performing double-channel super-resolution optimization on the original image in-doubt area independently, obtaining a first area optimized image and a second area optimized image, and transmitting the first area optimized image and the second area optimized image;
The similarity analysis and image restoration module is used for receiving the first region optimized image and the second region optimized image, comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, respectively merging and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image according to the same optimization channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed the preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed.
The system can process the image information of multiple channels simultaneously by using the two-channel super-resolution optimization module, so that the comprehensiveness and accuracy of image quality optimization are improved; the image quality difference recognition and region extraction module is introduced, so that the system can accurately extract a difference region aiming at the quality difference existing in the original image, and is beneficial to targeted subsequent processing; the system quantitatively analyzes the difference region through the image quality difference analysis module, calculates the ratio between the area of the difference region and the area of the original image, and provides deeper image quality information; the decision module realizes self-adaptive decision by comparing with a preset threshold value, and a decision mechanism enables the system to adopt different processing strategies according to specific conditions, so that the flexibility and the adaptability of the system are improved;
The second double-channel super-resolution optimization module is introduced to specially process the suspicious region, and the system improves the processing capacity of the original image defects through further optimization of the suspicious region; the similarity analysis and image restoration module judges whether the restored images can be combined or not through similarity comparison when the suspicious region is processed, and the mechanism can better reserve the consistency and the integrity of the images when the image defects are processed; in the decision module, a strategy of randomly selecting one image as a final optimized image is adopted, so that optimization in a certain direction is avoided, and the robustness of the system is improved;
In conclusion, the system is more comprehensive and accurate in processing the image quality optimization, has certain self-adaptability and flexibility, and can cope with different types of images and quality problems thereof.
The various modifications and embodiments of the image quality optimization method based on deep learning in the foregoing embodiment are equally applicable to the image quality optimization system based on deep learning in this embodiment, and those skilled in the art will clearly know the implementation method of the image quality optimization system based on deep learning in this embodiment through the foregoing detailed description of the image quality optimization method based on deep learning, so that they will not be described in detail herein for brevity of description.
In addition, the application also provides an electronic device, which comprises a bus, a transceiver, a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the transceiver, the memory and the processor are respectively connected through the bus, and when the computer program is executed by the processor, the processes of the method embodiment for controlling output data are realized, and the same technical effects can be achieved, so that repetition is avoided and redundant description is omitted.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that it will be apparent to those skilled in the art that modifications and variations can be made without departing from the technical principles of the present invention, and these modifications and variations should also be regarded as the scope of the invention.
Claims (6)
1. A depth learning-based image quality optimization method, the method comprising:
Performing double-channel super-resolution optimization on the original image to obtain a first optimized image and a second optimized image;
Performing image quality difference identification on the first optimized image and the second optimized image, and extracting an image area with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image to obtain an image quality difference area;
calculating the ratio between the area of the image quality difference area and the original image area to obtain the image quality difference duty ratio;
Comparing the image quality difference duty ratio with a preset second-order threshold value: if the image quality difference ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting an area corresponding to the image quality difference area in the original image to obtain an in-doubt area of the original image;
performing double-channel super-resolution optimization on the suspicious region of the original image to obtain a first region optimized image and a second region optimized image;
Comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, combining and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively according to the same optimizing channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed;
the dual-channel super-resolution optimization method comprises the following steps:
interpolation is carried out on pixel values around known pixels in the original image to increase the resolution of the image, and a first optimized image is obtained;
extracting information from the original image and deducing missing pixels to increase the resolution of the image by using an image processing algorithm and a signal processing technology, so as to obtain a second optimized image;
The image quality difference region extraction method comprises the following steps:
constructing a two-classification model by using a deep learning method;
Constructing a training data set containing the quality difference of the marked image, and manually marking the area with the quality difference on the image of the training data set;
training the model using the prepared training data set;
Presetting a first-order threshold value, and determining a significant difference;
Once model training is completed, deducing the first and second optimized images by using a trained network to obtain image difference mapping;
extracting a region of which the image quality difference exceeds a first-order threshold value by comparing pixel values in the difference map with the set first-order threshold value, and obtaining an image quality difference region;
The dual-channel super-resolution optimization method for the original image suspicious region comprises the following steps:
Processing is independently carried out for each suspicious region, and the suspicious region is cut out;
constructing a deep learning model, wherein the model takes a low-resolution suspicious region as input and a high-resolution image as output;
inputting each original image suspicious region into a trained model to generate a first region optimized image and a second region optimized image;
The final optimized image output method comprises the following steps:
performing similarity calculation on the first region optimized image and the second region optimized image by comparing pixel values, structures and brightness differences between the first region optimized image and the second region optimized image by using a similarity measurement method;
Presetting a third threshold value as a standard for judging the similarity of the two areas;
Comparing the calculated similarity with a third threshold value, and judging the similarity of the first region optimized image and the second region optimized image; if the similarity exceeds a preset third threshold value, the optimization results of the two areas are similar, and merging and restoring are carried out; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the fact that the difference of the optimized results of the two regions is large is indicated that the original image has defects and cannot be optimized;
Combining the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively, and carrying out weighted average processing on pixel values of the two regions to obtain an image of finally integrating two region information;
and randomly selecting a restored image as a final optimized image of the original image to output.
2. The depth learning based image quality optimization method of claim 1, wherein the similarity measure comprises a structural similarity index, a mean square error.
3. The depth learning based image quality optimization method of claim 1, wherein the set influencing factors of the second order threshold include image quality criteria, sensitivity of an image processing application, characteristics of an original image, and user experience requirements.
4. An image quality optimization system based on deep learning, the system comprising:
The first double-channel super-resolution optimization module is used for carrying out double-channel super-resolution optimization on the original image, obtaining a first optimized image and a second optimized image, and sending the first optimized image and the second optimized image;
The image quality difference recognition and region extraction module is used for receiving the first optimized image and the second optimized image, carrying out image quality difference recognition on the first optimized image and the second optimized image, extracting an image region with the image quality difference exceeding a preset first-order threshold value from the first optimized image and the second optimized image, obtaining an image quality difference region and sending the image quality difference region;
The image quality difference analysis module is used for receiving the image quality difference area, calculating the ratio between the area of the image quality difference area and the area of the original image, obtaining the image quality difference duty ratio and sending the image quality difference duty ratio;
The decision module is used for receiving the image quality difference duty ratio, comparing the image quality difference duty ratio with a preset second order threshold value, and if the image quality difference duty ratio does not exceed the preset second order threshold value, randomly selecting one of the first optimized image and the second optimized image as a final optimized image of the original image to output; if the image quality difference ratio exceeds a preset second-order threshold value, extracting a region corresponding to the image quality difference region in the original image, obtaining an in-doubt region of the original image, and transmitting the in-doubt region;
The second double-channel super-resolution optimization module is used for receiving the original image in-doubt area, performing double-channel super-resolution optimization on the original image in-doubt area independently, obtaining a first area optimized image and a second area optimized image, and transmitting the first area optimized image and the second area optimized image;
The similarity analysis and image restoration module is used for receiving the first region optimized image and the second region optimized image, comparing the similarity of the first region optimized image with the similarity of the second region optimized image, if the similarity of the first region optimized image and the second region optimized image exceeds a preset third threshold value, respectively merging and restoring the first region optimized image and the second region optimized image with the first optimized image and the second optimized image according to the same optimization channel, and randomly selecting one restored image as a final optimized image of the original image to output; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the defect of the original image is indicated, and optimization cannot be performed;
the double-channel super-resolution optimization method comprises the following steps:
interpolation is carried out on pixel values around known pixels in the original image to increase the resolution of the image, and a first optimized image is obtained;
extracting information from the original image and deducing missing pixels to increase the resolution of the image by using an image processing algorithm and a signal processing technology, so as to obtain a second optimized image;
The image quality difference region extraction method comprises the following steps:
constructing a two-classification model by using a deep learning method;
Constructing a training data set containing the quality difference of the marked image, and manually marking the area with the quality difference on the image of the training data set;
training the model using the prepared training data set;
Presetting a first-order threshold value, and determining a significant difference;
Once model training is completed, deducing the first and second optimized images by using a trained network to obtain image difference mapping;
extracting a region of which the image quality difference exceeds a first-order threshold value by comparing pixel values in the difference map with the set first-order threshold value, and obtaining an image quality difference region;
The dual-channel super-resolution optimization method for the original image suspicious region comprises the following steps:
Processing is independently carried out for each suspicious region, and the suspicious region is cut out;
constructing a deep learning model, wherein the model takes a low-resolution suspicious region as input and a high-resolution image as output;
inputting each original image suspicious region into a trained model to generate a first region optimized image and a second region optimized image;
The final optimized image output method comprises the following steps:
performing similarity calculation on the first region optimized image and the second region optimized image by comparing pixel values, structures and brightness differences between the first region optimized image and the second region optimized image by using a similarity measurement method;
Presetting a third threshold value as a standard for judging the similarity of the two areas;
Comparing the calculated similarity with a third threshold value, and judging the similarity of the first region optimized image and the second region optimized image; if the similarity exceeds a preset third threshold value, the optimization results of the two areas are similar, and merging and restoring are carried out; if the similarity of the first region optimized image and the second region optimized image does not exceed a preset third threshold value, the fact that the difference of the optimized results of the two regions is large is indicated that the original image has defects and cannot be optimized;
Combining the first region optimized image and the second region optimized image with the first optimized image and the second optimized image respectively, and carrying out weighted average processing on pixel values of the two regions to obtain an image of finally integrating two region information;
and randomly selecting a restored image as a final optimized image of the original image to output.
5. A deep learning based image quality optimizing electronic device comprising a bus, a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the transceiver, the memory and the processor being connected by the bus, characterized in that the computer program when executed by the processor realizes the steps in the method according to any of claims 1-3.
6. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method according to any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410077127.7A CN117575916B (en) | 2024-01-19 | 2024-01-19 | Image quality optimization method, system, equipment and medium based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410077127.7A CN117575916B (en) | 2024-01-19 | 2024-01-19 | Image quality optimization method, system, equipment and medium based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117575916A CN117575916A (en) | 2024-02-20 |
| CN117575916B true CN117575916B (en) | 2024-04-30 |
Family
ID=89886780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410077127.7A Active CN117575916B (en) | 2024-01-19 | 2024-01-19 | Image quality optimization method, system, equipment and medium based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117575916B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598857A (en) * | 2020-05-11 | 2020-08-28 | 北京阿丘机器人科技有限公司 | Method and device for detecting surface defects of product, terminal equipment and medium |
| CN111754403A (en) * | 2020-06-15 | 2020-10-09 | 南京邮电大学 | Image super-resolution reconstruction method based on residual learning |
| WO2022057837A1 (en) * | 2020-09-16 | 2022-03-24 | 广州虎牙科技有限公司 | Image processing method and apparatus, portrait super-resolution reconstruction method and apparatus, and portrait super-resolution reconstruction model training method and apparatus, electronic device, and storage medium |
| CN115147274A (en) * | 2022-06-09 | 2022-10-04 | 青岛民航凯亚系统集成有限公司 | Method for acquiring super-resolution image, acquisition system device and storage medium |
| CN115239559A (en) * | 2022-07-21 | 2022-10-25 | 山东大学 | Depth map super-resolution method and system for fusion view synthesis |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11024009B2 (en) * | 2016-09-15 | 2021-06-01 | Twitter, Inc. | Super resolution using a generative adversarial network |
-
2024
- 2024-01-19 CN CN202410077127.7A patent/CN117575916B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598857A (en) * | 2020-05-11 | 2020-08-28 | 北京阿丘机器人科技有限公司 | Method and device for detecting surface defects of product, terminal equipment and medium |
| CN111754403A (en) * | 2020-06-15 | 2020-10-09 | 南京邮电大学 | Image super-resolution reconstruction method based on residual learning |
| WO2022057837A1 (en) * | 2020-09-16 | 2022-03-24 | 广州虎牙科技有限公司 | Image processing method and apparatus, portrait super-resolution reconstruction method and apparatus, and portrait super-resolution reconstruction model training method and apparatus, electronic device, and storage medium |
| CN115147274A (en) * | 2022-06-09 | 2022-10-04 | 青岛民航凯亚系统集成有限公司 | Method for acquiring super-resolution image, acquisition system device and storage medium |
| CN115239559A (en) * | 2022-07-21 | 2022-10-25 | 山东大学 | Depth map super-resolution method and system for fusion view synthesis |
Non-Patent Citations (2)
| Title |
|---|
| 张晗 ; .基于深度学习的图像超分辨率重建应用初探.智能计算机与应用.2020,(05),全文. * |
| 王一宁 ; 秦品乐 ; 李传朋 ; 崔雨豪 ; .基于残差神经网络的图像超分辨率改进算法.计算机应用.2018,(01),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117575916A (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102547365B (en) | Black edge detection method and device for video image | |
| CN110059700A (en) | The recognition methods of image moire fringes, device, computer equipment and storage medium | |
| CN114998310B (en) | Saliency detection method and system based on image processing | |
| CN110110634B (en) | Pathological image multi-staining separation method based on deep learning | |
| CN114782329A (en) | Bearing defect damage degree evaluation method and system based on image processing | |
| JP2021531571A (en) | Certificate image extraction method and terminal equipment | |
| US8482630B2 (en) | Apparatus and method for adjusting automatic white balance by detecting effective area | |
| KR20180109658A (en) | Apparatus and method for image processing | |
| CN105913429A (en) | Calculating method for visual perception response time delay index of intelligent terminal user | |
| CN117575916B (en) | Image quality optimization method, system, equipment and medium based on deep learning | |
| CN112150368A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| US7800783B2 (en) | Auto white balancing by modified grey world and geometric distortion measurement | |
| JP2014007611A (en) | Image processing apparatus and control method therefor | |
| CN114641792A (en) | Image processing method, image processing apparatus, and readable storage medium | |
| US11468269B2 (en) | Arithmetic device, arithmetic method, and discrimination system | |
| CN115546736A (en) | River channel sand collection monitoring processing method and system based on image collection | |
| CN114943634A (en) | Image steganalysis method, model and system | |
| CN115018787A (en) | Anomaly detection method and system based on gradient enhancement | |
| CN113963428A (en) | Model training method, occlusion detection method, system, electronic device, and medium | |
| CN112446428A (en) | Image data processing method and device | |
| EP4276755A1 (en) | Image segmentation method and apparatus, computer device, and readable storage medium | |
| US20230061884A1 (en) | Control method of image signal processor and control device for performing the same | |
| Singh et al. | A review on digital image forensics | |
| EP4254328A1 (en) | Method for evaluating video quality based on non-reference video | |
| CN116309914A (en) | Image signal analysis system and method using layout detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |