CN112508856B - Distortion type detection method for mixed distortion image - Google Patents

Distortion type detection method for mixed distortion image Download PDF

Info

Publication number
CN112508856B
CN112508856B CN202011278437.3A CN202011278437A CN112508856B CN 112508856 B CN112508856 B CN 112508856B CN 202011278437 A CN202011278437 A CN 202011278437A CN 112508856 B CN112508856 B CN 112508856B
Authority
CN
China
Prior art keywords
distortion
mixed
image
distortion type
type detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011278437.3A
Other languages
Chinese (zh)
Other versions
CN112508856A (en
Inventor
侯舒娟
窦博文
李海
张钦
宋政育
武毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202011278437.3A priority Critical patent/CN112508856B/en
Publication of CN112508856A publication Critical patent/CN112508856A/en
Application granted granted Critical
Publication of CN112508856B publication Critical patent/CN112508856B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

The distortion type detection method of the mixed distortion image comprises the steps of inputting the mixed distortion image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distortion image, wherein M is a positive integer; and inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distorted image, and obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector. The detection and discrimination tasks of the mixed distorted image can be completed, the feature extraction module is simple, and the related distortion type can be correspondingly adjusted according to the specific image processing task.

Description

Distortion type detection method for mixed distortion image
Technical Field
The disclosure belongs to the technical field of digital image processing, and particularly relates to a distortion type detection method for a mixed distortion image.
Background
In the information age, with the popularization of smart mobile terminal devices such as smart phones, digital images become one of important carrier forms of visual information, and have wide application in the fields of medical treatment, military, entertainment, weather and the like. Digital images can be distorted in different types and different degrees due to various reasons in the processes of acquisition, processing, transmission, storage and the like, for example, noise generated in the image acquisition process; compression distortion occurring during compression; partial loss of images due to information loss during transmission; and poor image resolution due to storage device resource limitations. These distortions can degrade the user experience and may be detrimental to subsequent processing or applications. Therefore, the research of image reconstruction has important significance, and accurate detection and discrimination of the image distortion type are very important for selection of a reconstruction algorithm.
At present, image distortion detection and discrimination algorithms can be generally divided into two categories: a method based on conventional signal processing and a method based on deep learning. The former mainly extracts the characteristic values of the distorted image through various signal processing means, and then detects and judges the distorted image based on the characteristic values; the latter mainly uses a deep learning network to fit a large amount of data, so that the data can autonomously extract the characteristics of distorted images and carry out image distortion detection and judgment.
In 2010, Anush Krishna Moorthy et al propose a BIQI algorithm for Image Quality assessment in A Two-Step frame for structuring blade Image Quality industries, wherein Image distortion detection and discrimination are involved. In 2012, Anush et al proposed the DIIVENE model in the "Blind Image Quality Assessment Assembly From Natural Scene Statistics to Perceptial Quality" for Image Quality Assessment. Wavelet decomposition is adopted, the decomposed wavelet coefficients are subjected to statistical processing to obtain characteristic values of the distorted image, and then a Support Vector Machine (SVM) is used for detecting and distinguishing the distorted image. The method has high accuracy, but can only detect a single distortion type, is not suitable for the mixed distortion condition, and has a complex characteristic extraction process.
In addition, in 2012, BRISQUE framework proposed by "No-Reference Image Quality Assessment in the Spatial Domain", 2016, Wu meiyin et al proposed that Image Distortion detection and Classification be implemented using convolutional network in "video Image Distortion detection and Classification based on convolutional neural network", and in 2018, Classification of Image Distortion types based on normalized coefficients of Spatial local mean and contrast difference in "Feature Selection for Image Distortion Classification" by Saad b.
In 2016, Alaql O et al studied feature extraction models and different combination modes in the existing Image Quality Assessment algorithm, and found an optimal feature extraction mode, which can reduce the number of feature values and achieve the purpose of model simplification. However, this method is based on a conventional signal processing method, and is not suitable for detection of the type of image mixing distortion.
In an actual application scene, distortion types in a distorted image are often mixed, and the number of the distortion types is random, but the existing image distortion detection and discrimination algorithms only aim at a single distorted image, and the problem of detection and discrimination of mixed distortion in the image cannot be effectively solved.
Disclosure of Invention
In view of this, the present disclosure provides a distortion type detection method for a mixed distorted image, which can complete detection and discrimination tasks of the mixed distorted image, has a simple feature extraction module, and can correspondingly adjust the related distortion type according to a specific image processing task.
According to an aspect of the present disclosure, a distortion type detection method of a mixed distorted image is provided, the method including:
inputting the mixed distorted image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distorted image, wherein M is a positive integer;
and inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distorted image, and obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector.
In one possible implementation, the distortion types include four distortion types, noise, image missing, low resolution, and JPEG compression distortion.
In one possible implementation, the mixed distorted image includes at least two of four distortion types.
In one possible implementation, obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector includes:
the value of each element of the distortion type detection result vector is 1 or-1, and when the value of the element of the distortion type detection result vector is 1, the mixed distorted image contains the distortion type corresponding to the element;
otherwise, when the value of the element of the distortion type detection result vector is-1, the mixed distorted image does not contain the distortion type corresponding to the element.
The distortion type detection method of the mixed distortion image comprises the steps of inputting the mixed distortion image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distortion image, wherein M is a positive integer; and inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distorted image, and obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector. The detection and discrimination tasks of the mixed distorted image can be completed, the feature extraction module is simple, and the related distortion type can be correspondingly adjusted according to the specific image processing task.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
FIG. 1 illustrates a flow diagram of a method for distortion type detection of a hybrid distorted image according to an embodiment of the present disclosure;
FIG. 2 shows a flowchart of a distortion type detection method for a mixed distorted image according to another embodiment of the present disclosure;
FIG. 3 shows a schematic diagram of a residual error network architecture according to another embodiment of the present disclosure;
fig. 4 shows a schematic structural diagram of a residual block of a residual network according to another embodiment of the present disclosure;
fig. 5 shows a schematic structural diagram of a multi-label classifier according to another embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
The invention discloses a distortion type detection method of a mixed distortion image, which is a mixed distortion image detection and discrimination algorithm based on a residual error network and multi-label classification. The mixed distortion image set comprises four distortion types of low resolution, image missing, noise and JEPG compression distortion. The number and the introduction sequence of the distortion types contained in the mixed distortion image set are random.
Fig. 1 shows a flowchart of a distortion type detection method for a mixed distorted image according to an embodiment of the present disclosure. As shown in fig. 1, the method may include:
step S1: and inputting the mixed distorted image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distorted image.
The mixed distortion image may be a color image or a grayscale image, and is not limited herein.
The mixed distortion image can be a digital image captured by an image collector (such as a camera, a CMOS camera, a video camera, etc.) under different application scenes, or an image obtained through channel transmission, or an image obtained through different processing, etc. There are different numbers of different types of distortion types in the mixed distorted image.
In one example, the distortion types may include four distortion types, noise, image dropout, low resolution, and JPEG compression distortion. And the mixed distorted image includes at least two of the four distortion types.
Wherein, the noise is additive white Gaussian noise with zero mean value; the image deletion is covered or shielded by a white strip pattern; the low-resolution distortion is an image obtained by performing double-cubic interpolation and downsampling on an original image twice.
The number of distortion types and the introduced distortion sequence contained in the mixed distortion image are random, namely 6 images simultaneously containing two distortion types are random combinations of any two of four distortion types including noise, image missing, low resolution and JPEG compression distortion. The 4 kinds of images containing three distortion types are random combinations of any three of four distortion types including noise, image deletion, low resolution and JPEG compression distortion. There are 1 image containing 4 distortion types at the same time. And is not limited herein. Of course, there may be 4 images with a single distortion type in the mixed distortion image, each including one of four distortion types, namely noise, image missing, low resolution and JPEG compression distortion. The detection and identification of the image distortion type in this case can be performed not only by the method of the present disclosure, but also by an existing image distortion type detection method. The present disclosure is mainly directed to detecting and identifying the mixed distorted image containing multiple distortion types.
And inputting the mixed distortion image into a Residual error Network (ResNet), extracting the characteristics of N layers to obtain an M-dimensional characteristic vector, and inputting the M-dimensional characteristic vector into a multi-label classifier. Wherein N and M are both positive integers.
Step S2: and inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distorted image, and obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector.
Fig. 5 shows a schematic structural diagram of a multi-label classifier according to another embodiment of the present disclosure.
In one example, as shown in fig. 5, the value of each element of the distortion type detection result vector may be set to 1 or-1, and when the value of an element of the distortion type detection result vector is 1, the mixed distortion image contains the distortion type corresponding to the element; otherwise, when the value of the element of the distortion type detection result vector is-1, the mixed distortion image does not contain the distortion type corresponding to the element.
Of course, the value of the element of the distortion type detection result vector may be-1 when the mixed distorted image contains the distortion type corresponding to the element, and the value of the element of the distortion type detection result vector may be 1 when the mixed distorted image does not contain the distortion type corresponding to the element. The values of the elements of the distortion type detection result vector may be set to other values, which is not limited herein.
Application example:
fig. 2 shows a flowchart of a distortion type detection method of a mixed distorted image according to another embodiment of the present disclosure.
As shown in fig. 2, a mixed distortion image containing any two distortion types among four distortion types of noise, image deletion, low resolution, and JPEG compression distortion is input into a residual network.
FIG. 3 shows a schematic diagram of a residual error network architecture according to another embodiment of the present disclosure; fig. 4 shows a schematic structural diagram of a residual block of a residual network according to another embodiment of the present disclosure.
As shown in fig. 3, the residual network structure adopts a 34-layer residual network to perform feature extraction on an input mixed distortion image. As shown in fig. 4, each residual block of the residual network contains two convolutional layers, and thus, a residual network of 34 layers contains 17 residual blocks. In the residual error network, each iteration layer is activated by using a ReLU function, and 2048-dimensional feature vectors of the mixed distortion image are obtained after 34 layers of residual error network processing.
As shown in fig. 5, the multi-label classifier adopts a fully connected perceptron, takes 2048-dimensional feature vector values of the mixed distorted image extracted by the residual error network as input, and takes a detection result vector of the distortion type of the mixed distorted image as output. The loss function adopted by the FC layer of the fully-connected perceptron is SoftMarginLoss, the loss function has multi-label classification learning capacity, and the final obtained result is four distortion types of output noise, image deletion, low resolution and JPEG compression distortion of the multi-label classifier shown in figure 5, namely a four-dimensional distortion type detection result vector, wherein each element (noise, image deletion, low resolution or JPEG compression distortion) is 1 or-1, 1 represents that the mixed distortion image contains a corresponding distortion type, and-1 represents that the mixed distortion image does not contain a corresponding distortion type. The detection and the discrimination of the image of the mixed distortion type can be realized through the value of each element in the distortion type detection result vector.
The distortion type detection method of the mixed distortion image comprises the steps of inputting the mixed distortion image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distortion image, wherein M is a positive integer; and inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distorted image, and obtaining the distortion type of the mixed distorted image according to the value of the distortion type detection result vector. The detection and discrimination tasks of the mixed distorted image can be completed, the feature extraction module is simple, and the related distortion type can be correspondingly adjusted according to the specific image processing task.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (4)

1. A distortion type detection method for a mixed distortion image, the method comprising:
inputting the mixed distorted image into a residual error network for feature extraction to obtain an M-dimensional feature vector of the mixed distorted image, wherein M is a positive integer;
inputting the M-dimensional feature vector into a multi-label classifier to obtain a distortion type detection result vector of the mixed distortion image;
obtaining the distortion type of the mixed distorted image according to the value of each element in the distortion type detection result vector, wherein when the value of the element is equal to a first set value, the mixed distorted image contains the distortion type corresponding to the element; and when the distortion type is equal to the second set value, the mixed distortion image does not contain the distortion type corresponding to the element.
2. A distortion type detection method according to claim 1, wherein the distortion types include four distortion types of noise, image dropout, low resolution, and JPEG compression distortion.
3. A distortion type detection method according to claim 2, wherein the mixed distorted image includes at least two of the four distortion types.
4. The distortion type detection method according to claim 1, wherein obtaining the distortion type of the mixed distorted image based on the setting values of the elements of the distortion type detection result vector comprises:
the value of each element of the distortion type detection result vector is 1 or-1, and when the value of the element of the distortion type detection result vector is 1, the mixed distortion image contains the distortion type corresponding to the element;
otherwise, when the value of the element of the distortion type detection result vector is-1, the mixed distortion image does not contain the distortion type corresponding to the element.
CN202011278437.3A 2020-11-16 2020-11-16 Distortion type detection method for mixed distortion image Active CN112508856B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011278437.3A CN112508856B (en) 2020-11-16 2020-11-16 Distortion type detection method for mixed distortion image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011278437.3A CN112508856B (en) 2020-11-16 2020-11-16 Distortion type detection method for mixed distortion image

Publications (2)

Publication Number Publication Date
CN112508856A CN112508856A (en) 2021-03-16
CN112508856B true CN112508856B (en) 2022-09-09

Family

ID=74958033

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011278437.3A Active CN112508856B (en) 2020-11-16 2020-11-16 Distortion type detection method for mixed distortion image

Country Status (1)

Country Link
CN (1) CN112508856B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113592807B (en) * 2021-07-28 2024-04-09 北京世纪好未来教育科技有限公司 Training method, image quality determining method and device and electronic equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102333233A (en) * 2011-09-23 2012-01-25 宁波大学 Stereo image quality objective evaluation method based on visual perception
CN107770517A (en) * 2017-10-24 2018-03-06 天津大学 Full reference image quality appraisement method based on image fault type

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104754359B (en) * 2015-01-26 2017-07-21 清华大学深圳研究生院 A kind of depth map encoding distortion prediction method of Two Dimensional Free viewpoint video
CN106780446A (en) * 2016-12-01 2017-05-31 重庆邮电大学 It is a kind of to mix distorted image quality evaluating method without reference

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102333233A (en) * 2011-09-23 2012-01-25 宁波大学 Stereo image quality objective evaluation method based on visual perception
CN107770517A (en) * 2017-10-24 2018-03-06 天津大学 Full reference image quality appraisement method based on image fault type

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于一类分类的成本信息失真检测模型构建;甄晓非等;《统计与决策》;20130530(第10期);全文 *

Also Published As

Publication number Publication date
CN112508856A (en) 2021-03-16

Similar Documents

Publication Publication Date Title
Athar et al. A comprehensive performance evaluation of image quality assessment algorithms
Bondi et al. Tampering detection and localization through clustering of camera-based CNN features
CN110084757B (en) Infrared depth image enhancement method based on generation countermeasure network
CN109636754B (en) Extremely-low-illumination image enhancement method based on generation countermeasure network
WO2020246861A1 (en) Method and apparatus for training neural network model for enhancing image detail
CN108376392B (en) Image motion blur removing method based on convolutional neural network
Luo et al. A survey of passive technology for digital image forensics
Akshatha et al. Digital camera identification using PRNU: A feature based approach
Villalba et al. Smartphone image clustering
CN110782399A (en) Image deblurring method based on multitask CNN
CN110210498B (en) Digital image equipment evidence obtaining system based on residual learning convolution fusion network
CN111008651B (en) Image reproduction detection method based on multi-feature fusion
Hu et al. A two-stage unsupervised approach for low light image enhancement
CN109034230B (en) Single image camera tracing method based on deep learning
CN111369548A (en) No-reference video quality evaluation method and device based on generation countermeasure network
CN112508856B (en) Distortion type detection method for mixed distortion image
Fan et al. Global sensing and measurements reuse for image compressed sensing
Mehrish et al. Robust PRNU estimation from probabilistic raw measurements
CN108550111B (en) Residual error example regression super-resolution reconstruction method based on multi-level dictionary learning
Behare et al. Source camera identification using photo response noise uniformity
Chen et al. Color image splicing localization algorithm by quaternion fully convolutional networks and superpixel-enhanced pairwise conditional random field
El-Yamany et al. A generic approach CNN-based camera identification for manipulated images
CN112767311A (en) Non-reference image quality evaluation method based on convolutional neural network
Madake et al. Image Tampering Detection Using Error Level Analysis and Metadata Analysis
CN111951199A (en) Image fusion method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant