WO2022145769A1 - Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image - Google Patents

Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image Download PDF

Info

Publication number
WO2022145769A1
WO2022145769A1 PCT/KR2021/018219 KR2021018219W WO2022145769A1 WO 2022145769 A1 WO2022145769 A1 WO 2022145769A1 KR 2021018219 W KR2021018219 W KR 2021018219W WO 2022145769 A1 WO2022145769 A1 WO 2022145769A1
Authority
WO
WIPO (PCT)
Prior art keywords
quality index
image
label
neural network
query image
Prior art date
Application number
PCT/KR2021/018219
Other languages
English (en)
Korean (ko)
Inventor
송철환
Original Assignee
오드컨셉 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 오드컨셉 주식회사 filed Critical 오드컨셉 주식회사
Publication of WO2022145769A1 publication Critical patent/WO2022145769A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

  • the present invention relates to a method and apparatus for calculating image quality through image classification, and more particularly, to a method and apparatus for calculating an image quality index by adjusting a weight of an image quality operation based on a label for an image.
  • the image filtering technology may use the quality index for each image, but the conventional technology for calculating the quality index has a limitation that the quality determined by the computer and the user may be different depending on the type of image. For example, the computer determines that the quality of the general photo and the pictorial photo is the same, but the user may recognize that the pictorial photo is of higher quality than the general photo.
  • An object of the present invention is to solve the above-mentioned problem, and to calculate the quality index of an image by varying the weight according to the type of the image.
  • the present invention for achieving this object is a step A of receiving at least one query image from a user terminal, a step B of generating a feature vector by applying a first neural network model to the query image, and the query based on the feature vector. It is characterized in that it comprises a step C of identifying a label of the image and calculating a first quality index, and a step D of calculating a third quality index of the query image using the label and the first quality index.
  • the present invention provides a query image input module that receives at least one query image from a user terminal, generates a feature vector by applying a first neural network model to the query image, and identifies a label of the query image based on the feature vector and an image analysis module for calculating a first quality index, and a quality index calculation module for calculating a third quality index of the query image using the label and the first quality index.
  • the present invention as described above, it is possible to calculate the quality index of the image by changing the weight according to the type of the image. Through this, the gap between the image quality judged by the computer and the image quality judged by humans can be narrowed.
  • FIG. 1 is a block diagram showing the configuration of an image quality calculating device according to an embodiment of the present invention
  • FIG. 2 is a flowchart for explaining an image quality calculation method according to an embodiment of the present invention.
  • FIG. 3 is a flowchart illustrating a process of learning a neural network model for image quality calculation according to an embodiment of the present invention.
  • each of the components may be implemented as a hardware processor, the above components may be integrated into one hardware processor, or the above components may be combined with each other and implemented as a plurality of hardware processors.
  • FIG. 1 is a diagram for explaining the configuration of an image quality calculating apparatus according to an embodiment of the present invention.
  • the image quality calculating device 10 may receive at least one query image and calculate a quality index for the query image.
  • the image quality calculating device 10 may analyze the query image to identify a label corresponding to the query image, and calculate a quality index by changing a weight according to the identified label.
  • the image quality calculation device 10 may include a query image input module 100 , an image analysis module 200 , and a quality index calculation module 300 .
  • the image quality calculation device 10 may process operations of the query image input module 100 , the image analysis module 200 , and the quality index calculation module 300 through at least one processor.
  • the query image input module 100 may receive at least one query image from the user terminal.
  • the image analysis module 200 may identify a label corresponding to the received query image. Specifically, the image analysis module 200 may classify the query image by extracting a feature vector from the query image and identifying a label of the query image based on the feature vector.
  • the first neural network model will be used.
  • the first neural network model is trained based on machine learning.
  • the first neural network model according to an embodiment of the present invention will be based on a convolutional neural network (CNN).
  • Convolutional neural networks are a class of multilayer perceptrons designed to use minimal preprocessing.
  • a convolutional neural network consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and pooling layers. Thanks to this structure, the convolutional neural network can fully utilize the input data of the two-dimensional structure.
  • the image analysis module 200 may include an encoder 230 and a decoder 260 as the first neural network model based on CNN is used.
  • the encoder 230 may generate a feature vector representing detailed features in the received query image, and the decoder 260 may reconstruct data from the feature vector using a deconvolution layer.
  • the encoder 230 of the image analysis module 200 includes a convolution layer, an activation function layer (Relu layer), a dropout layer, and a max pooling layer (Max- It can be created by combining pooling layers).
  • the encoder 230 may use a conventional method such as a Scale Invariant Feature Transform (SIFT) algorithm to extract a feature vector of the received query image.
  • SIFT Scale Invariant Feature Transform
  • the decoder 260 may be generated by combining an upsampling layer, a deconvolution layer, a sigmoid layer, and a dropout layer.
  • the decoder 260 may identify a label corresponding to the query image based on the feature vector corresponding to the query image, and further calculate a first quality index of the query image.
  • the decoder 260 may normalize the feature vector by applying a softmax function to the feature vector of the query image.
  • the softmax function is a function that provides normalization of the output value so that it can classify the output value used in the artificial neural network.
  • the decoder 260 may apply a softmax function to classify the label of the query image based on the feature vector.
  • the decoder 260 may identify a label corresponding to the query image using a result of applying the softmax function to the feature vector.
  • a label according to an embodiment of the present invention may include a portrait pictorial, an object (animal/landscape) pictorial, a cover photo, a portrait interview photo, a general photo, a general photo + text, a captured photo, a low-quality photo, and the like. Labels may be further added or deleted according to the administrator's settings.
  • the decoder 260 when an interview picture for Mr. A is input as a query image, the decoder 260 generates a feature vector for the query image and applies the softmax function to ⁇ (personal picture, 13%), (object ( Animal/Landscape) pictorial, 2%), (Cover photo, 7%), (person interview photo, 54%), (Normal photo, 17%), (Normal photo+text, 3%), (Capture photo, 3 %), (low quality photo, 1%) ⁇ .
  • the decoder 260 will select the label with the highest probability as the label corresponding to the query image.
  • the decoder 260 may further calculate the first quality index of the query image based on the feature vector.
  • the quality index is a value indicating the degree of quality of the query image, and the decoder 260 according to an embodiment of the present invention will use a conventional method in calculating the first quality index.
  • the quality index calculation module 300 may calculate a final quality index (third quality index) based on the label corresponding to the query image identified by the image analysis module 200 and the first quality index.
  • the quality index calculation module 300 may calculate the third quality index by using the first quality index and the second quality index according to the label of the query image.
  • the quality index calculation module 300 may vary the second quality index according to the label of the query image.
  • the quality index calculation module 300 according to an embodiment of the present invention may select a second quality index within the range by varying the range of the second quality index according to the priority of a table preset by the user.
  • the range of the second quality index of the person picture is (4, 5)
  • the range of the second quality index of the object (animal/landscape) picture is (3.7, 4.7)
  • the range of the second quality index of the cover picture is (3.5)
  • the range of the second quality index of interview photos is (3, 4)
  • the range of the second quality index of general photos is (2.5, 3.5)
  • the range of the second quality index of general photos + text is ( 2, 3)
  • the range of the second quality index of the captured photo may be set to (1, 2)
  • the range of the second quality index of the low-quality photo may be set to (0, 1).
  • the second quality index corresponding to each label according to an embodiment of the present invention will be selected by the second neural network model within a preset range as described above.
  • the second neural network model may precede learning with a training data set to select a second quality index.
  • the learning process of the second neural network model will be described in detail below.
  • the quality index calculation module 300 is a third quality index (final quality index) based on the first quality index of the query image calculated in the first neural network model for the query image and the second quality index calculated in the second neural network model can create
  • the image analysis module 200 and the quality index calculation module 300 may be trained through the following process.
  • the image analysis module 200 and the quality index calculation module 300 may be learned through supervised learning.
  • supervised learning a model is trained in a state in which a label (correct answer) for training data is given.
  • the image analysis module 200 may receive a training data set from the user terminal.
  • the training data set will include a training image, a first label (type of image) of the training image, and a fourth quality index of the training image.
  • the training data may have the form (training image, first label, fourth quality index).
  • the image analysis module 200 may generate a feature vector for the training image through the first neural network model, and identify a second label corresponding to the training data based on this.
  • the image analysis module 200 may compare with the first label for the training image included in the training data set to determine whether the second label is correctly identified.
  • the image analysis module 200 will train the first neural network model by giving a positive feedback to the first neural network model if the first label and the second label are the same, and a negative feedback if not the same.
  • the quality index calculation module 300 may calculate the final quality index (the fifth quality index) for the training data by using the second neural network model.
  • the quality index calculation module 300 may calculate a fifth quality index for the training image through the feature vector and the second label generated by the image analysis module 200 through the training data set.
  • the quality index calculation module 300 may calculate a loss (loss) based on the fifth quality index for the training image and the fourth quality index included in the training data set.
  • the quality index calculation module 300 may train the second neural network model by adjusting a parameter for selecting the second quality index for each label of the second neural network model based on the loss value.
  • the image analysis module 200 and the quality index calculation module 300 may more accurately calculate the third quality index for the query image.
  • FIG. 2 is a flowchart illustrating an image quality calculation method according to an embodiment of the present invention.
  • an image quality calculation method will be described with reference to FIG. 2 .
  • a detailed embodiment overlapping with the image quality calculation apparatus described above may be omitted.
  • the image quality calculating device may receive at least one query image from the user terminal ( S110 ).
  • the electronic device may analyze the received query image to identify a label corresponding to the query image, and calculate a quality index by varying a weight according to the label.
  • the electronic device may use the first neural network model to analyze the query image to identify a label corresponding to the query image, and use the second neural network model to calculate a quality index by changing a weight according to the label.
  • the electronic device may extract a feature vector of the received query image (S120).
  • the electronic device may use a conventional method such as a Scale Invariant Feature Transform (SIFT) algorithm to extract a feature vector of a query image.
  • SIFT Scale Invariant Feature Transform
  • the electronic device may identify a label corresponding to the query image based on the feature vector ( S130 ).
  • the electronic device may identify a label corresponding to the query image by normalizing the feature vector by applying a Softmax function to the feature vector.
  • the electronic device may select a label having the highest probability among the result values of the softmax function for the feature vector as a label corresponding to the query image.
  • the label according to an embodiment of the present invention may include a person picture, an object (animal/landscape) picture, a cover picture, a person interview picture, a general picture, a general picture + text, a captured picture, a low quality picture, etc. It may be added or deleted according to the setting of .
  • the electronic device may further calculate the first quality index of the feature vector of the query image extracted in step 120 ( S135 ).
  • the quality index is a value indicating the degree of quality of the query image, and the electronic device will use a conventional method in calculating the first quality index.
  • the electronic device may calculate a final quality index (third quality index) based on the label of the query image obtained in steps 130 and 135 and the first quality index ( S140 ).
  • the electronic device may calculate the third quality index by using the first quality index and the second quality index according to the label of the query image.
  • the electronic device may vary the second quality index according to the label of the query image.
  • the electronic device according to an embodiment of the present invention may select a second quality index within the range by varying the range of the second quality index according to the priority of a label preset by the user.
  • the quality index of an image generated through the image quality calculation method according to an embodiment of the present invention may be applied to services such as image filtering and ranking.
  • the electronic device may learn the neural network model through supervised learning, and referring to FIG. 3 , the electronic device may receive a training data set from the user terminal ( S210 ).
  • the training data set will include a training image, a first label (type of image) of the training image, and a fourth quality index of the training image.
  • the training data may have the form (training image, first label, fourth quality index).
  • the electronic device may generate a feature vector for the training image through the first neural network model, and identify a second label corresponding to the training data based on this ( S220 ).
  • the electronic device may determine whether the second label is correctly identified by comparing the second label with the first label for the training image included in the training data set ( S230 ).
  • the electronic device will train the first neural network model by giving a positive feedback to the first neural network model if the first label and the second label are the same, and a negative feedback to the first neural network model if they are not identical ( S240 ).
  • the electronic device may calculate a final quality index (fifth quality index) for the training data using the second neural network model ( S250 ).
  • the electronic device may calculate a fifth quality index for the training image based on the feature vector and the second label generated by the image analysis module 200 through the training data set.
  • the electronic device may calculate a loss (loss) based on the fifth quality index for the training image and the fourth quality index included in the training data set ( S260 ).
  • the electronic device may train the second neural network model by adjusting a parameter for selecting the second quality index for each label of the second neural network model based on the loss value ( S270 ).
  • the electronic device may more accurately calculate the third quality index for the query image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention se rapporte à un procédé et à un appareil permettant de calculer un indice de qualité d'image et un objectif de la présente invention est de calculer un indice de qualité d'une image en faisant varier le poids selon le type d'image. À cette fin, la présente invention comprend : l'étape A consistant à recevoir au moins une image d'interrogation en provenance d'un terminal d'utilisateur ; l'étape B consistant à générer un vecteur de caractéristiques en appliquant un premier modèle de réseau neuronal à l'image d'interrogation ; l'étape C consistant à identifier une étiquette de l'image d'interrogation sur la base du vecteur de caractéristiques et à calculer un premier indice de qualité ; et l'étape D consistant à calculer un troisième indice de qualité de l'image d'interrogation en utilisant l'étiquette et le premier indice de qualité.
PCT/KR2021/018219 2021-01-04 2021-12-03 Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image WO2022145769A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2021-0000283 2021-01-04
KR1020210000283A KR20220098504A (ko) 2021-01-04 2021-01-04 이미지 분류를 통한 이미지 퀄리티 연산 방법 및 장치

Publications (1)

Publication Number Publication Date
WO2022145769A1 true WO2022145769A1 (fr) 2022-07-07

Family

ID=82260490

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2021/018219 WO2022145769A1 (fr) 2021-01-04 2021-12-03 Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image

Country Status (2)

Country Link
KR (1) KR20220098504A (fr)
WO (1) WO2022145769A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170079852A (ko) * 2015-12-31 2017-07-10 네이버 주식회사 이미지 압축 품질을 최적화 하기 위한 방법 및 시스템
KR20190019822A (ko) * 2017-08-18 2019-02-27 삼성전자주식회사 이미지의 시맨틱 분리를 위한 시스템 및 방법
US20200401889A1 (en) * 2018-03-19 2020-12-24 Samsung Electronics Co., Ltd. Electronic device, image processing method of electronic device, and computer-readable medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170079852A (ko) * 2015-12-31 2017-07-10 네이버 주식회사 이미지 압축 품질을 최적화 하기 위한 방법 및 시스템
KR20190019822A (ko) * 2017-08-18 2019-02-27 삼성전자주식회사 이미지의 시맨틱 분리를 위한 시스템 및 방법
US20200401889A1 (en) * 2018-03-19 2020-12-24 Samsung Electronics Co., Ltd. Electronic device, image processing method of electronic device, and computer-readable medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DODGE SAMUEL; KARAM LINA: "Understanding how image quality affects deep neural networks", 2016 EIGHTH INTERNATIONAL CONFERENCE ON QUALITY OF MULTIMEDIA EXPERIENCE (QOMEX), 6 June 2016 (2016-06-06), pages 1 - 6, XP032916292, DOI: 10.1109/QoMEX.2016.7498955 *
GUAN JINGWEI; CHAM WAI-KUEN: "Distortion based image quality index", 2016 ASIA-PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION ANNUAL SUMMIT AND CONFERENCE (APSIPA), ASIA PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION, 13 December 2016 (2016-12-13), pages 1 - 4, XP033044870, DOI: 10.1109/APSIPA.2016.7820899 *

Also Published As

Publication number Publication date
KR20220098504A (ko) 2022-07-12

Similar Documents

Publication Publication Date Title
WO2018217019A1 (fr) Dispositif de détection d'un code malveillant variant sur la base d'un apprentissage de réseau neuronal, procédé associé, et support d'enregistrement lisible par ordinateur dans lequel un programme d'exécution dudit procédé est enregistré
WO2017213398A1 (fr) Modèle d'apprentissage pour détection de région faciale saillante
WO2020145691A1 (fr) Procédé et système d'optimisation d'image faisant appel à l'intelligence artificielle
WO2022260386A1 (fr) Procédé et appareil permettant de composer un arrière-plan et un visage en utilisant un réseau d'apprentissage profond
WO2020222391A1 (fr) Système et procédé pour couche d'ondelettes réversible pour réseaux neuronaux
WO2022145769A1 (fr) Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image
CN116758379B (zh) 一种图像处理方法、装置、设备及存储介质
WO2021101052A1 (fr) Procédé et dispositif de détection de trame d'action fondée sur un apprentissage faiblement supervisé, à l'aide d'une suppression de trame d'arrière-plan
CN111507396A (zh) 缓解神经网络对未知类样本产生错误分类的方法及装置
WO2023158068A1 (fr) Système et procédé d'apprentissage pour améliorer le taux de détection d'objets
WO2021071258A1 (fr) Dispositif et procédé d'apprentissage d'image de sécurité mobile basés sur l'intelligence artificielle
WO2022108275A1 (fr) Procédé et dispositif de génération d'un visage virtuel à l'aide de l'intelligence artificielle
WO2020101121A1 (fr) Procédé d'analyse d'image basée sur l'apprentissage profond, système et terminal portable
WO2022145787A1 (fr) Procédé et dispositif de classification multi-étiquettes sur la base d'un masque
WO2022131720A1 (fr) Dispositif et procédé pour générer une image de construction
WO2022092672A1 (fr) Procédé pour ajouter des résultats de prédiction en tant que données d'apprentissage à l'aide d'un modèle de prédiction par ia
WO2011007970A1 (fr) Procédé et appareil pour traiter une image
WO2022260392A1 (fr) Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal
CN113191376A (zh) 图像处理方法、装置、电子设备和可读存储介质
WO2020213757A1 (fr) Procédé de détermination de similarité de mots
WO2023214633A1 (fr) Procédé et dispositif d'amélioration d'une qualité d'image sur la base d'un réseau neuronal à super-résolution
WO2022114324A1 (fr) Appareil et procédé de génération de vidéo sur la base d'un langage naturel
WO2022145788A1 (fr) Procédé et appareil de détection d'image redondante
WO2022182163A1 (fr) Procédé et dispositif de fourniture automatique de traitement de données, de génération de modèles d'intelligence artificielle et d'amélioration de performances
WO2020175729A1 (fr) Appareil et procédé pour détecter un point de caractéristique faciale à l'aide d'une carte de points caractéristiques gaussiens et d'un schéma de régression

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21915567

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21915567

Country of ref document: EP

Kind code of ref document: A1