WO2022260392A1 - Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal - Google Patents

Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal Download PDF

Info

Publication number
WO2022260392A1
WO2022260392A1 PCT/KR2022/007999 KR2022007999W WO2022260392A1 WO 2022260392 A1 WO2022260392 A1 WO 2022260392A1 KR 2022007999 W KR2022007999 W KR 2022007999W WO 2022260392 A1 WO2022260392 A1 WO 2022260392A1
Authority
WO
WIPO (PCT)
Prior art keywords
artificial neural
neural network
query image
network model
noise
Prior art date
Application number
PCT/KR2022/007999
Other languages
English (en)
Korean (ko)
Inventor
장한힘
권민수
이명제
Original Assignee
주식회사 에너자이
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020210075401A external-priority patent/KR102393759B1/ko
Application filed by 주식회사 에너자이 filed Critical 주식회사 에너자이
Publication of WO2022260392A1 publication Critical patent/WO2022260392A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present invention relates to a method and system for generating an artificial neural network model operating in a terminal, and more particularly, to a method and system for generating a small artificial neural network model that removes noise from an image in a terminal without passing through a server.
  • an artificial neural network model performing a noise removal function generally requires high-specification computing resources because performance is excellent as the size increases. Therefore, artificial neural network models with excellent performance have a disadvantage that cannot be executed in widely used smart terminals.
  • the present invention is to solve the above problems, and an object of the present invention is to provide a method for generating a small artificial neural network model with excellent performance that operates in a smart terminal.
  • Another object of the present invention is to provide a process of learning a large artificial neural network model including a small artificial neural network model in order to ensure noise removal performance of the small artificial neural network model.
  • the present invention provides a method for generating a second artificial neural network model in which an electronic device removes noise applicable to a smart terminal, including steps a of generating a first query image and a second query image; Step b of learning a first artificial neural network model based on the query image and the second query image, and step c of extracting a second artificial neural network model from the first artificial neural network model and transmitting the same to a smart terminal. do.
  • the present invention generates a first query image and a second query image, learns a first artificial neural network model based on the first query image and the second query image, and converts a second artificial neural network model from the first artificial neural network model. It is characterized by having an electronic device including a control module for extracting, a communication module for transmitting the second artificial neural network model to a smart terminal, and a storage module for storing the second artificial neural network model.
  • the performance of the small artificial neural network model can be guaranteed to the level of a large artificial neural network model operating in a server.
  • FIG. 1 is a diagram showing the configuration of a system for learning an artificial neural network model for noise removal in a smart terminal according to an embodiment of the present invention.
  • FIG. 2 is a diagram for explaining a process of learning an artificial neural network model for noise removal in a smart terminal according to an embodiment of the present invention.
  • FIG. 3 is a flowchart illustrating a method of learning an artificial neural network model for noise removal in a smart terminal according to an embodiment of the present invention.
  • FIG. 4 is a diagram for explaining in detail a learning method of a first artificial neural network model according to an embodiment of the present invention.
  • each component may be implemented as a hardware processor, and the above components may be integrated and implemented as one hardware processor, or the above components may be combined with each other and implemented as a plurality of hardware processors.
  • FIG. 1 is a block diagram of a system for learning an artificial neural network model that operates in a smart terminal according to an embodiment of the present invention and performs a function of removing noise from an image.
  • an electronic device 10 of the present invention is a device for learning an artificial neural network model and transmitting it to a smart terminal so that the smart terminal 20 can remove noise, and includes a control module 11 and a communication module. (13), a storage module 15 may be included.
  • the control module 11 may generate and learn a first artificial neural network model so that the first artificial neural network model can identify noise in the query image and remove it.
  • the control module 11 may utilize the training data set to learn the first artificial neural network model.
  • the training data set according to an embodiment of the present invention includes a first query image with noise and a second query image that is the same as the first query image but without noise, and the control module 11 controls the first artificial query image.
  • a first artificial neural network model may be created and trained through the first query image and the second query image so that the neural network model can accurately identify noise.
  • the control module 110 may extract a second artificial neural network model from the first artificial neural network model when it is determined that the first artificial neural network model has a certain level of accuracy or higher.
  • the second artificial neural network model is a part of the first artificial neural network model, and is sufficiently operable even in a smart terminal 20 having low hardware performance due to less required computing resources compared to the first artificial neural network model.
  • the communication module 13 may transmit the second artificial neural network model to the smart terminal 20 .
  • the storage module 15 may include an image database storing a plurality of images for learning the first artificial neural network model and a model database storing the first artificial neural network model.
  • the smart terminal 20 may remove noise from the query image received from the user using the second artificial neural network model received from the electronic device 10, and specifically, the control module 21, the communication module 23, A storage module 25 may be included.
  • the control module 21 may remove noise from the image received from the user by using the second artificial neural network model received from the electronic device 10 .
  • the communication module 23 may receive an image from a user and receive a second artificial neural network model from the electronic device 10 .
  • the storage module 25 may store the second artificial neural network model.
  • the first artificial neural network model 30 shown in FIG. 2 may be learned based on a training data set.
  • the first artificial neural network model 30 shown in FIG. 2 may be stored in the storage module 15 shown in FIG. 1 and executed by the control module 11 .
  • control module 11 may generate a training data set to learn the first artificial neural network model 30 .
  • the control module 11 may generate a training data set using images stored in the image database of the storage module 15 .
  • control module 11 may generate a first query image and a second query image having a predetermined size using images stored in an image database.
  • the control module 11 may generate a first query image of a patch unit having a predetermined size from an image stored in an image database, and may generate a second query image by randomly generating noise in the first query image.
  • the control module 11 may generate various noises for the first query image by using a noise generation algorithm such as a Gaussian noise generation algorithm, and may generate a second query image in addition to the first query image.
  • a noise generation algorithm such as a Gaussian noise generation algorithm
  • the control module 11 may learn the first artificial neural network model 30 based on the first query image and the second query image.
  • the first artificial neural network model 30 is a super network and may be composed of a target network 40 and a main network 50 having a coupling relationship with each other.
  • the target network is formed in a u-net structure, it may be composed of at least one pooling layer and an unpooling layer.
  • the first artificial neural network model 30 generated by the noise removal system of the present invention is characterized in that it can use both microscopic and macroscopic features of the noise extraction target image as it includes a target network having a unitet structure as part of it.
  • the control module 11 uses the second query image as input data to generate noise of the second query image. can be extracted.
  • control module 11 may use the second query image as input data of the target network 40 and extract first noise for the second query image from the target network 40 .
  • the control module 11 may generate a third query image from which the first noise extracted from the second query image is removed through the target network 30 .
  • the control module 11 may use the second query image and the third query image as input data of the main network 50 and extract second noise for the second query image from the main network 50 .
  • the control module 11 may generate a fourth query image obtained by removing the second noise extracted from the second query image through the main network 50 .
  • the control module 11 may generate a third query image and a fourth query image by offsetting the first noise and the second noise in each of the second query images.
  • the control module 11 may calculate a first loss value by comparing the third query image extracted through the target network 40 with the first query image.
  • the control module 11 may learn the target network 40 using the first loss value.
  • the control module 11 may calculate a second loss value by comparing the fourth query image extracted through the main network 50 with the first query image.
  • the control module 11 may learn the super network 30 using the first loss value and the second loss value. That is, the control module 11 may use both the first loss value and the second loss value to generate a loss value more effective for learning of the target network 40 included in the super network 30 .
  • control module 11 may use both the first loss value and the second loss value in learning the super network 50, and may specifically assign a weight to each of the first loss value and the second loss value. have.
  • the noise removal system according to an embodiment of the present invention is applied through the target network 40 (the second artificial neural network model), a higher weight may be assigned to the first loss value for the target network 40 .
  • the control module 11 may use feature values between query images to calculate the first loss value and the second loss value.
  • control module 11 may use feature values of the first query image and the third query image to calculate the first loss value.
  • the control module 11 may calculate a first loss value by subtracting feature values of the first query image and the third query image.
  • the control module 11 may use feature values of the first query image and the fourth query image to calculate the second loss value.
  • the control module 11 may calculate a second loss value by subtracting feature values of the first query image and the fourth query image.
  • the control module 11 may set the difference between two query images to be compared as a loss value by subtracting the feature value of each query image, so that the difference between the two query images is minimized, that is, the loss value is the minimum value.
  • the target network 40 and the super network 50 may be learned.
  • the control module 11 sets a weight for the second loss value to 0 and a weight for the first loss value to 1 so that the target Only network 40 can be learned. Accordingly, the performance of the target network 40 itself can be further improved.
  • the control module 11 may extract the target network 40 and set it as a second artificial neural network model when the first loss value and the second loss value are greater than or equal to a preset second loss threshold.
  • FIG. 3 is a flowchart illustrating a method of learning an artificial neural network model for noise removal in a smart terminal according to an embodiment of the present invention.
  • a method of learning an artificial neural network model for noise removal will be described with reference to FIG. 3 .
  • detailed embodiments overlapping with the system for learning the artificial neural network model for noise elimination described above may be omitted.
  • the electronic device may generate a training data set.
  • the electronic device may generate a training data set by using an image stored in an image database in order to generate and train the first artificial neural network model.
  • the electronic device may generate a first query image and a second query image having a preset size using images stored in an image database.
  • the electronic device may generate a first query image in a patch unit having a preset size from an image stored in an image database, and may randomly add noise to the first query image to generate a second query image.
  • the electronic device may generate various noises for the first query image by using a noise generation algorithm such as a Gaussian noise generation algorithm and generate a second query image in addition to the first query image.
  • a noise generation algorithm such as a Gaussian noise generation algorithm
  • the electronic device may learn a first artificial neural network model based on a training data set including a first query image and a second query image.
  • the electronic device may use a super network as a first neural network model, and the super network may include a target network and a main network having a coupling relationship with each other.
  • FIG. 4 is a diagram for explaining in detail a learning method of a first artificial neural network model according to an embodiment of the present invention.
  • the electronic device may extract the first noise for the second query image from the target network by using the second query image as input data of the target network.
  • the electronic device may generate a third query image from which the first noise extracted from the second query image is removed through the target network.
  • the electronic device may use the second query image and the third query image as input data of the main network and extract second noise for the second query image from the main network.
  • the electronic device may generate a fourth query image obtained by removing the second noise extracted from the second query image through the main network.
  • the electronic device may create a third query image and a fourth query image by offsetting the first noise and the second noise in each of the second query images.
  • the electronic device may calculate a first loss value by comparing the third query image extracted through the target network with the first query image.
  • the electronic device may calculate a second loss value by comparing the fourth query image extracted through the main network with the first query image.
  • the electronic device may learn the target network and the super network using the first loss value and/or the second loss value.
  • the electronic device may learn the target network using the first loss value and learn the super network using the first loss value and the second loss value.
  • the electronic device may use both the first loss value and the second loss value in order to generate a loss value more effective for learning the target network included in the super network.
  • the electronic device may learn the super network by assigning weights to each of the first loss value and the second loss value.
  • a higher weight may be assigned to the first loss value for the target network.
  • the electronic device may use feature values between query images to calculate the first loss value and the second loss value.
  • the electronic device may use feature values of the first query image and the third query image to calculate the first loss value.
  • the electronic device may calculate a first loss value by subtracting feature values of the first query image and the third query image.
  • the electronic device may use feature values of the first query image and the fourth query image to calculate the second loss value.
  • the electronic device may calculate a second loss value by subtracting feature values of the first query image and the fourth query image.
  • the electronic device may set the difference between two query images to be compared as a loss value by subtracting the feature value of each query image, and the target network such that the difference between the two query images is minimized, that is, the loss value becomes the minimum value. and super networks.
  • the electronic device learns only the target network by setting the weight for the second loss value to 0 and the weight for the first loss value to 1. can do. Accordingly, the performance of the target network itself can be further improved.
  • the electronic device may extract a target network and set it as a second artificial neural network model. Specifically, when the first loss value and the second loss value are equal to or greater than a preset second loss threshold, the electronic device may extract the target network and set it as the second artificial neural network model.
  • the electronic device may transmit the second artificial neural network model to the smart terminal.
  • the smart terminal may provide a noise removal service to the user through the small-sized second artificial neural network model.
  • the control module 21 of the smart terminal controls the operation of the second artificial neural network model based on the CPU, NPU or GPU.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé et un système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal et l'objectif de la présente invention est de fournir un procédé d'entraînement d'un petit modèle de réseau neuronal artificiel ayant une excellente performance fonctionnant dans un terminal intelligent. La présente invention pour atteindre un tel objectif est caractérisée par un procédé par lequel un dispositif électronique génère un second modèle de réseau neuronal artificiel qui élimine le bruit, le second modèle de réseau neuronal artificiel pouvant être appliqué à un terminal intelligent, le procédé comprenant : une étape a consistant à générer une première image d'interrogation et une seconde image d'interrogation ; une étape b consistant à entraîner un premier modèle de réseau neuronal artificiel sur la base de la première image d'interrogation et de la seconde image d'interrogation ; et une étape c consistant à extraire le second modèle de réseau neuronal artificiel à partir du premier modèle de réseau neuronal artificiel et le transmettre au terminal intelligent.
PCT/KR2022/007999 2021-06-10 2022-06-07 Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal WO2022260392A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR1020210075401A KR102393759B1 (ko) 2021-06-10 2021-06-10 단말에서 동작하는 이미지 프로세싱 인공 신경망 모델 생성 방법 및 시스템
KR10-2022-0052703 2021-06-10
KR10-2021-0075401 2021-06-10
KR1020220052703A KR20220166712A (ko) 2021-06-10 2022-04-28 단말에서 동작하는 이미지 프로세싱 인공 신경망 모델 생성 방법 및 시스템

Publications (1)

Publication Number Publication Date
WO2022260392A1 true WO2022260392A1 (fr) 2022-12-15

Family

ID=84426378

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/007999 WO2022260392A1 (fr) 2021-06-10 2022-06-07 Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal

Country Status (2)

Country Link
KR (1) KR20220166712A (fr)
WO (1) WO2022260392A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20240082759A (ko) 2022-12-02 2024-06-11 주식회사 엘지에너지솔루션 Ctp 타입의 고에너지 밀도 전지팩

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101871098B1 (ko) * 2017-01-12 2018-06-25 포항공과대학교 산학협력단 이미지 처리 방법 및 장치
KR20190119548A (ko) * 2019-10-02 2019-10-22 엘지전자 주식회사 이미지 노이즈 처리방법 및 처리장치
KR20210048736A (ko) * 2019-10-24 2021-05-04 망고슬래브 주식회사 머신 러닝 기반 디블러 엔진
KR102393759B1 (ko) * 2021-06-10 2022-05-06 주식회사 에너자이(ENERZAi) 단말에서 동작하는 이미지 프로세싱 인공 신경망 모델 생성 방법 및 시스템

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101871098B1 (ko) * 2017-01-12 2018-06-25 포항공과대학교 산학협력단 이미지 처리 방법 및 장치
KR20190119548A (ko) * 2019-10-02 2019-10-22 엘지전자 주식회사 이미지 노이즈 처리방법 및 처리장치
KR20210048736A (ko) * 2019-10-24 2021-05-04 망고슬래브 주식회사 머신 러닝 기반 디블러 엔진
KR102393759B1 (ko) * 2021-06-10 2022-05-06 주식회사 에너자이(ENERZAi) 단말에서 동작하는 이미지 프로세싱 인공 신경망 모델 생성 방법 및 시스템

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BAO LONG; YANG ZENGLI; WANG SHUANGQUAN; BAI DONGWOON; LEE JUNGWON: "Real Image Denoising Based on Multi-Scale Residual Dense Block and Cascaded U-Net with Block-Connection", 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), IEEE, 14 June 2020 (2020-06-14), pages 1823 - 1831, XP033799114, DOI: 10.1109/CVPRW50498.2020.00232 *
SEYED-IMAN MIRZADEH; MEHRDAD FARAJTABAR; ANG LI; HASSAN GHASEMZADEH: "Improved Knowledge Distillation via Teacher Assistant: Bridging the Gap Between Student and Teacher", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 9 February 2019 (2019-02-09), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081027190 *

Also Published As

Publication number Publication date
KR20220166712A (ko) 2022-12-19

Similar Documents

Publication Publication Date Title
WO2020111314A1 (fr) Appareil et procédé d'interrogation-réponse basés sur un graphe conceptuel
WO2018151503A2 (fr) Procédé et appareil destinés à la reconnaissance de gestes
WO2022260392A1 (fr) Procédé et système pour générer un modèle de réseau neuronal artificiel de traitement d'image fonctionnant dans un terminal
WO2022131497A1 (fr) Appareil d'apprentissage et procédé de génération d'image, et appareil et procédé de génération d'image
WO2022146080A1 (fr) Algorithme et procédé de modification dynamique de la précision de quantification d'un réseau d'apprentissage profond
WO2021225294A1 (fr) Appareil et procédé d'apprentissage par transfert utilisant une technique de régularisation à base d'échantillons
WO2013157705A1 (fr) Procédé de déduction d'un centre d'intérêt d'un utilisateur par l'intermédiaire des centres d'intérêt de voisins sociaux et de thèmes d'activités sociales dans des sns, et système à cet effet
WO2021125517A1 (fr) Système d'intelligence artificielle dédié
WO2019190076A1 (fr) Procédé de suivi des yeux et terminal permettant la mise en œuvre dudit procédé
WO2023277448A1 (fr) Procédé et système d'entraînement de modèle de réseau neuronal artificiel pour traitement d'image
WO2022139325A1 (fr) Système informatique pour apprentissage adaptatif multi-domaine basé sur un réseau neuronal unique sans sur-apprentissage, et procédé associé
WO2021101052A1 (fr) Procédé et dispositif de détection de trame d'action fondée sur un apprentissage faiblement supervisé, à l'aide d'une suppression de trame d'arrière-plan
WO2023113372A1 (fr) Appareil et procédé d'extraction d'échantillon reposant sur une étiquette pour amélioration de la performance d'un modèle de classification d'apprentissage profond pour données déséquilibrées
WO2023132597A1 (fr) Système d'apprentissage fédéré basé sur un groupe local et procédé de commande d'apprentissage fédéré
WO2023149660A1 (fr) Procédé et système d'apprentissage fédéré basé sur une signature de groupe, et support d'enregistrement pour la mise en œuvre de ceux-ci
KR102393759B1 (ko) 단말에서 동작하는 이미지 프로세싱 인공 신경망 모델 생성 방법 및 시스템
WO2020101121A1 (fr) Procédé d'analyse d'image basée sur l'apprentissage profond, système et terminal portable
WO2023033194A1 (fr) Procédé et système de distillation de connaissances spécialisés pour l'éclaircissement de réseau neuronal profond à base d'élagage
WO2021071258A1 (fr) Dispositif et procédé d'apprentissage d'image de sécurité mobile basés sur l'intelligence artificielle
WO2018012685A1 (fr) Appareil de vérification des performances d'un dispositif électronique intelligent, système de vérification des performances d'un dispositif électronique intelligent, et support d'enregistrement lisible par ordinateur
WO2022131404A1 (fr) Système et procédé d'analyse de données sur dispositif
WO2022145769A1 (fr) Procédé et appareil permettant de calculer une qualité d'image au moyen d'une classification d'image
WO2019208869A1 (fr) Appareil et procédé de détection des caractéristiques faciales à l'aide d'un apprentissage
WO2021256603A1 (fr) Procédé de production de modèle de détection d'intrusion utilisant un modèle à apprentissage profond
WO2024010200A1 (fr) Procédé et dispositif d'inférence de modèle d'ia

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22820520

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22820520

Country of ref document: EP

Kind code of ref document: A1