CN118710501A - Training method of image super-resolution reconstruction system, image reconstruction method and system - Google Patents
Training method of image super-resolution reconstruction system, image reconstruction method and system Download PDFInfo
- Publication number
- CN118710501A CN118710501A CN202411186704.2A CN202411186704A CN118710501A CN 118710501 A CN118710501 A CN 118710501A CN 202411186704 A CN202411186704 A CN 202411186704A CN 118710501 A CN118710501 A CN 118710501A
- Authority
- CN
- China
- Prior art keywords
- image
- task
- reconstructed
- degradation
- prompt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 title claims abstract description 56
- 230000015556 catabolic process Effects 0.000 claims abstract description 120
- 238000006731 degradation reaction Methods 0.000 claims abstract description 120
- 230000006870 function Effects 0.000 claims abstract description 37
- 238000013507 mapping Methods 0.000 claims description 94
- 239000011159 matrix material Substances 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 15
- 230000000694 effects Effects 0.000 claims description 9
- 238000010276 construction Methods 0.000 abstract 1
- 230000003044 adaptive effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004088 simulation Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 3
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 3
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 2
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 2
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000002595 magnetic resonance imaging Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
技术领域Technical Field
本发明涉及计算机视觉技术领域,具体涉及一种图像超分辨率重建系统的训练方法、图像重建方法及系统。The present invention relates to the field of computer vision technology, and in particular to a training method for an image super-resolution reconstruction system, an image reconstruction method and a system.
背景技术Background Art
图像超分辨率技术,作为一种提升图像质量的关键手段,已在自然图像分析、医学成像等多个领域发挥着举足轻重的作用。其根本目的是从低分辨率图像中复原高分辨率细节,增强视觉体验和分析准确性。尽管众多方法为图像超分辨率领域带来了显著进步,但这些方法大多针对特定场景定制,难以跨越场景和图像类型实现良好的泛化能力。此外,不断变化的数据需求迫使模型需不断学习新的场景,此过程对计算资源构成了巨大挑战。Image super-resolution technology, as a key means to improve image quality, has played a vital role in many fields such as natural image analysis and medical imaging. Its fundamental purpose is to restore high-resolution details from low-resolution images to enhance visual experience and analysis accuracy. Although many methods have brought significant progress in the field of image super-resolution, most of these methods are customized for specific scenarios and it is difficult to achieve good generalization capabilities across scenarios and image types. In addition, the ever-changing data requirements force the model to continuously learn new scenarios, which poses a huge challenge to computing resources.
近年来,基于提示的持续学习策略在图像分类等任务上取得了一定成就,其中使用多层感知器作为自适应提示生成器,能够有效平衡历史任务的知识保存和新任务的学习。但在面对超分辨率任务的像素级精度要求或者复杂的退化模式时,该方法效果受限。传统的提示生成策略难以在高效覆盖退化任务的多维度细节需求的同时维护已有知识与新任务学习之间的平衡。连续图像超分辨率领域图像数据间的显著差异,特别是低级别特征上的不一致性,如纹理和色彩,也加剧了跨任务迁移学习的难度。加之图像退化机制的多样性和复杂性,从简单插值到医学成像中的专业退化,进一步扩大了任务间的领域鸿沟。In recent years, cue-based continuous learning strategies have achieved certain success in tasks such as image classification. Among them, the use of multi-layer perceptrons as adaptive cue generators can effectively balance the knowledge preservation of historical tasks and the learning of new tasks. However, this method is limited in effectiveness when faced with the pixel-level accuracy requirements of super-resolution tasks or complex degradation patterns. Traditional cue generation strategies have difficulty maintaining a balance between existing knowledge and new task learning while efficiently covering the multi-dimensional detail requirements of degradation tasks. The significant differences between image data in the field of continuous image super-resolution, especially the inconsistency in low-level features such as texture and color, also increase the difficulty of cross-task transfer learning. In addition, the diversity and complexity of image degradation mechanisms, from simple interpolation to specialized degradation in medical imaging, further widens the domain gap between tasks.
发明内容Summary of the invention
为了解决现有技术中存在的上述问题,本发明提供了一种图像超分辨率重建系统的训练方法、图像重建方法及系统,具体包括:In order to solve the above problems existing in the prior art, the present invention provides a training method, an image reconstruction method and a system for an image super-resolution reconstruction system, which specifically include:
第一方面,本发明提供了一种图像超分辨率重建系统的训练方法,应用于图像超分辨率重建系统,图像超分辨率重建系统包括骨干网络和提示映射生成模块;In a first aspect, the present invention provides a training method for an image super-resolution reconstruction system, which is applied to the image super-resolution reconstruction system, wherein the image super-resolution reconstruction system includes a backbone network and a prompt map generation module;
该方法,包括:The method comprises:
针对任一退化任务:For any degenerate task:
获取退化任务对应的多个待重建图像;Acquire multiple images to be reconstructed corresponding to the degradation task;
基于骨干网络,根据各待重建图像和预先确定的退化任务的任务标识,训练提示映射生成模块,得到退化任务对应的提示映射函数,进而得到训练好的图像超分辨率重建系统。Based on the backbone network, according to the task identifiers of each image to be reconstructed and the predetermined degradation task, a prompt mapping generation module is trained to obtain the prompt mapping function corresponding to the degradation task, and then a trained image super-resolution reconstruction system is obtained.
第二方面,本发明提供了一种图像重建方法,包括:In a second aspect, the present invention provides an image reconstruction method, comprising:
获取待重建图像;Acquire an image to be reconstructed;
提取待重建图像的查询特征;Extracting query features of the image to be reconstructed;
根据待重建图像的查询特征,以及预先确定的多项退化任务对应的查询特征组,确定待重建图像对应的任务;Determining the task corresponding to the image to be reconstructed according to the query feature of the image to be reconstructed and the query feature group corresponding to the plurality of pre-determined degradation tasks;
将待重建图像的查询特征和待重建图像对应的任务,输入根据如第一方面所提供的任一图像超分辨率重建系统的训练方法训练得到的图像超分辨率重建系统中的提示映射生成模块,以使提示映射生成模块根据待重建图像的查询特征和待重建图像对应的任务,以及预先训练得到各项退化任务对应的提示映射函数,得到待重建图像对应的提示信息;Inputting the query features of the image to be reconstructed and the tasks corresponding to the image to be reconstructed into a prompt map generation module in the image super-resolution reconstruction system trained according to the training method of any image super-resolution reconstruction system provided in the first aspect, so that the prompt map generation module obtains prompt information corresponding to the image to be reconstructed according to the query features of the image to be reconstructed and the tasks corresponding to the image to be reconstructed, and the prompt mapping functions corresponding to the degradation tasks obtained in advance through training;
将待重建图像的查询特征和待重建图像对应的提示映射函数,输入根据如第一方面所提供的任一图像超分辨率重建系统的训练方法训练得到的图像超分辨率重建系统中的骨干网络,以使骨干网络根据待重建图像的查询特征和待重建图像对应的提示信息,重建待重建图像。The query features of the image to be reconstructed and the prompt mapping function corresponding to the image to be reconstructed are input into the backbone network in the image super-resolution reconstruction system trained according to the training method of any image super-resolution reconstruction system provided in the first aspect, so that the backbone network reconstructs the image to be reconstructed according to the query features of the image to be reconstructed and the prompt information corresponding to the image to be reconstructed.
第三方面,本发明还提供了一种图像超分辨率重建系统,包括:In a third aspect, the present invention further provides an image super-resolution reconstruction system, comprising:
图像预处理模块、任务匹配模块、提示映射模块和骨干网络;Image preprocessing module, task matching module, cue mapping module and backbone network;
图像预处理模块,用于获取待重建图像,并提取待重建图像的查询特征;An image preprocessing module, used to obtain the image to be reconstructed and extract query features of the image to be reconstructed;
任务匹配模块,根据图像预处理模块提取的待重建图像的查询特征,以及存储的预先确定的多项退化任务对应的查询特征组,确定待重建图像对应的任务;A task matching module determines the task corresponding to the image to be reconstructed according to the query features of the image to be reconstructed extracted by the image preprocessing module and the query feature group corresponding to the plurality of pre-determined degradation tasks stored;
提示映射模块,用于根据图像预处理模块所提取的待重建图像的查询特征、任务匹配模块所确定的待重建图像对应的任务,以及预先训练得到各项退化任务对应的提示映射函数,得到待重建图像对应的提示信息;A prompt mapping module is used to obtain prompt information corresponding to the image to be reconstructed based on the query features of the image to be reconstructed extracted by the image preprocessing module, the task corresponding to the image to be reconstructed determined by the task matching module, and the prompt mapping functions corresponding to various degradation tasks obtained by pre-training;
骨干网络,用于根据图像预处理模块所提取的待重建图像的查询特征、提示映射模块得到的待重建图像对应的提示信息,重建待重建图像。The backbone network is used to reconstruct the image to be reconstructed according to the query features of the image to be reconstructed extracted by the image preprocessing module and the prompt information corresponding to the image to be reconstructed obtained by the prompt mapping module.
本发明的有益效果:Beneficial effects of the present invention:
本发明提供的图像超分辨率重建系统的训练方法、图像重建方法及系统,该训练方法通过针对任一退化任务获取退化任务对应的多个待重建图像,基于骨干网络,根据各待重建图像和预先确定的退化任务的任务标识,训练提示映射生成模块,得到退化任务对应的提示映射函数,进而得到训练好的图像超分辨率重建系统,该训练好的图像超分辨率重建系统能够实现动态构造自适应提示信息,针对不同的退化任务自适应地提供定制化的上下文信息,即使在保持特征多样性不变的情况下,亦能显著减少所需的模型训练参数量,提高了模型的灵活性,显著增强了模型的适应性与运算效率,还有效减轻了在跨领域应用中常见的知识遗忘现象,实现了持续学习。The present invention provides a training method for an image super-resolution reconstruction system, an image reconstruction method and a system. The training method obtains a plurality of to-be-reconstructed images corresponding to any degradation task, and based on a backbone network, trains a prompt mapping generation module according to the task identifiers of the to-be-reconstructed images and predetermined degradation tasks, obtains a prompt mapping function corresponding to the degradation task, and then obtains a trained image super-resolution reconstruction system. The trained image super-resolution reconstruction system can dynamically construct adaptive prompt information, adaptively provide customized context information for different degradation tasks, and significantly reduce the required model training parameter amount even when keeping feature diversity unchanged, thereby improving the flexibility of the model, significantly enhancing the adaptability and computational efficiency of the model, and effectively alleviating the common knowledge forgetting phenomenon in cross-domain applications, thereby realizing continuous learning.
以下将结合附图及实施例对本发明做进一步详细说明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本发明提供的一种图像超分辨率重建系统的训练方法的流程示意图;FIG1 is a schematic flow chart of a training method for an image super-resolution reconstruction system provided by the present invention;
图2为本发明提供的一种基于窗口的Transformer模块的架构示意图;FIG2 is a schematic diagram of the architecture of a window-based Transformer module provided by the present invention;
图3为本发明提供的一种图像重建方法的流程示意图;FIG3 is a schematic flow chart of an image reconstruction method provided by the present invention;
图4为本发明提供的一种图像重建系统的架构示意图。FIG. 4 is a schematic diagram of the architecture of an image reconstruction system provided by the present invention.
具体实施方式DETAILED DESCRIPTION
下面结合具体实施例对本发明做进一步详细的描述,但本发明的实施方式不限于此。The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.
图1为本发明提供的一种图像超分辨率重建系统的训练方法的流程示意图,该方法应用于图像超分辨率重建系统,图像超分辨率重建系统包括骨干网络和提示映射生成模块。FIG1 is a flow chart of a training method for an image super-resolution reconstruction system provided by the present invention. The method is applied to an image super-resolution reconstruction system, and the image super-resolution reconstruction system includes a backbone network and a prompt map generation module.
如图1所示,针对任一退化任务,该方法包括:As shown in FIG1 , for any degradation task, the method includes:
S101、获取退化任务对应的多个待重建图像。S101. Acquire a plurality of to-be-reconstructed images corresponding to a degradation task.
退化任务对应不同的图像退化情况。The degradation tasks correspond to different image degradation situations.
图像退化是指图像在形成、记录、处理和传输过程中,由于成像系统、记录设备、传输介质和处理方法等的不完善,导致获取的图像的质量有所下降。这种现象表现为图像变得模糊、失真或含有噪声等。图像退化的原因多种多样,包括目标或拍摄装置的移动导致的运动模糊、长时间曝光引起的模糊、焦点没对准、光角引起的模糊、大气扰动引起的模糊等。此外,曝光时间太短导致拍摄装置捕获的光子太少也会引起图像退化。图像退化不仅影响图像的清晰度和细节,还可能掩盖图像中的重要信息,从而影响图像的分析和应用。Image degradation refers to the decrease in the quality of the image acquired during the formation, recording, processing and transmission of the image due to the imperfections of the imaging system, recording equipment, transmission medium and processing method. This phenomenon manifests itself as the image becoming blurred, distorted or noisy. There are many reasons for image degradation, including motion blur caused by the movement of the target or shooting device, blur caused by long exposure, focus misalignment, blur caused by light angle, blur caused by atmospheric disturbance, etc. In addition, image degradation can also occur if the exposure time is too short and the shooting device captures too few photons. Image degradation not only affects the clarity and details of the image, but may also cover up important information in the image, thereby affecting the analysis and application of the image.
S102、基于骨干网络,根据各待重建图像和预先确定的退化任务的任务标识,训练提示映射生成模块,得到退化任务对应的提示映射函数,进而得到训练好的图像超分辨率重建系统。S102, based on the backbone network, according to each image to be reconstructed and the task identifier of the predetermined degradation task, training the prompt mapping generation module, obtaining the prompt mapping function corresponding to the degradation task, and then obtaining a trained image super-resolution reconstruction system.
通过该方法能够训练得到每个退化任务对应的提示映射函数,建立退化任务和提示映射函数之间的对应关系。This method can be used to train the prompt mapping function corresponding to each degradation task and establish the corresponding relationship between the degradation task and the prompt mapping function.
示例性的,在S102之前,先确定并存储多项退化任务对应的查询特征组,以及各个查询特征组对应的任务标识,从而建立退化任务、查询特征组以及任务标识三者之间的对应关系。进而,得到待重建图像的查询特征后,将待重建图像的查询特征与各项退化任务对应的查询特征组进行匹配,若匹配成功,则确定待重建图像属于与其匹配成功的退化任务。再进一步,根据退化任务和任务标识的对应关系,确定待重建图像对应的任务标识。Exemplarily, before S102, the query feature groups corresponding to the plurality of degradation tasks and the task identifiers corresponding to the query feature groups are first determined and stored, so as to establish a correspondence between the degradation tasks, the query feature groups and the task identifiers. Then, after obtaining the query features of the image to be reconstructed, the query features of the image to be reconstructed are matched with the query feature groups corresponding to the degradation tasks. If the match is successful, it is determined that the image to be reconstructed belongs to the degradation task that is successfully matched therewith. Furthermore, according to the correspondence between the degradation tasks and the task identifiers, the task identifier corresponding to the image to be reconstructed is determined.
确定多项退化任务对应的查询特征组和任务标识时,可以是依次确定每项退化任务对应的查询特征组和任务标识,也可以是同步确定不同退化任务的查询特征组和任务标识。When determining the query feature groups and task identifiers corresponding to multiple degradation tasks, the query feature groups and task identifiers corresponding to each degradation task may be determined in sequence, or the query feature groups and task identifiers of different degradation tasks may be determined simultaneously.
在一种可能的实现方式中,确定退化任务的任务标识,包括如下步骤A1-A3:In a possible implementation, determining the task identifier of the degraded task includes the following steps A1-A3:
A1、分别获取多项退化任务对应的多个训练图像。A1. Obtain multiple training images corresponding to multiple degradation tasks respectively.
A2、分别提取各训练图像的查询特征,以得到各退化任务对应的查询特征组。A2. Extract query features of each training image respectively to obtain a query feature group corresponding to each degradation task.
具体的,分别提取每一项退化任务对应的多个训练图像的查询特征,每一项退化任务对应的全部训练图像的查询特征即为该任务对应的查询特征组。Specifically, query features of multiple training images corresponding to each degradation task are extracted respectively, and the query features of all training images corresponding to each degradation task are the query feature group corresponding to the task.
A3、对各任务查询特征组分别进行聚类,并将各个聚类中心确定为相应退化任务的任务标识。A3. Cluster each task query feature group separately, and determine each cluster center as the task identifier of the corresponding degenerate task.
可选的,采用K-Means分类算法对各任务查询特征组分别进行聚类,每个聚类中心代表一类图像特征的概括。Optionally, a K-Means classification algorithm is used to cluster each task query feature group, and each cluster center represents a summary of a class of image features.
示例性的,源自第项退化任务中的第幅训练图像的查询特征表示为,其中,表示特征空间的维数,表示维度为特征空间。随后,运用K-Means分类算法,将来自第项退化任务的所有个训练图像的查询特征划分为个聚类群体。任务标识则由个聚类中心构成,表示第项退化任务对应的训练图像的总数。For example, from The first of the degradation tasks training images The query feature is expressed as ,in, represents the dimension of the feature space, The dimension is Then, the K-Means classification algorithm is used to classify the All degenerate tasks The query features of training images Divide into The task identifier is The cluster centers are composed of Indicates The total number of training images corresponding to the degradation task.
通过该方法确定的任务标识,不仅有效区分了不同退化任务的特点,还能用于精确匹配输入的待重建图像至相应的任务类别,确保了任务识别的高效性和准确性。The task identification determined by this method not only effectively distinguishes the characteristics of different degraded tasks, but can also be used to accurately match the input image to be reconstructed to the corresponding task category, ensuring the efficiency and accuracy of task identification.
进一步的,在一种可能的实现方式中,基于骨干网络,根据各待重建图像和预先确定的退化任务的任务标识,训练提示映射生成模块,得到退化任务对应的提示映射函数,包括如下步骤B1-B5:Furthermore, in a possible implementation, based on the backbone network, according to each image to be reconstructed and the task identifier of the predetermined degradation task, a prompt mapping generation module is trained to obtain a prompt mapping function corresponding to the degradation task, including the following steps B1-B5:
B1、在第n轮训练中,提取第n个待重建图像的查询特征,第n个待重建图像为任一待重建图像。B1. In the nth round of training, a query feature of the nth image to be reconstructed is extracted, and the nth image to be reconstructed is any image to be reconstructed.
其中,n为大于等于1的正整数。Wherein, n is a positive integer greater than or equal to 1.
可选的,提取待重建图像的查询特征,包括:通过命令行插图处理器(ContrastiveLanguage-Image Pretraining,CLIP),即CLIP图像编辑器,提取待重建图像的查询特征。Optionally, extracting the query features of the image to be reconstructed includes: extracting the query features of the image to be reconstructed through a command line illustration processor (Contrastive Language-Image Pretraining, CLIP), that is, a CLIP image editor.
其中,CLIP图像编码器最后一层线性变换的归一化输出被界定为单张图像的查询特征。Among them, the normalized output of the last layer of linear transformation of the CLIP image encoder is defined as the query feature of a single image.
B2、根据第n个待重建图像的查询特征、预先确定的退化任务的任务标识和第n个提示映射基,构建第n个提示映射函数。B2. Constructing an nth prompt mapping function according to the query feature of the nth image to be reconstructed, the task identifier of the predetermined degradation task and the nth prompt mapping basis.
其中,第n个提示映射基为第n-1轮训练得到的提示映射基。Among them, the nth prompt mapping basis is the prompt mapping basis obtained by the n-1th round of training.
具体的,提示映射基为线性变换器,表现为大小为的二维矩阵,表示提示映射基的输入维度,表示提示映射基的输出维度。Specifically, the hint mapping basis is a linear transformer, which is expressed as A two-dimensional matrix, represents the input dimension of the hint mapping basis, Represents the output dimension of the hint map basis.
在一种可能的实现方式中,提示映射基为线性变换器,提示映射基满足:In a possible implementation, the hint mapping basis is a linear transformer, and the hint mapping basis satisfies:
, ,
其中,表示第项退化任务的第个任务标识对应的提示映射基,表示维度分别为、和的实数域的三维矩阵,表示基于窗口的Transformer模块的总层数,表示提示映射基的输入维度,表示提示映射基的输出维度。in, Indicates The first step of the degradation task The prompt mapping base corresponding to the task identifier, The dimensions are , and The three-dimensional matrix of the real number field, represents the total number of layers of the window-based Transformer module, represents the input dimension of the hint mapping basis, Represents the output dimension of the hint map basis.
为了缓解可训练参数膨胀问题,可以将进一步的将上述实现方式中的提示映射基替换为两个低秩矩阵,具体的,在另一种可能的实现方式中,提示映射基,表示为:In order to alleviate the problem of trainable parameter expansion, the prompt mapping basis in the above implementation can be further replaced with two low-rank matrices. Specifically, in another possible implementation, the prompt mapping basis is expressed as:
, ,
其中,,,表示第项退化任务的第个任务标识对应的提示映射基,表示第项退化任务的第个任务标识对应的权重矩阵,表示第项退化任务的各个任务标识对应的共享矩阵,表示中间变量,远小于和,表示基于窗口的Transformer模块的总层数,表示提示映射基的输入维度,表示提示映射基的输出维度,表示维度分别为、和的实数域的三维矩阵,表示维度分别为、和的实数域的三维矩阵,表示各退化任务对应的任务标识的索引,表示退化任务的索引。in, , , Indicates The first step of the degradation task The prompt mapping base corresponding to the task identifier, Indicates The first step of the degradation task The weight matrix corresponding to the task identifier is Indicates The shared matrix corresponding to each task identifier of the degenerate task, represents the intermediate variable, Much smaller than and , represents the total number of layers of the window-based Transformer module, represents the input dimension of the hint mapping basis, represents the output dimension of the hint mapping basis, The dimensions are , and The three-dimensional matrix of the real number field, The dimensions are , and The three-dimensional matrix of the real number field, Represents the index of the task identifier corresponding to each degradation task, Indicates the index of the degenerate task.
通过该方法,不仅实现了训练参数量的大幅缩减,还巧妙利用矩阵分解赋予了参数更强的表达力。具体的,作为各项退化任务标识特定的权重矩阵,专注于捕捉各项退化任务内部多层次的细节知识,而退化任务共享矩阵则汇聚了一项退化任务内的所有聚类簇间的泛化性信息,形成了一套高效的任务级知识框架。通过将中间维度R设置得远小于原输入维度和输出维度的值,能够保证仅需极少量的训练数据即可生成提示映射基,进一步促进了模型的轻量化、提升了学习训练的效率。This method not only achieves a significant reduction in the number of training parameters, but also cleverly uses matrix decomposition to give the parameters stronger expressiveness. Specifically, As a specific weight matrix for each degradation task, it focuses on capturing the multi-level detailed knowledge within each degradation task, while the degradation task shares the matrix It gathers the generalization information between all clusters in a degenerate task and forms an efficient task-level knowledge framework. and output dimensions The value of can ensure that only a very small amount of training data is needed to generate the prompt mapping basis, which further promotes the lightweight of the model and improves the efficiency of learning and training.
进一步的,在一种可能的实现方式中,提示映射函数,表示为:Furthermore, in a possible implementation, the prompt mapping function is expressed as:
, ,
其中,表示第项退化任务的第个图像对应的提示映射函数,表示和的内积,表示第项退化任务的第个图像对应的查询特征,表示第项退化任务的第个任务标识,表示第项退化任务的第个任务标识对应的提示映射基,表示各项退化任务对应的图像的索引,表示退化任务的索引,表示各退化任务对应的任务标识的索引,。in, Indicates The first step of the degradation task The prompt mapping function corresponding to the image, express and The inner product of Indicates The first step of the degradation task The query features corresponding to the images are Indicates The first step of the degradation task A task identifier, Indicates The first step of the degradation task The prompt mapping base corresponding to the task identifier, Represents the index of the image corresponding to each degradation task, represents the index of the degenerate task, Represents the index of the task identifier corresponding to each degradation task, .
该方法通过巧妙融合多粒度的任务标识和低秩提示映射基,以及共享的低秩矩阵的设计,构建了一个强大的机制来对抗灾难性遗忘。这一设计不仅促进了新知识的高效吸收,同时有效保留了对先前任务的学习成果,确保模型在不断进化的同时,维持全面的任务适应性和长期的性能稳定性。This method builds a powerful mechanism to combat catastrophic forgetting by cleverly integrating multi-granular task identification and low-rank cue mapping bases, as well as the design of a shared low-rank matrix. This design not only promotes the efficient absorption of new knowledge, but also effectively retains the learning results of previous tasks, ensuring that the model maintains comprehensive task adaptability and long-term performance stability while continuously evolving.
B3、基于骨干网络,根据第n个待重建图像的查询特征和第n个提示映射函数,重构第n个待重建图像。B3. Based on the backbone network, reconstruct the nth image to be reconstructed according to the query features of the nth image to be reconstructed and the nth prompt mapping function.
具体的,将第n个待重建图像的查询特征和第n个提示映射函数输入骨干网络,得到重构的第n个待重建图像。Specifically, the query feature of the nth image to be reconstructed and the nth prompt mapping function are input into the backbone network to obtain a reconstructed nth image to be reconstructed.
SwinIR骨架模型能够灵活适应各种输入图像的超分辨率处理,SwinIR骨干模型由一个头部卷积,一个含有长距离跳跃残差连接的特征增强模块,以及一个基于像素洗牌操作的上采样模块组成。特征增强模块包括六个特征增强阶段,每个阶段包含六个基于窗口的Transformer模块。在每个Transformer模块中,线性层能够提取自我注意机制中涉及的查询、键和值参数。The SwinIR skeleton model can flexibly adapt to the super-resolution processing of various input images. The SwinIR backbone model consists of a head convolution, a feature enhancement module with long-distance skip residual connections, and an upsampling module based on pixel shuffle operations. The feature enhancement module includes six feature enhancement stages, each of which contains six window-based Transformer modules. In each Transformer module, the linear layer can extract the query, key, and value parameters involved in the self-attention mechanism.
基于上述特性,可以采用SwinIR骨架模型作为骨干网络。Based on the above characteristics, the SwinIR skeleton model can be used as the backbone network.
通常持续学习是基于分类任务,不能直接迁移到图像任务,为了专门适应退化任务,本发明通过提示映射函数来调整键和值变量,从而使持续学习能够适配到图像领域。Usually, continuous learning is based on classification tasks and cannot be directly transferred to image tasks. In order to specifically adapt to degradation tasks, the present invention adjusts the key and value variables through the prompt mapping function, so that continuous learning can be adapted to the image field.
如图2所示,在一种可能的实现方式中,骨干网络采用SwinIR骨架模型,骨干网络的特征增强模块包括多个特征增强阶段,各特征增强阶段包括多层基于窗口的Transformer模块;任一层基于窗口的Transformer模块中的线性层输出的参数表示为:As shown in FIG2 , in a possible implementation, the backbone network adopts the SwinIR skeleton model, and the feature enhancement module of the backbone network includes multiple feature enhancement stages, each of which includes multiple layers of window-based Transformer modules; the parameters of the linear layer output in any layer of the window-based Transformer module are expressed as:
, ,
, ,
其中,表示第层基于窗口的Transformer模块所输出的关于第项退化任务的第个图像的键,表示输入第层基于窗口的Transformer模块中的第项退化任务的第个图像的特征信息(即第层基于窗口的Transformer模块输出的特征信息),表示第层基于窗口的Transformer模块对应的键权重矩阵,表示第项退化任务的第个图像对应的提示映射函数,表示第层基于窗口的Transformer模块输出的关于第项退化任务的第个图像的值,表示第层基于窗口的Transformer模块对应的值权重矩阵,表示索引操作,表示基于窗口的Transformer模块的索引,表示退化任务的索引,表示各项退化任务对应的图像的索引。in, Indicates The output of the window-based Transformer module for the The first step of the degradation task The key of the image, Indicates input The first layer in the window-based Transformer module The first step of the degradation task The characteristic information of the image (i.e. The feature information output by the window-based Transformer module of the layer), Indicates The key weight matrix corresponding to the window-based Transformer module of the layer, Indicates The first step of the degradation task The prompt mapping function corresponding to the image, Indicates The output of the window-based Transformer module is about The first step of the degradation task The value of the image, Indicates The value weight matrix corresponding to the window-based Transformer module of the layer, Represents an index operation. represents the index of the window-based Transformer module, represents the index of the degenerate task, Indicates the index of the image corresponding to each degradation task.
如上,对于第层基于窗口的Transformer模块,给定输入特征和权重矩阵,表示输入特征的一个维度,该维度对应的物理含义可根据实际情况设置,通过上述方法能够实现依据不同待重建图像对应的提示映射函数,对基于窗口的Transformer模块对应的键和值变量进行适应性调整,实现持续学习。As above, for Layer is a window-based Transformer module, given the input features and the weight matrix , Represents a dimension of input features. The physical meaning corresponding to this dimension can be set according to the actual situation. Through the above method, it is possible to realize the prompt mapping function corresponding to different images to be reconstructed, adaptively adjust the key and value variables corresponding to the window-based Transformer module, and realize continuous learning.
B4、当第n个待重建图像的重建效果不满足预设条件时,根据第n个待重建图像的重建效果更新当前提示映射基,得到第n+1个提示映射基,执行第n+1轮训练,直至第x轮训练得到的重建效果满足预设条件。B4. When the reconstruction effect of the nth image to be reconstructed does not meet the preset conditions, the current prompt mapping basis is updated according to the reconstruction effect of the nth image to be reconstructed to obtain the n+1th prompt mapping basis, and the n+1th round of training is performed until the reconstruction effect obtained by the xth round of training meets the preset conditions.
其中,x为大于等于n+1的正整数。Here, x is a positive integer greater than or equal to n+1.
需要说明的是,针对不同的退化任务对应的待重建图像,其对应的重建效果以及预设条件不尽相同。例如,针对曝光过度的图像,重建效果可以对应图像内容补全情况,预设条件可以对应预设的图像内容的补全程度;针对因像素低导致的模糊图像,其对应的重建效果可以对应分辨率增强情况,预设条件可以是预设的分辨率。It should be noted that the reconstruction effects and preset conditions for the images to be reconstructed corresponding to different degradation tasks are not the same. For example, for an overexposed image, the reconstruction effect may correspond to the image content completion, and the preset condition may correspond to the preset image content completion degree; for a blurred image caused by low pixels, the corresponding reconstruction effect may correspond to the resolution enhancement, and the preset condition may be the preset resolution.
B5、将第x轮训练对应的提示映射函数,确定为退化任务对应的提示映射函数。B5. Determine the prompt mapping function corresponding to the xth round of training as the prompt mapping function corresponding to the degradation task.
本发明提供的图像超分辨率重建系统的训练方法,应用于图像超分辨率重建系统,图像超分辨率重建系统包括骨干网络和提示映射生成模块;该方法针对任一退化任务,通过获取退化任务对应的多个待重建图像,基于骨干网络,根据各待重建图像和预先确定的退化任务的任务标识,训练提示映射生成模块,得到退化任务对应的提示映射函数,进而得到训练好的图像超分辨率重建系统,该系统能够不断学习不同的退化任务,能够实现动态构造自适应提示信息,针对不同的退化任务自适应地提供定制化的上下文信息,即使在保持特征多样性不变的情况下,亦能显著减少所需的模型训练参数量,提高了模型的灵活性,显著增强了模型的适应性与运算效率,还有效减轻了在跨领域应用中常见的知识遗忘现象,实现了持续学习。The training method of an image super-resolution reconstruction system provided by the present invention is applied to the image super-resolution reconstruction system, wherein the image super-resolution reconstruction system comprises a backbone network and a prompt mapping generation module. The method, for any degradation task, obtains a plurality of to-be-reconstructed images corresponding to the degradation task, and based on the backbone network, trains the prompt mapping generation module according to the task identifiers of each to-be-reconstructed image and a predetermined degradation task, obtains a prompt mapping function corresponding to the degradation task, and then obtains a trained image super-resolution reconstruction system. The system can continuously learn different degradation tasks, can dynamically construct adaptive prompt information, and adaptively provide customized context information for different degradation tasks. Even when the feature diversity is kept unchanged, the required model training parameter amount can be significantly reduced, the flexibility of the model is improved, the adaptability and operation efficiency of the model are significantly enhanced, and the common knowledge forgetting phenomenon in cross-domain applications is effectively alleviated, thereby realizing continuous learning.
图3为本发明提供的一种图像重建方法的流程示意图,如图3所示,该方法包括:FIG3 is a flow chart of an image reconstruction method provided by the present invention. As shown in FIG3 , the method includes:
S301、获取待重建图像。S301: Obtain an image to be reconstructed.
待重建图像即为待进行超分辨率重建的图像。The image to be reconstructed is an image to be super-resolution reconstructed.
待重建图像可以是任一类型的图像采集设备采集的图像,例如,可以是通过相机拍摄的照片,也可以是通过医学设备采集的核磁共振图像、B超声波图像等医学图像。The image to be reconstructed may be an image acquired by any type of image acquisition device, for example, a photo taken by a camera, or a medical image such as a magnetic resonance imaging image, a B-ultrasound image, etc. acquired by a medical device.
S302、提取待重建图像的查询特征。S302: Extract query features of the image to be reconstructed.
可选的,提取待重建图像的查询特征,包括:通过CLIP图像编辑器,提取待重建图像的查询特征。Optionally, extracting the query feature of the image to be reconstructed includes: extracting the query feature of the image to be reconstructed by using a CLIP image editor.
其中,CLIP图像编码器最后一层线性变换的归一化输出被界定为单张图像的查询特征。Among them, the normalized output of the last layer of linear transformation of the CLIP image encoder is defined as the query feature of a single image.
S303、根据待重建图像的查询特征,以及预先确定的多项退化任务对应的查询特征组,确定待重建图像对应的退化任务。S303 : Determine the degradation task corresponding to the image to be reconstructed according to the query feature of the image to be reconstructed and the query feature group corresponding to the predetermined plurality of degradation tasks.
确定多项退化任务对应的查询特征组的方法,具体可参见上述方法实施例中关于如何确定退化任务的任务标识的具体描述,在此不做赘述。The method for determining the query feature groups corresponding to the multiple degradation tasks may be specifically referred to the specific description of how to determine the task identifier of the degradation task in the above method embodiment, which will not be described in detail here.
S304、将待重建图像的查询特征和待重建图像对应的退化任务,输入根据如本发明提供的任一图像超分辨率重建系统的训练方法训练得到的图像超分辨率重建系统中的提示映射生成模块,以使提示映射生成模块根据待重建图像的查询特征和待重建图像对应的任务,以及预先训练得到各项退化任务对应的提示映射函数,得到待重建图像对应的提示信息。S304. Input the query features of the image to be reconstructed and the degradation tasks corresponding to the image to be reconstructed into a prompt mapping generation module in the image super-resolution reconstruction system trained according to the training method of any image super-resolution reconstruction system provided by the present invention, so that the prompt mapping generation module obtains prompt information corresponding to the image to be reconstructed according to the query features of the image to be reconstructed and the tasks corresponding to the image to be reconstructed, as well as the prompt mapping functions corresponding to the degradation tasks obtained in advance through training.
需要说明的是,根据上述训练方法对应的实施例中的相关描述可知,退化任务、查询特征组、任务标识,以及提示映射函数之间存在映射关系,因此,也可以根据待重建图像的查询特征,以及预先确定的多项退化任务对应的任务标识,确定待重建图像对应的任务标识;将待重建图像的查询特征和待重建图像对应的任务标识,输入根据如本发明提供的任一图像超分辨率重建系统的训练方法训练得到的图像超分辨率重建系统中的提示映射生成模块,以使提示映射生成模块根据待重建图像的查询特征和待重建图像对应的任务标识,以及预先训练得到各项退化任务对应的提示映射函数,得到待重建图像对应的提示信息。It should be noted that, according to the relevant description in the embodiment corresponding to the above-mentioned training method, there is a mapping relationship between the degradation task, the query feature group, the task identifier, and the prompt mapping function. Therefore, the task identifier corresponding to the image to be reconstructed can also be determined according to the query feature of the image to be reconstructed and the task identifiers corresponding to multiple degradation tasks determined in advance; the query feature of the image to be reconstructed and the task identifier corresponding to the image to be reconstructed are input into the prompt mapping generation module in the image super-resolution reconstruction system trained according to the training method of any image super-resolution reconstruction system provided by the present invention, so that the prompt mapping generation module obtains the prompt information corresponding to the image to be reconstructed according to the query feature of the image to be reconstructed and the task identifier corresponding to the image to be reconstructed, and the prompt mapping functions corresponding to the degradation tasks obtained by pre-training.
S305、将待重建图像的查询特征和待重建图像对应的提示信息,输入根据如本发明提供的任一图像超分辨率重建系统的训练方法训练得到的图像超分辨率重建系统中的骨干网络,以使骨干网络根据待重建图像的查询特征和待重建图像对应的提示信息,重建待重建图像。S305. Input the query features of the image to be reconstructed and the prompt information corresponding to the image to be reconstructed into the backbone network in the image super-resolution reconstruction system trained according to the training method of any image super-resolution reconstruction system provided by the present invention, so that the backbone network reconstructs the image to be reconstructed according to the query features of the image to be reconstructed and the prompt information corresponding to the image to be reconstructed.
本发明还提供了一种图像超分辨率重建系统,参见图4,包括:The present invention also provides an image super-resolution reconstruction system, see FIG4 , comprising:
图像预处理模块41、任务匹配模块42、提示映射模块43和骨干网络44。Image preprocessing module 41, task matching module 42, prompt mapping module 43 and backbone network 44.
图像预处理模块41,用于获取待重建图像,并提取待重建图像的查询特征。图4中,待重建图像表示为,待重建图像的查询特征表示为。The image preprocessing module 41 is used to obtain the image to be reconstructed and extract the query features of the image to be reconstructed. In FIG4 , the image to be reconstructed is represented as , the query feature of the image to be reconstructed is expressed as .
任务匹配模块42,根据图像预处理模块提取的待重建图像的查询特征,以及存储的预先确定的多项退化任务对应的查询特征组,确定待重建图像对应的任务。The task matching module 42 determines the task corresponding to the image to be reconstructed according to the query feature of the image to be reconstructed extracted by the image preprocessing module and the query feature group corresponding to the plurality of pre-determined degradation tasks stored.
提示映射模块43,用于根据图像预处理模块所提取的待重建图像的查询特征、任务匹配模块所确定的待重建图像对应的任务,以及预先训练得到各项退化任务对应的提示映射函数,得到待重建图像对应的提示信息。The prompt mapping module 43 is used to obtain prompt information corresponding to the image to be reconstructed based on the query features of the image to be reconstructed extracted by the image preprocessing module, the task corresponding to the image to be reconstructed determined by the task matching module, and the prompt mapping function corresponding to each degradation task obtained by pre-training.
骨干网络44,用于根据图像预处理模块所提取的待重建图像的查询特征、提示映射模块得到的待重建图像对应的提示信息,重建待重建图像。图4中,超分辨率重建图像表示为,CONV表示卷积层,WTB表示基于窗口的Transformer模块。The backbone network 44 is used to reconstruct the image to be reconstructed according to the query features of the image to be reconstructed extracted by the image preprocessing module and the prompt information corresponding to the image to be reconstructed obtained by the prompt mapping module. In FIG. 4 , the super-resolution reconstructed image is represented as , CONV represents the convolutional layer, and WTB represents the window-based Transformer module.
需要说明的是,基于提示技术的图像超分辨率重建系统中的不同模块可部署在同一电子设备中,也可部署在不同的电子设备中。当其部署在不同的电子设备时,各电子设备之间可以通过任一方式通信连接。It should be noted that different modules in the image super-resolution reconstruction system based on the prompting technology can be deployed in the same electronic device or in different electronic devices. When they are deployed in different electronic devices, the electronic devices can be connected to each other by any communication method.
对于系统实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,具体内容及有益效果等相关之处参见方法实施例的部分说明即可。As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the specific contents and beneficial effects and other related matters can be referred to the partial description of the method embodiment.
为了进一步佐证本发明的有益效果,本发明还提供了仿真数据,具体如下:In order to further prove the beneficial effects of the present invention, the present invention also provides simulation data, which are as follows:
1、仿真条件1. Simulation conditions
本仿真实验是在中央处理器Intel XEON E5-2680V4 209 CPU,NVIDIA GTX 3090GPU和Ubuntu 16.04操作系统上,运用美国Facebook公司开源的pytorch1.13进行仿真。数据库采用Nathan Silberman和Rob Fergus于2012年创建的室内图像NYU数据库、Cai等人提出的真实图像RealSR数据库、单图像超分辨率挑战赛NTIRE开源的DIV2K数据库和来自伦敦三家不同医院的核磁共振扫描得到的图像组成的IXI数据库。This simulation experiment was conducted on the CPU Intel XEON E5-2680V4 209 CPU, NVIDIA GTX 3090 GPU and Ubuntu 16.04 operating system, using the open source pytorch1.13 from Facebook. The database used the NYU database of indoor images created by Nathan Silberman and Rob Fergus in 2012, the RealSR database of real images proposed by Cai et al., the DIV2K database open sourced by the single image super-resolution challenge NTIRE, and the IXI database consisting of images obtained from MRI scans from three different hospitals in London.
数据库中的低分辨率图像分别使用四种不同的退化模式生成,包括双边插值、真实世界的退化、复杂的退化由随机组合形成的复杂退化和频域降采样退化。The low-resolution images in the database are generated using four different degradation modes, including bilateral interpolation, real-world degradation, complex degradation formed by random combination, and frequency domain downsampling degradation.
2、仿真内容2. Simulation content
与现行的持续学习提示策略CODA-P相比,进行了深入的仿真实验。CODA-P运用了一套注意力引导的键-查询机制学习单元,动态地根据输入条件权重组合这些单元以形成适应性的提示。如表1所示,经过四个数据库的持续训练后,本发明提供的基于提示技术的图像重建方法显示出更低的遗忘率,降低了5.5%,同时在平均峰值信噪比和平均结构相似度上均取得优越性能,分别提升3.48%和17.7%。In-depth simulation experiments were conducted compared with the current continuous learning prompt strategy CODA-P. CODA-P uses a set of attention-guided key-query mechanism learning units, and dynamically combines these units according to the input condition weights to form adaptive prompts. As shown in Table 1, after continuous training on four databases, the image reconstruction method based on prompt technology provided by the present invention shows a lower forgetting rate, which is reduced by 5.5%, while achieving superior performance in both average peak signal-to-noise ratio and average structural similarity, which are improved by 3.48% and 17.7% respectively.
表1 四个数据集上持续训练的实验结果Table 1 Experimental results of continuous training on four datasets
从表1所示的实验结果清晰表明,本发明提供的基于提示技术的图像重建方法凭借其创新性的自适应提示映射函数,能够灵活应对多样化的图像退化模式,同时维持较低的遗忘率,有力地验证了本发明技术优势。The experimental results shown in Table 1 clearly show that the image reconstruction method based on the hint technology provided by the present invention can flexibly cope with various image degradation modes while maintaining a low forgetting rate by virtue of its innovative adaptive hint mapping function, which strongly verifies the technical advantages of the present invention.
术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本发明的描述中,“多个”的含义是两个或两个以上,除非另有明确具体的限定。The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明,不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干简单推演或替换,都应当视为属于本发明的保护范围。The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202411186704.2A CN118710501B (en) | 2024-08-28 | 2024-08-28 | Training method of image super-resolution reconstruction system, image reconstruction method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202411186704.2A CN118710501B (en) | 2024-08-28 | 2024-08-28 | Training method of image super-resolution reconstruction system, image reconstruction method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN118710501A true CN118710501A (en) | 2024-09-27 |
CN118710501B CN118710501B (en) | 2024-11-26 |
Family
ID=92806044
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202411186704.2A Active CN118710501B (en) | 2024-08-28 | 2024-08-28 | Training method of image super-resolution reconstruction system, image reconstruction method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN118710501B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115482573A (en) * | 2022-09-29 | 2022-12-16 | 歌尔科技有限公司 | Facial expression recognition method, device and equipment and readable storage medium |
WO2023098688A1 (en) * | 2021-12-03 | 2023-06-08 | 华为技术有限公司 | Image encoding and decoding method and device |
CN118429188A (en) * | 2024-05-14 | 2024-08-02 | 合肥工业大学 | Sequence image super-resolution reconstruction method based on Transformer and CNN hybrid network |
CN118485661A (en) * | 2024-06-19 | 2024-08-13 | 中国人民解放军空军军医大学 | Controllable mobile ultrasound image quality improvement method, system, device and medium |
-
2024
- 2024-08-28 CN CN202411186704.2A patent/CN118710501B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023098688A1 (en) * | 2021-12-03 | 2023-06-08 | 华为技术有限公司 | Image encoding and decoding method and device |
CN115482573A (en) * | 2022-09-29 | 2022-12-16 | 歌尔科技有限公司 | Facial expression recognition method, device and equipment and readable storage medium |
CN118429188A (en) * | 2024-05-14 | 2024-08-02 | 合肥工业大学 | Sequence image super-resolution reconstruction method based on Transformer and CNN hybrid network |
CN118485661A (en) * | 2024-06-19 | 2024-08-13 | 中国人民解放军空军军医大学 | Controllable mobile ultrasound image quality improvement method, system, device and medium |
Non-Patent Citations (3)
Title |
---|
JIANG, A 等: "DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution", DALPSR: LEVERAGE DEGRADATION-ALIGNED LANGUAGE PROMPT FOR REAL-WORLD IMAGE SUPER-RESOLUTION, 15 August 2024 (2024-08-15) * |
徐文博;孙广玲;陆小锋;: "预训练网络引导的人脸图像超分辨率重建", 工业控制计算机, no. 06, 25 June 2020 (2020-06-25) * |
文渊博 等: "基于视觉提示学习的天气退化图像恢复", 计算机学报, 27 June 2024 (2024-06-27) * |
Also Published As
Publication number | Publication date |
---|---|
CN118710501B (en) | 2024-11-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Chen et al. | Hdrunet: Single image hdr reconstruction with denoising and dequantization | |
Dong et al. | Deep spatial–spectral representation learning for hyperspectral image denoising | |
Li et al. | MDCN: Multi-scale dense cross network for image super-resolution | |
CN107122826B (en) | Processing method and system for convolutional neural network, and storage medium | |
WO2022267641A1 (en) | Image defogging method and system based on cyclic generative adversarial network | |
Zhao et al. | Invertible image decolorization | |
CN109325931A (en) | Multimodal Image Fusion Method Based on Generative Adversarial Network and Super-Resolution Network | |
CN109146788A (en) | Super-resolution image reconstruction method and device based on deep learning | |
CN113284051B (en) | Face super-resolution method based on frequency decomposition multi-attention machine system | |
CN112446835B (en) | Image restoration method, image restoration network training method, device and storage medium | |
CN110490804A (en) | A method of based on the generation super resolution image for generating confrontation network | |
CN112507617A (en) | Training method of SRFlow super-resolution model and face recognition method | |
CN110728626A (en) | Image deblurring method and apparatus and training thereof | |
Guan et al. | Srdgan: learning the noise prior for super resolution with dual generative adversarial networks | |
CN116977651A (en) | An image denoising method based on dual-branch and multi-scale feature extraction | |
CN118396853A (en) | Image super-resolution reconstruction method based on generation type countermeasure network | |
CN117745541A (en) | Image super-resolution reconstruction method based on lightweight mixed attention network | |
CN110084751A (en) | Image re-construction system and method | |
Jeevan et al. | Wavemixsr: Resource-efficient neural network for image super-resolution | |
Liu et al. | Facial image inpainting using multi-level generative network | |
Sun et al. | Two-stage deep single-image super-resolution with multiple blur kernels for Internet of Things | |
WO2024227444A1 (en) | Image quality enhancement method and system for endoscopic image | |
CN118710501A (en) | Training method of image super-resolution reconstruction system, image reconstruction method and system | |
CN116977169A (en) | Data processing method, apparatus, device, readable storage medium, and program product | |
CN117196940A (en) | Super-resolution reconstruction method suitable for real scene image based on convolutional neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |