WO2024113403A1 - 成像系统景深扩展方法、系统、电子设备及存储介质 - Google Patents

成像系统景深扩展方法、系统、电子设备及存储介质 Download PDF

Info

Publication number
WO2024113403A1
WO2024113403A1 PCT/CN2022/137712 CN2022137712W WO2024113403A1 WO 2024113403 A1 WO2024113403 A1 WO 2024113403A1 CN 2022137712 W CN2022137712 W CN 2022137712W WO 2024113403 A1 WO2024113403 A1 WO 2024113403A1
Authority
WO
WIPO (PCT)
Prior art keywords
depth
image
field extension
field
defocus distance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/137712
Other languages
English (en)
French (fr)
Inventor
陈涛
李剑平
郭冠男
杨振宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024113403A1 publication Critical patent/WO2024113403A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio

Definitions

  • the present application relates to the field of imaging technology, and in particular to a method, system, electronic device and storage medium for extending the depth of field of an imaging system.
  • Marine plankton is a type of aquatic drifting organisms with weak swimming ability, mainly including plankton and plants. They are huge in number and widely distributed in the world's oceans. They are the basic components of the food web of food, materials and energy in the ocean. Therefore, observing marine plankton is not only the basis for understanding major marine scientific issues such as the impact of human activities and global changes on marine ecosystems and the response of marine ecosystems to global climate change, but also an indispensable technical means in applications such as marine ecological environment monitoring, biodiversity surveys, fishery biological resource assessments, and disaster-causing biological outbreak monitoring.
  • plankton has always faced the challenge of a balance between imaging quality and sampling flux.
  • the complex composition of seawater and the properties of plankton make the optical properties of the imaging medium and target faced by underwater imaging instruments always changeable, which can easily lead to the deterioration of the quality of the acquired images, affecting the resolution of underwater dark-field imaging images and the accuracy of subsequent image recognition and analysis; on the other hand, in order to obtain sufficient resolution to identify and measure tiny individual plankton, imaging often requires a larger magnification.
  • the present application provides a method, system, electronic device and storage medium for extending the depth of field of an imaging system, which can solve the problem in the related art that the imaging resolution and depth of field cannot be taken into account when collecting images of living plankton in water, resulting in a small water volume corresponding to a single frame image and low efficiency in in-situ observation of plankton.
  • the technical solution is as follows:
  • a method for extending the depth of field of an imaging system includes: acquiring a defocused image; inputting the defocused image into a depth of field extension network model to perform depth of field extension, and obtaining a depth of field extension reconstructed image.
  • the depth of field extension network model includes a defocus distance estimation subnetwork and a depth of field extension subnetwork.
  • the defocus distance estimation subnetwork is used to estimate the defocus distance of the defocus image to obtain a defocus distance estimation value, and encode the defocus distance estimation value and pass it into the depth of field extension subnetwork to guide the depth of field extension subnetwork to perform depth of field extension;
  • the depth of field extension subnetwork is used to extract features of the defocus image, and decode it in combination with the defocus distance estimation value to output a depth of field extension reconstructed image.
  • a depth of field extended imaging system includes: an image acquisition module: used to acquire a defocused image; an image reconstruction module: used to reconstruct the acquired defocused image into a depth of field extended image through a depth of field extension network model;
  • the depth of field extension network model includes a defocus distance estimation subnetwork and a depth of field extension subnetwork, the defocus distance estimation subnetwork is used to estimate the defocus distance of the defocused image, obtain a defocus distance estimation value, and encode the defocus distance estimation value and pass it into the depth of field extension subnetwork to guide the depth of field extension subnetwork to perform depth of field extension;
  • the depth of field extension subnetwork is used to extract features from the defocused image, and decode it in combination with the defocus distance estimation value, and output a depth of field extended reconstructed image.
  • an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program in the memory through the communication bus; when the computer program is executed by the processor, the imaging system depth of field extension method as described above is implemented.
  • a storage medium stores a computer program, and when the computer program is executed by a processor, the method for extending the depth of field of an imaging system as described above is implemented.
  • a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that when the computer device executes the computer program, the imaging system depth of field extension method as described above is implemented.
  • the acquired defocused image is input into the depth of field extension network model for depth of field extension to obtain a depth of field extension reconstructed image.
  • the resolution is improved, solving the problem that the existing technical solution cannot take into account both imaging resolution and depth of field when collecting images of living plankton in water, resulting in a small water volume corresponding to a single frame image and low efficiency of in-situ observation of plankton.
  • the depth of field extension network model in this application also has a good depth of field extension effect for other categories and more complex plankton images that are not included in the training data, and the generalization performance of the model is good.
  • FIG1 is a flow chart of a method for extending depth of field of an imaging system according to an exemplary embodiment
  • FIG2 is a structural diagram of a depth of field extension network model according to an exemplary embodiment
  • FIG3 is an image quality evaluation result of a depth of field extension network model according to an exemplary embodiment
  • FIG4 is a reconstructed image output by a depth extension network model according to an exemplary embodiment
  • FIG5 is a reconstructed image of different types of plankton output by a depth-extended network model according to an exemplary embodiment
  • FIG6 is a structural diagram of an extended depth of field imaging system according to an exemplary embodiment
  • Fig. 7 is a structural block diagram of an electronic device according to an exemplary embodiment.
  • RMSE Root Mean Squared Error
  • MSE-loss the full spelling in English is Mean Square Error-loss, mean square error loss function.
  • in situ imaging of living underwater plankton has always been limited by the complex composition of seawater and the properties of plankton, which makes the imaging medium and optical properties faced by underwater imaging instruments always changeable, resulting in deterioration of the quality of collected images and low resolution of imaging images; on the other hand, in order to obtain sufficient resolution to identify and measure tiny individual plankton, the imaging depth of field is sacrificed, resulting in a small water sample volume for a single imaging sampling and low observation efficiency.
  • the imaging system depth of field extension method provided in the present application can effectively extend the depth of field while ensuring the imaging resolution of underwater plankton, thereby improving the recognition accuracy. Accordingly, the imaging system depth of field extension method is applicable to the depth of field extension imaging system.
  • the depth of field extension imaging system can be deployed in imaging optoelectronic equipment, such as imaging flow cytometers, underwater silhouette imagers, dark field imagers, and the like.
  • FIG1 shows a method for extending the depth of field of an imaging system.
  • the method may include the following steps:
  • Step 210 Acquire a defocused image.
  • the dark field environment of underwater living plankton is photographed using underwater dark field imaging equipment to obtain a defocused image of the underwater living plankton in the underwater dark field imaging equipment.
  • Step 230 input the defocused image into a depth of field extension network model to perform depth of field extension to obtain a depth of field extension reconstructed image.
  • the above-mentioned depth of field extension network model includes a defocus distance estimation subnetwork and a depth of field extension subnetwork.
  • the defocus distance estimation subnetwork is used to estimate the defocus distance of the underwater dark field defocus image, obtain the defocus distance estimation value, and encode the defocus distance estimation value and pass it into the depth of field extension subnetwork to guide the depth of field extension subnetwork to perform depth of field expansion;
  • the depth of field extension subnetwork is used to extract features of the underwater dark field defocus image, and decode it in combination with the defocus distance estimation value to output a depth of field extended reconstructed image.
  • the above-mentioned defocus distance estimation subnetwork includes a first feature extractor, a first aggregator, a regressor and an encoder, specifically:
  • the first feature extractor is used to extract features of the underwater dark field defocused image.
  • the first aggregator is used to aggregate the features of the underwater dark field defocused image into a plurality of vectors.
  • the regressor is used to perform regression analysis based on the aggregated vectors and the actual defocus distance, and take the mean of the defocus distance regression values as the defocus distance estimation value of the underwater dark field defocus image.
  • the encoder is used to perform dimension transformation on the defocus distance estimation value.
  • the residual network is set as the first feature extractor and features are extracted from the underwater dark field defocused image, including mean&std, quantile and moment features.
  • the first aggregator aggregates three vectors according to the three features extracted by the first feature extractor, and inputs them into the regressor.
  • the regressor is set to be designed based on the partial least squares method (PLSR), and the three input vectors are regressed with the actual defocus distance of the underwater dark field defocused image, and the mean of the defocus distance regression value is taken as the defocus distance estimation value of the underwater dark field defocused image.
  • the defocus distance estimation value is transformed by the encoder to generate two one-dimensional vectors ⁇ and ⁇ .
  • the depth of field extension subnetwork includes a second feature extractor, a second aggregator, and a decoder, specifically:
  • the second feature extractor is used to extract a feature map of the underwater dark field defocused image.
  • the second aggregator is used to combine the dimensionally transformed defocus distance estimation vector with the feature map to obtain a fusion map.
  • the decoder is used to transform the dimension of the fusion image to obtain the RGB value of the clear image, and fuse the restored depth of field extension reconstructed image.
  • the second feature extractor is composed of 4 e-block modules, each of which contains 3 convolutional layers and adopts a skip connection structure to extract the feature map fm ex of the input underwater dark field defocused image.
  • the second aggregator is used to combine the defocus distance estimation vectors ⁇ , ⁇ after dimension transformation with the feature map fm ex extracted by the second feature extractor to obtain a combined feature map fm ref .
  • the aggregation formula is as follows:
  • the decoder performs dimension transformation on the combined feature map fm ref to obtain the RGB value of the clear image, and fuses the obtained RGB values to obtain the restored depth of field extension reconstructed image.
  • the depth of field extension network model is trained based on an underwater dark field image training set; the underwater dark field image training set includes a plurality of underwater dark field blurry-clear image pairs.
  • defocused-focused images at different defocus distances are obtained, each defocused image corresponds to a focused image, and the image pairs are marked according to the defocus distance of the defocused images.
  • Multiple pairs of defocused-focused image pairs are collected at the same defocus distance, thereby constructing an underwater dark field defocused-focused image training set with defocus distance labels.
  • the depth of field extension network model is trained based on the obtained underwater dark field defocused-focused image training set, and the model performance is evaluated based on the similarity score of the depth of field extension reconstructed image output by the model.
  • the present application uses SSIM and RMSE as well as focus evaluation scores as evaluation indicators to calculate the similarity scores of all input images and output images.
  • the SSIM and RMSE values of the input image and the model output image at different defocus distances relative to the clear image are A and B respectively.
  • C is the clarity value of the image at different defocus distances
  • DI represents the defocused image
  • FE represents the output image of the depth of field extension network model.
  • the similarity between the input and output of the depth of field extension network model and the focused and clear image decreases, but the trend of change is relatively slow.
  • the focus evaluation score indicator that combines the subjective feelings of the human eye
  • the score of the input image decreases more sharply with the increase of the defocus distance
  • the focus evaluation score of the output image of the depth of field extension network model is basically stable at around 4.0, which shows that the output image is closer to the subjective clarity of the human eye.
  • the output image of the depth of field extension network model has a smaller score standard layer at different defocus distances, and the network model is also relatively stable in restoring defocused images of plankton containing diverse content.
  • the reconstructed image output by the depth of field extension network model As shown in Figure 4, the reconstructed image output by the depth of field extension network model, as the defocus distance increases, the defocus blur of the input image increases, the details of the corresponding output image restoration gradually decrease, and the artifacts also gradually increase from the beginning.
  • the depth of field extension network model proposed in this application has a good improvement in the depth of field extension and clarity restoration of the image within a certain defocus distance.
  • the loss function of the depth of field extension network model is a weighted sum of a context loss function and a mean square error loss function.
  • Contextual loss contextual loss function
  • MSE loss can better take into account global features, it is very sensitive to image alignment errors.
  • the formula of the loss function of the above-mentioned depth of field extension network model is as follows:
  • Loss is the loss function of the depth of field expansion network model
  • Lcont Contextualloss
  • Lmse MSE loss
  • the initial weight k1 of Contextualloss is set to 0.97
  • the initial weight k2 of MSE loss is set to 0.03.
  • the depth of field extension network model can more accurately restore the features of the defocused image and better human eye visual effects, thereby achieving a better depth of field extension effect.
  • the present application uses defocused images that do not appear in the training data to evaluate the generalization of the depth of field extension network model, where DI represents the defocused image, FE represents the image reconstructed by the depth of field extension network model, and the number in the upper right corner is the focus evaluation score.
  • the depth of field extension network model in this application also achieves an effective depth of field extension effect for these untrained and complex plankton images. Not only that, as shown in E and F, the depth of field extension network model in this application also achieves an effective depth of field extension effect for particulate matter images that have never been trained.
  • the following is a system embodiment of the present application, which can be used to execute the imaging system depth of field extension method involved in the present application.
  • the imaging system depth of field extension method involved in the present application please refer to the method embodiment of the imaging system depth of field extension method involved in the present application.
  • An extended depth of field imaging system is provided in an embodiment of the present application, including but not limited to an image acquisition module 310 and an image reconstruction module 330 .
  • the image acquisition module 310 is used to acquire a defocused image.
  • Image reconstruction module 330 used to reconstruct the acquired defocused image into a depth-extended image according to the depth-extended network model 331.
  • the depth of field extension model 331 includes a defocus distance estimation sub-model 3310 and a depth of field extension sub-model 3330.
  • Defocus distance estimation submodel 3310 used to estimate the defocus distance of the defocus image, obtain the defocus distance estimation value, and encode the defocus distance estimation value and pass it to the depth of field extension subnetwork to guide the depth of field extension subnetwork to perform depth of field extension.
  • Depth of field extension sub-model 3330 used to extract features from the defocused image, decode it in combination with the defocus distance estimation value, and output a depth of field extension reconstructed image.
  • the depth of field extended imaging system provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing imaging depth expansion.
  • the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the depth of field extended imaging system will be divided into different functional modules to complete all or part of the functions described above.
  • depth of field extended imaging system and the depth of field extended method of the imaging system provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.
  • the electronic device 4000 may be a smart device.
  • the smart device may specifically include an imaging flow cytometer, an underwater silhouette imager, a dark field imager, and the like.
  • the electronic device 4000 includes at least one processor 4001 , at least one communication bus 4002 , and at least one memory 4003 .
  • the processor 4001 and the memory 4003 are connected, such as through a communication bus 4002.
  • the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and/or data reception.
  • the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.
  • Processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
  • the communication bus 4002 may include a path for transmitting information between the above components.
  • the communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc.
  • the communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc.
  • FIG. 7 only uses one thick line, but it does not mean that there is only one bus or one type of bus.
  • the memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.
  • ROM Read Only Memory
  • RAM Random Access Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • CD-ROM Compact Disc Read Only Memory
  • optical disk storage including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.
  • magnetic disk storage medium or other magnetic storage device or any other medium
  • the memory 4003 stores a computer program
  • the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .
  • the depth of field extension method of the imaging system in the above-mentioned embodiments is implemented.
  • a storage medium is provided in an embodiment of the present application, on which a computer program is stored.
  • the computer program is executed by a processor, the depth of field extension method of the imaging system in the above embodiments is implemented.
  • a computer program product includes a computer program, and the computer program is stored in a storage medium.
  • a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the imaging system depth of field extension method in each of the above embodiments.
  • the depth of field extension network model is trained according to the constructed data set, and the trained network model is used to reconstruct the original defocused images of underwater living plankton, thereby obtaining focused images of underwater living plankton with large depth of field and high resolution, which effectively solves the problem in related technologies that when collecting images of living plankton in water, it is impossible to take into account both imaging resolution and depth of field, resulting in a small water volume corresponding to a single-frame image and low efficiency in in situ observation of plankton.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

本申请提供了一种成像系统景深扩展方法、系统、电子设备及存储介质，涉及成像技术领域。其中，该方法包括：获取离焦图像；将所述离焦图像输入景深扩展网络模型进行景深扩展，得到景深扩展重建图像；所述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络。本申请解决了相关技术中在采集水中活体浮游生物图像时无法兼顾成像分辨率和景深，导致单帧图像对应的水体体积小，对浮游生物的原位观测效率低的问题。

Description

成像系统景深扩展方法、系统、电子设备及存储介质 技术领域
本申请涉及成像技术领域，具体而言，本申请涉及一种成像系统景深扩展方法、系统、电子设备及存储介质。
背景技术
海洋浮游生物是一类游泳能力较弱的水生漂流生物，主要包括浮游动植物等，它们数量庞大，广泛分布于全球的海洋中，是海洋中食物、物质和能量食物网的基础组成部分。因此，对海洋浮游生物进行观测，不仅是理解人类活动和全球变化度海洋生态系统的影响与海洋生态系统对全球气候变化响应等重大海洋科学问题的基础，也是海洋生态环境监测、生物多样性调查、渔业生物资源评估、致灾生物爆发监测等应用中不可或缺的技术手段。
然而，浮游生物原位成像一直在面临着成像质量和采样通量之间存在制衡的挑战。一方面，复杂的海水成分和浮游生物属性使得水下成像仪器所面临的成像介质和目标的光学属性一直多变，极易导致采集的图像质量恶化，影响水下暗场成像图像的分辨率和后续图像识别分析的准确率；另一方面，为了获取足够的分辨率以识别和测量微小的浮游生物个体，成像往往需要较大的倍率。这就导致景深较浅，单次成像采样的水体体积小，进而导致对海水成像采样的通量和效率较低，不利于在单位时间内对一定尺度的海域获得更有统计意义的观测结果。
由上可知，如何平衡水下活体浮游生物暗场成像分辨率和景深的问题仍有待解决。
发明内容
本申请提供了一种成像系统景深扩展方法、系统、电子设备及存储介质，可以解决相关技术中存在的在采集水中活体浮游生物图像时无法兼顾成像分辨 率和景深，导致单帧图像对应的水体体积小，对浮游生物的原位观测效率低的问题。所述技术方案如下：
根据本申请的一个方面，一种成像系统景深扩展方法，所述方法包括：获取离焦图像；将所述离焦图像输入景深扩展网络模型进行景深扩展，得到景深扩展重建图像。
所述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络，所述离焦距离估计子网络用于对所述离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展；所述景深扩展子网络用于对所述离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
根据本申请的一个方面，一种景深扩展成像系统，所述系统包括：图像获取模块：用于获取离焦图像；图像重建模块：用于将获取到的离焦图像通过景深扩展网络模型重建为景深扩展图像；所述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络，所述离焦距离估计子网络用于对所述离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展；所述景深扩展子网络用于对所述离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
根据本申请的一个方面，一种电子设备，包括：至少一个处理器、至少一个存储器、以及至少一条通信总线，其中，存储器上存储有计算机程序，处理器通过通信总线读取存储器中的计算机程序；计算机程序被处理器执行时实现如上所述的成像系统景深扩展方法。
根据本申请的一个方面，一种存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如上所述的成像系统景深扩展方法。
根据本申请的一个方面，一种计算机程序产品，计算机程序产品包括计算机程序，计算机程序存储在存储介质中，计算机设备的处理器从存储介质读取 计算机程序，处理器执行计算机程序，使得计算机设备执行时实现如上所述的成像系统景深扩展方法。
本申请提供的技术方案带来的有益效果是：
在上述技术方案中，通过获取离焦图像，并构建景深扩展网络模型，将获取到的离焦图像输入景深扩展网络模型进行景深扩展，得到景深扩展重建图像。在实现了成像景深扩展的同时提升了分辨率，解决了现有技术方案中存在的采集水中活体浮游生物图像时无法兼顾成像分辨率和景深，导致单帧图像对应的水体体积小，对浮游生物的原位观测效率低的问题，同时本申请中的景深扩展网络模型对训练数据中不曾包含的其他类别和更为复杂的浮游生物图像同样具有较好的景深扩展效果，模型的泛化性能良好。
附图说明
为了更清楚地说明本申请提供的技术方案，下面将对本申请各实施例描述中所需要使用的附图作简单地介绍。
图1是根据一示例性实施例示出的一种成像系统景深扩展方法流程图；
图2是根据一示例性实施例示出的景深扩展网络模型的结构图；
图3是根据一示例性实施例示出的景深扩展网络模型的图像质量评价结果；
图4是根据一示例性实施例示出的景深扩展网络模型输出的重建图像；
图5是根据一示例性实施例示出的不同种类的浮游生物经景深扩展网络模型输出的重建图像；
图6是根据一示例性实施例示出的景深扩展成像系统的结构图；
图7是根据一示例性实施例示出的一种电子设备的结构框图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能 的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本申请的限制。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。
下面是本申请涉及的几个名词进行的介绍和解释：
SSIM，英文全拼为Structural Similarity，结构相似度。
RMSE，英文全拼为Root Mean Squared Error，均方根误差。
PLSR，英文全拼为Partial least squares regression，偏最小二乘法。
MSE-loss，英文全拼为Mean Square Error-loss，均方误差损失函数。
如前上述，一直以来，对水下活体浮游生物原位成像一方面受复杂的海水成分和浮游生物属性使得水下成像仪器所面临的成像介质和光学属性一直多变，导致采集的图像质量恶化，成像图像分辨率低；另一方面为了获取足够的分辨率以识别和测量微小的浮游生物个体就会牺牲成像景深，导致单次成像采样的水样体积小，观测效率低。
由上可知，相关技术中仍存在成像分辨率和景深无法兼顾的问题。
为此，本申请提供的成像系统景深扩展方法，能够有效地在保证水下浮游生物成像分辨率的同时扩展景深，提高识别准确率，相应地，该成像系统景深扩展方法适用于景深扩展成像系统。该景深扩展成像系统可部署于成像光电设备，例如成像流式细胞仪、水下剪影成像仪、暗场成像仪等等。
为使本申请的目的、技术方案和优点更加清楚，下面以水下暗场离焦图像为例，结合附图对本申请实施方式进一步地详细描述。
如图1所示为一种成像系统景深扩展方法，该方法可以包括以下步骤：
步骤210，获取离焦图像。
利用水下暗场成像设备对水下活体浮游生物的暗场环境进行拍摄，获取水下活体浮游生物在水下暗场成像设备中的离焦图像。
步骤230，将所述离焦图像输入景深扩展网络模型进行景深扩展，得到景深扩展重建图像。
如图2所示，上述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络。
具体的，离焦距离估计子网络用于对水下暗场离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展；景深扩展子网络用于对所述水下暗场离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
在一示例性实施例中，上述离焦距离估计子网络包括第一特征提取器、第一聚合器、回归器以及编码器，具体的：
第一特征提取器用于提取所述水下暗场离焦图像的特征。
第一聚合器用于将所述水下暗场离焦图像的特征聚合成若干个向量。
回归器用于根据聚合到的若干个向量与实际离焦距离进行回归分析，并取离焦距离回归值均值作为所述水下暗场离焦图像的离焦距离估计值。
编码器用于对所述离焦距离估计值进行维度变换。
具体的，将残差网络设置为第一特征提取器并从水下暗场离焦图像中提取特征，包括mean&std、quantile以及moment特征。第一聚合器根据第一特征提取器提取到的三个特征分别聚合成三个向量，并输入至回归器中。回归器设置为基于偏最小二乘法(PLSR)设计的，将输入的三个向量与水下 暗场离焦图像的实际离焦距离进行回归分析，并取离焦距离回归值均值作为水下暗场离焦图像的离焦距离估计值。并通过编码器对离焦距离估计值进行维度变换，从而生成两个一维矢量α，β。
在一示例性实施例中，上述景深扩展子网络包括第二特征提取器、第二聚合器以及解码器，具体的：
第二特征提取器用于提取所述水下暗场离焦图像的特征图。
第二聚合器用于将维度变换后的离焦距离估计矢量与所述特征图进行结合得到融合图。
解码器用于将所述融合图进行维度变换，得到清晰图像的RGB值，融合得到复原后的景深扩展重建图像。
具体的，第二特征提取器由4个e-block模块构成，每个e-block模块包含有3个卷积层，并采用跳跃连接的结构，用于提取输入的水下暗场离焦图像的特征图fm ex。第二聚合器用于将维度变换后的离焦距离估计矢量α，β与第二特征提取器提取到的特征图fm ex相结合得到组合后的特征图fm ref。具体的，聚合公式如下所示：
fm ref＝αfm ex+β
最后根据解码器对得到的组合后的特征图fm ref进行维度变换，得到清晰图像的RGB值，并融合得到的RGB值，得到复原后的景深扩展重建图像。
在一示例性实施例中，上述景深扩展网络模型是基于水下暗场图像训练集训练得到的；水下暗场图像训练集包括多个水下暗场模糊-清晰图像对。
具体的，获取不同离焦距离下的离焦-聚焦图像，每一幅离焦模糊图像对应一幅聚焦清晰图像，并根据离焦模糊图像的离焦距离对图像对进行标记，同一离焦距离下采集多对离焦-聚焦图像对，从而构建了一个带有离焦距离标签的水下暗场离焦-聚焦图像训练集。根据得到的水下暗场离焦-聚焦图像训练集对景深扩展网络模型进行训练，根据模型输出的景深扩展重建图像的相似度得分对模型性能进行评价。
在一种可能的实施例中，如图3所示，本申请利用SSIM和RMSE以及聚焦评价分值作为评价指标，计算了全部输入图像和输出图像的相似度得分。其中，不同离焦距离下的输入图像和模型输出图像相对于清晰图像的SSIM和RMSE值分别为A和B。C为不同离焦距离下图像的清晰程度值，DI表示离焦图像，FE表示景深扩展网络模型输出图像。
对于SSIM和RMSE两个图像相似度而言，随着离焦距离的增加，景深扩展网络模型的输入和输出与聚焦清晰图像的相似度都在减小，但变化趋势较为缓慢。而在结合了人眼主观感受的聚焦评价得分指标下，输入图像的得分随着离焦距离的增加下降更为剧烈，而景深扩展网络模型输出图像的聚焦评价得分基本稳定在了4.0左右，这表明输出图像更接近人眼主观认为的清晰。在三种评价指标下，景深扩展网络模型的输出图像在不同离焦距离下的得分标准层较小，网络模型对含有多样内容的浮游生物离焦图像复原的效果也比较稳定。
如图4所示为景深扩展网络模型输出的重建图像，随着离焦距离的增加，输入图像的离焦模糊程度加重，对应输出的图像恢复的细节逐渐减少，伪影也相应从开始展现到逐步加重。本申请提出的景深扩展网络模型在一定离焦距离内对图像的景深扩展和清晰度恢复均有不错的提升。
在一示例性实施例中，上述景深扩展网络模型的损失函数为上下文损失函数与均方误差损失函数的加权和。
具体的，已知Contextualloss(上下文损失函数)具有对图像对齐误差不敏感的优势，但同时也存在对全局特征空间分布考虑不足的缺陷，容易导致网络输出图像出现斑块化的现象；而MSE loss虽然能够较好的兼顾全局特征，但对图像对齐误差非常敏感。
在一示例性实施例中，上述景深扩展网络模型的损失函数的公式如下所示：
Loss＝k 1*L cont+k 2*L mse
其中Loss为景深扩展网络模型的损失函数，L cont为Contextualloss，L mse 为MSE loss。Contextualloss的初始权重k 1设置为0.97，MSE loss的初始权重k 2设置为0.03。
通过将二者加权结合用来训练景深扩展网络模型，能使景深扩展网络模型更准确的恢复离焦图像的特征和更好的人眼视觉效果，实现更好的景深扩展效果。
如图5所示为不同种类的浮游生物经景深扩展网络模型输出的重建图像。在一示例性实施例中，本申请使用了训练数据中未出现的离焦图像对景深扩展网络模型的泛化性进行了评定，其中，DI表示离焦图像，FE表示经景深扩展网络模型重建后的图像，右上角数字为聚焦评价得分值。
图5中A和B中的莹虾、抱卵多毛和桡足虽然包含在训练数据集中，但其离焦程度更为复杂，例如莹虾身体的不同部位离焦情况不同；处于同一张图中的多毛和桡足的离焦情况也不同。而C和D中带卵的怪水蚤和尖笔帽螺不仅同样具有复杂的聚焦状态，其生物类别也不曾包含在训练数据集中。从每张FE图的视觉效果和聚焦得分结果来看，本申请中的景深扩展网络模型对这些未曾训练过并且内容复杂的浮游生物图像同样取得了有效的景深扩展效果，不仅如此，如E和F所示，本申请中的景深扩展网络模型对从未训练过的颗粒物图像一样得到了有效的景深扩展效果。
下述为本申请系统实施例，可以用于执行本申请所涉及的成像系统景深扩展方法。对于本申请系统实施例中未披露的细节，请参照本申请所涉及的成像系统景深扩展方法的方法实施例。
请参阅图6，本申请实施例中提供了一种景深扩展成像系统，包括但不限于图像获取模块310、图像重建模块330。
其中，图像获取模块310，用于获取离焦图像。
图像重建模块330：用于根据景深扩展网络模型331将获取到的离焦图像重建为景深扩展图像。
其中，景深扩展模型331包括离焦距离估计子模型3310和景深扩展子模型 3330。
离焦距离估计子模型3310：用于对离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展。
景深扩展子模型3330：用于对所述离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
需要说明的是，上述实施例所提供的景深扩展成像系统在进行成像景深扩展时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即景深扩展成像系统的内部结构将划分为不同的功能模块，以完成以上描述的全部或者部分功能。
另外，上述实施例所提供的景深扩展成像系统与成像系统景深扩展方法的实施例属于同一构思，其中各个模块执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
请参阅图7，本申请实施例中提供了一种电子设备4000，该电子设备4000可以是智能设备，智能设备具体可以包括成像流式细胞仪、水下剪影成像仪、暗场成像仪等等。
在图7中，该电子设备4000包括至少一个处理器4001、至少一条通信总线4002以及至少一个存储器4003。
其中，处理器4001和存储器4003相连，如通过通信总线4002相连。可选地，电子设备4000还可以包括收发器4004，收发器4004可以用于该电子设备与其他电子设备之间的数据交互，如数据的发送和/或数据的接收等。需要说明的是，实际应用中收发器4004不限于一个，该电子设备4000的结构并不构成对本申请实施例的限定。
处理器4001可以是CPU(Central Processing Unit，中央处理器)，通用处理器，DSP(Digital Signal Processor，数据信号处理器)，ASIC(Application Specific Integrated Circuit，专用集成电路)，FPGA(Field Programmable Gate  Array，现场可编程门阵列)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器4001也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等。
通信总线4002可包括一通路，在上述组件之间传送信息。通信总线4002可以是PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(Extended Industry Standard Architecture，扩展工业标准结构)总线等。通信总线4002可以分为地址总线、数据总线、控制总线等。为便于表示，图7中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
存储器4003可以是ROM(Read Only Memory，只读存储器)或可存储静态信息和指令的其他类型的静态存储设备，RAM(Random Access Memory，随机存取存储器)或者可存储信息和指令的其他类型的动态存储设备，也可以是EEPROM(Electrically Erasable Programmable Read Only Memory，电可擦可编程只读存储器)、CD-ROM(Compact Disc ReadOnly Memory，只读光盘)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。
存储器4003上存储有计算机程序，处理器4001通过通信总线4002读取存储器4003中存储的计算机程序。
该计算机程序被处理器4001执行时实现上述各实施例中的成像系统景深扩展方法。
此外，本申请实施例中提供了一种存储介质，该存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述各实施例中的成像系统景深扩展方法。
本申请实施例中提供了一种计算机程序产品，该计算机程序产品包括计算 机程序，该计算机程序存储在存储介质中。计算机设备的处理器从存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行上述各实施例中的成像系统景深扩展方法。
与相关技术相比，一方面，通过采集水中活体浮游生物不同离焦距离下的图像对，并利用采集到的图像对构建数据集，解决了相关技术中训练模型时缺少符合水下真实世界训练数据的问题；另一方面，根据构建的数据集训练景深扩展网络模型，利用训练好的网络模型对原始水下活体浮游生物离焦图像进行重建，得到了兼具大景深和高分辨率的水下活体浮游生物聚焦图像，有效解决了相关技术中在采集水中活体浮游生物图像时无法兼顾成像分辨率和景深，导致单帧图像对应的水体体积小，对浮游生物的原位观测效率低的问题。
应该理解的是，虽然附图的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，其可以以其他的顺序执行。而且，附图的流程图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，其执行顺序也不必然是依次进行，而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
以上所述仅是本申请的部分实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (10)

  1. 一种成像系统景深扩展方法，其特征在于，所述方法包括：
    获取离焦图像；
    将所述离焦图像输入景深扩展网络模型进行景深扩展，得到景深扩展重建图像；
    所述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络；
    所述离焦距离估计子网络用于对所述离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展；
    所述景深扩展子网络用于对所述离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
  2. 如权利要求1所述的方法，其特征在于，所述离焦距离估计子网络包括第一特征提取器、第一聚合器、回归器以及编码器；
    所述第一特征提取器用于提取所述离焦图像的特征；
    所述第一聚合器用于将所述离焦图像的特征聚合成若干个向量；
    所述回归器用于根据聚合到的若干个向量与实际离焦距离进行回归分析，并取离焦距离回归值均值作为所述离焦图像的离焦距离估计值；
    所述编码器用于对所述离焦距离估计值进行维度变换。
  3. 如权利要求1所述的方法，其特征在于，所述景深扩展子网络包括第二特征提取器、第二聚合器以及解码器；
    所述第二特征提取器用于提取所述离焦图像的特征图；
    所述第二聚合器用于将维度变换后的离焦距离估计矢量与所述特征图进行结合得到融合图；
    所述解码器用于将所述融合图进行维度变换，得到清晰图像的RGB值，融合得到复原后的景深扩展重建图像。
  4. 如权利要求3所述的方法，其特征在于，所述第二聚合器的聚合公式为：
    fm ref＝αfm ex+β
    其中，fm ref为聚合后的特征图，fm ex为第二特征提取器提取到的特征图，α和β为将离焦距离估计值进行维度变换后得到的离焦距离估计矢量。
  5. 如权利要求1所述的方法，其特征在于，所述景深扩展网络模型是基于图像数据集训练得到的；所述图像训练集包括多个模糊-清晰图像对。
  6. 如权利要求1所述的方法，其特征在于，所述景深扩展网络模型的损失函数为上下文损失函数与均方误差损失函数的加权和。
  7. 如权利要求6所述的方法，其特征在于，所述景深扩展网络模型的损失函数公式为：
    Loss＝k 1*L cont+k 2*L mse
    其中，Loss为景深扩展网络模型的损失函数，L cont为上下文损失函数，L mse为均方误差损失函数。上下文损失函数的初始权重k 1设置为0.97，均方误差损失函数的初始权重k 2设置为0.03。
  8. 一种景深扩展成像系统，其特征在于，所述系统包括：
    图像获取模块，用于获取离焦图像；
    图像重建模块，用于将获取到的离焦图像通过景深扩展网络模型重建为景深扩展图像；
    所述景深扩展网络模型包括离焦距离估计子网络和景深扩展子网络；
    所述离焦距离估计子网络用于对离焦图像的离焦距离进行估计，得到离焦距离估计值，并将所述离焦距离估计值进行编码后传入所述景深扩展子网络，指导景深扩展子网络进行景深扩展；
    所述景深扩展子网络用于对所述离焦图像进行特征提取，并结合所述离焦距离估计值进行解码，输出景深扩展重建图像。
  9. 一种电子设备，其特征在于，包括：至少一个处理器、至少一个存储器、以及至少一条通信总线，其中，
    所述存储器上存储有计算机程序，所述处理器通过所述通信总线读取所述存储器中的所述计算机程序；
    所述计算机程序被所述处理器执行时实现权利要求1至7中任一项所述的成像系统景深扩展方法。
  10. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述成像系统景深扩展方法。
PCT/CN2022/137712 2022-11-28 2022-12-08 成像系统景深扩展方法、系统、电子设备及存储介质 Ceased WO2024113403A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211502402.2 2022-11-28
CN202211502402.2A CN118102131A (zh) 2022-11-28 2022-11-28 成像系统景深扩展方法、系统、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024113403A1 true WO2024113403A1 (zh) 2024-06-06

Family

ID=91153701

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137712 Ceased WO2024113403A1 (zh) 2022-11-28 2022-12-08 成像系统景深扩展方法、系统、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN118102131A (zh)
WO (1) WO2024113403A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130010067A1 (en) * 2011-07-08 2013-01-10 Ashok Veeraraghavan Camera and Method for Focus Based Depth Reconstruction of Dynamic Scenes
CN112070887A (zh) * 2020-09-08 2020-12-11 哈尔滨工业大学 一种基于深度学习的全切片数字成像景深扩展法
CN114897752A (zh) * 2022-05-09 2022-08-12 四川大学 一种基于深度学习的单透镜大景深计算成像系统及方法
CN115170429A (zh) * 2022-07-20 2022-10-11 清华大学深圳国际研究生院 基于深度学习的水下原位显微成像仪景深扩展方法及系统
CN115359105A (zh) * 2022-08-01 2022-11-18 荣耀终端有限公司 景深扩展图像生成方法、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130010067A1 (en) * 2011-07-08 2013-01-10 Ashok Veeraraghavan Camera and Method for Focus Based Depth Reconstruction of Dynamic Scenes
CN112070887A (zh) * 2020-09-08 2020-12-11 哈尔滨工业大学 一种基于深度学习的全切片数字成像景深扩展法
CN114897752A (zh) * 2022-05-09 2022-08-12 四川大学 一种基于深度学习的单透镜大景深计算成像系统及方法
CN115170429A (zh) * 2022-07-20 2022-10-11 清华大学深圳国际研究生院 基于深度学习的水下原位显微成像仪景深扩展方法及系统
CN115359105A (zh) * 2022-08-01 2022-11-18 荣耀终端有限公司 景深扩展图像生成方法、设备及存储介质

Also Published As

Publication number Publication date
CN118102131A (zh) 2024-05-28

Similar Documents

Publication Publication Date Title
CN116524299B (zh) 一种图像样本生成方法、装置、设备及存储介质
CN115410050B (zh) 基于机器视觉的肿瘤细胞检测设备及其方法
Jia et al. Effective meta-attention dehazing networks for vision-based outdoor industrial systems
Peng et al. Raune-net: a residual and attention-driven underwater image enhancement method
CN109410180B (zh) 衰减系数的确定方法、装置、计算机设备和存储介质
Gao et al. TEGAN: Transformer embedded generative adversarial network for underwater image enhancement
CN116912649A (zh) 基于相关注意力引导的红外与可见光图像融合方法及系统
CN114119484A (zh) 一种甲状腺癌或乳腺癌的检测方法及相关装置
CN113344815B (zh) 一种用于图像补全的多尺度金字塔型跳跃连接方法
Chu et al. SISC: A feature interaction-based metric for underwater image quality assessment
CN118172688B (zh) 基于DeeplabV3+和Unet网络架构的无人机小麦倒伏影像分割方法
Liu et al. Dual UNet low-light image enhancement network based on attention mechanism
CN113327221B (zh) 融合roi区域的图像合成方法、装置、电子设备及介质
CN116206196B (zh) 一种海洋低光环境多目标检测方法及其检测系统
Yin et al. A multi-task deep learning framework integrating segmentation and reconstruction for lensless imaging
CN116051532A (zh) 基于深度学习的工业零件缺陷检测方法、系统及电子设备
CN121169692A (zh) 基于可学习权重线性累积策略的脑部mri超分辨率重建方法
CN114862685B (zh) 一种图像降噪方法、及图像降噪模组
CN117541519B (zh) 光场像差校正方法、装置、电子设备及存储介质
Fitrianto et al. Implementation Of Transfer Learning In Cat Breed Detection Using Web-Based Convolutional Neural Network (CNN)
CN118102131A (zh) 成像系统景深扩展方法、系统、电子设备及存储介质
CN119399223A (zh) 一种基于kan网络的医学图像分割方法
Zhao et al. MFANet: Multi-feature aggregation network for multi-focus image fusion
Wang Video Dehazing Based on Convolutional Neural Network Driven Using Our Collected Dataset
Poudel Deep Learning Framework for Inverse Problems in Computational Imaging: A Lensless Imaging and Super-resolution Magnetic Resonance Imaging Case

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22966989

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22966989

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 22966989

Country of ref document: EP

Kind code of ref document: A1