WO2023193461A1 - 一种太赫兹单像素成像方法与系统 - Google Patents

一种太赫兹单像素成像方法与系统 Download PDF

Info

Publication number
WO2023193461A1
WO2023193461A1 PCT/CN2022/137703 CN2022137703W WO2023193461A1 WO 2023193461 A1 WO2023193461 A1 WO 2023193461A1 CN 2022137703 W CN2022137703 W CN 2022137703W WO 2023193461 A1 WO2023193461 A1 WO 2023193461A1
Authority
WO
WIPO (PCT)
Prior art keywords
terahertz
network model
image
pixel imaging
image reconstruction
Prior art date
Application number
PCT/CN2022/137703
Other languages
English (en)
French (fr)
Inventor
鲁远甫
祝永乐
佘荣斌
Original Assignee
深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳先进技术研究院 filed Critical 深圳先进技术研究院
Publication of WO2023193461A1 publication Critical patent/WO2023193461A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
    • G01N21/35—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light
    • G01N21/3581—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light using far infrared light; using Terahertz radiation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • This application relates to the fields of optical imaging and deep learning, and in particular to a terahertz single-pixel imaging method and system.
  • Terahertz waves have the advantages of low photon energy and penetrating non-polar substances. They have great potential in terahertz imaging, spectral analysis and high-speed communications. Among them, terahertz imaging has begun to be used in fields such as national defense security and biological imaging. However, the development of terahertz pixelated detector arrays has been slow due to a lack of suitable materials. At present, most multi-pixel terahertz detector arrays are narrow-band or need to work in a low-temperature refrigeration environment. However, traditional terahertz single-point scanning imaging cannot meet the requirements of fast imaging, which greatly restricts the actual promotion of terahertz imaging technology. .
  • terahertz single-pixel imaging systems are implemented through compressed sensing, Hadamard-based and Fourier-based methods.
  • She et al. used a 220um silicon-based graphene modulator and Fourier stripes to achieve sub-wavelength terahertz image reconstruction, using Due to the sparse characteristics of the image, image reconstruction under a 10% modulation mask is achieved by collecting low-frequency coefficients and inverse Fourier transform;
  • Rayko et al. use silicon total internal reflection prisms and Hadamard masks to achieve near-real-time terahertz single-pixel video, using Hadamard
  • the sparse feature of the domain uses undersampling technology to reduce sampling time.
  • terahertz single-pixel imaging technology has the following problems to be solved: (1) The imaging speed is slow. Since single-pixel imaging sends a single detector to receive the light intensity of several mask patterns through a modulator, the imaging speed depends on the modulation time, the number of projections, the speed of projection and the response time of the detector, etc.; as the image resolution increases In this case, increasing the number of samples will also slow down the imaging speed. (2) Poor imaging quality: Terahertz waves are extremely susceptible to coherent light interference during transmission. At the same time, detection errors caused by hardware thermal noise greatly affect the imaging quality.
  • the undersampling method can greatly shorten the imaging time, it will cause the loss of high-frequency information in the output image and deteriorate the image; from an observable point of view, when the sampling rate is lower than a certain ratio, the reconstructed pattern will appear Severe distortion or even distortion, so the sampling rate is not suitable below a certain lower limit.
  • the traditional deep learning algorithms currently used in single-pixel imaging solutions cannot fall below a certain sampling rate, especially for complex patterns.
  • the splicing of algorithms will also bring redundancy to the algorithms, greatly increasing the requirements for the imaging system. .
  • Embodiments of the present application provide a terahertz single-pixel imaging method and system to solve the problems of slow imaging speed, poor imaging quality, and redundant system algorithms in related technologies.
  • One aspect of the present invention provides a terahertz single-pixel imaging method, including the following steps:
  • the image reconstruction network model includes a fully connected block and a triangular dense block, wherein the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection.
  • the convolution block includes a convolution layer with an activation layer, a normal convolution layer and a residual connection.
  • sampling scheme is Hadamard coding, Fourier coding and wavelet transform coding.
  • the step S3 includes: sending the mask pattern to the digital micromirror device, irradiating the digital micromirror device with a laser, and reflecting the mask pattern onto the modulator of the target pattern area, and then The terahertz laser is emitted and the laser intensity is modulated by a terahertz modulator. At the same time, the terahertz laser is received through a terahertz single-point detector to obtain under-sampled and fully-sampled one-dimensional data.
  • step S4 also includes: performing image processing on inverse transformation of the fully sampled one-dimensional data, comparing it with the reconstructed image, and performing image quality evaluation.
  • step S1 also includes:
  • a corresponding data set is selected according to the scene of the reconstructed image, and a certain degree of random noise or additional image processing is added to the input image data.
  • step S1 includes:
  • a training set is generated according to the sampling scheme, and the training set includes an original image and a one-dimensional signal converted into a one-dimensional signal by inverse transformation of the image according to the sampling scheme.
  • Another aspect of the present invention provides an end-to-end network terahertz single-pixel imaging system, including:
  • a training unit constructs an image reconstruction network model, selects a sampling scheme, and trains the image reconstruction network model based on the sampling scheme;
  • a generation unit generates a corresponding mask pattern according to the sampling scheme
  • the acquisition unit modulates the terahertz laser through the mask pattern.
  • the modulated terahertz laser interacts with the target, and the modulated signal formed is received by the detector, and undersampled one-dimensional data is obtained;
  • the reconstruction unit imports the undersampled one-dimensional data into the trained image reconstruction network model to obtain the reconstructed image
  • the image reconstruction network model includes a fully connected block and a triangular dense block, where the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection.
  • the convolution block includes a convolution layer with an activation layer, a normal convolution layer and a residual connection.
  • sampling scheme is Hadamard coding, Fourier coding and wavelet transform coding.
  • the current mainstream deep learning strategy for single-pixel imaging is to reconstruct the image through traditional imaging algorithms and then put it into the trained image enhancement network.
  • the two algorithms are independent of each other.
  • the end-to-end convolutional neural network can train single-pixel imaging reconstruction algorithms and image enhancement algorithms, which not only saves additional information storage space, but also directly completes the algorithm reconstruction from one-dimensional signals to two-dimensional images.
  • the present invention proposes a deep network model SIDL based on triangular dense blocks. Through residual dense connection and depth downsampling, it combines shallow information and deep information to restore the undersampled pattern with low spatial resolution to a large extent. High-quality images greatly reduce the sampling rate requirements for complex patterns, achieving a measurement rate much lower than the Nyquist theorem sampling rate to increase imaging speed.
  • the image reconstruction network model based on triangular dense blocks proposed by the present invention makes the overall network memorized.
  • Each layer of the network has all the information previously input. No matter how deep the network is, gradient explosion will not occur;
  • depth compression can mine the depth information of undersampled images, so that effective information is obtained after each downsampling training, thereby improving the quality of the image and making it closer to the real image, and it can also adapt to changes in different feature channels.
  • the present invention belongs to an adaptive algorithm.
  • the encoding method of the end-to-end convolutional neural network before input is not unique. You can choose an existing or artificially set encoding method to collect one-dimensional measurement signals.
  • the reconstruction required will be trained based on the original image adaptability. Network parameters; compared with traditional SIDL, which can only train existing single-pixel imaging methods, this model reduces system requirements.
  • the present invention provides a terahertz single-pixel imaging method and system.
  • the end-to-end convolutional deep neural network is combined with the terahertz single-pixel imaging.
  • the residual dense connection and depth compression in the network model are combined into a triangle. Dense blocks, therefore, can reduce the algorithmic redundancy of the system and can adapt to most single-pixel imaging algorithms currently used.
  • network training is not prone to gradient disappearance and is more suitable for single-pixel imaging.
  • Figure 1 is a schematic diagram of a terahertz single-pixel imaging system in an embodiment of the present invention
  • Figure 2 is a flow chart of terahertz single-pixel imaging in an embodiment of the present invention
  • Figure 3 is a schematic structural diagram of a network reconstruction network model in an embodiment of the present invention.
  • Figure 4 is a schematic structural diagram of a convolution block in an embodiment of the present invention.
  • Figure 5 shows some of the original images of handwritten digit recognition in the embodiment of the present invention, the image reconstruction images using traditional algorithms with 0.8% sampling rate and 2.5% sampling rate, and the reconstruction images with SIDL algorithm.
  • the main optical path of the imaging device consists of a terahertz laser and a detector.
  • the modulation part is composed of a laser, a digital micromirror array (Digital Mirror Device, DMD), a projection lens and an intrinsic semiconductor.
  • DMD Digital Mirror Device
  • the laser is masked with DMD and then projected on the modulator through the lens through indium tin oxide (ITO) glass.
  • ITO is transparent to visible light and reflects terahertz light.
  • the terahertz light is collimated through the lens and penetrates the modulator and the target object X is focused on the detector by the lens.
  • the terahertz light is modulated by the mask and the modulated intensity signal I is output on the detector.
  • Any two-dimensional image can be regarded as weighted by a complete set of orthogonal mask patterns.
  • Each mask pattern corresponds to a frequency point in the transformation domain.
  • the relationship between the target pattern and the transformation domain function is expressed by the formula ( 1) given.
  • I (x, y) is the object target
  • M, N are the length and width of the object target
  • u, v are the point coordinates of the frequency in the transformation domain
  • f is a two-dimensional matrix function
  • the size is given by (x, y, u, v) is determined
  • a uv is the weight size, which is uniquely determined by (u, v).
  • K M
  • the sampling method of patterns with missing information is called undersampling.
  • the data obtained by using Hadamard coding based on undersampling is incomplete data in the transform domain; when the data is inversely transformed, the spatial resolution will be reduced due to lack of information, causing the overall image quality to deteriorate. Gotta be blurry.
  • An embodiment of the present invention provides a terahertz single-pixel imaging method, which includes the following steps:
  • Sampling schemes can be Hadamard coding, Fourier coding and wavelet transform coding.
  • the present invention proposes a new type of deep convolutional neural network SIDL, which can train various single-pixel imaging algorithms and even terahertz single-pixel imaging, realize direct conversion from one-dimensional data to two-dimensional images, while ensuring high-quality reconstruction of the image.
  • the schematic diagram of the image reconstruction network model structure is shown in Figure 3.
  • the image reconstruction network model SIDL consists of fully connected blocks and triangle dense blocks.
  • the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection.
  • the input data is restored to feature map 0 through the fully connected layer.
  • feature map 0 is consistent with the reconstructed output size.
  • After entering the first convolution block enter the first feature channel 1, and obtain feature map 1.
  • the number of feature channels is 64.
  • N-1 feature maps are obtained through N-1 convolution blocks in sequence; the feature map is down-sampled.
  • the number of building blocks is half of the number of the previous layer, and the number of feature channels is twice that of the previous layer.
  • feature map 11 obtains feature map 21 through the downsampling layer.
  • the feature map of each feature channel will enter the fusion layer to obtain the total feature map, and enter the feature channel of the previous layer through the upsampling layer. Finally, all feature maps are fused through the first fusion layer, and the output map is restored through a convolution layer and residual connection.
  • This network model integrates residual dense connections and depth compression (i.e., downsampling). At the same time, each compression will reduce the network depth of the corresponding layer, improving training while ensuring that there will be no gradient disappearance due to too small receptive fields. time. Its advantages are: first, the entire network has memory, so that each layer of the network has all the information previously input, so that gradient explosion will not occur no matter how deep the network is; second, deep compression can mine under-sampled images. The depth information makes it possible to obtain effective information after each downsampling training, thereby improving the quality of the image and making it closer to the real image.
  • the schematic diagram of the convolution block is shown in Figure 4. It consists of three convolution layers with activation layers, one convolution layer and residual connection.
  • convolution blocks instead of convolution layers as the basic unit of the network can improve the network The number of layers realizes feature learning; the residual connection is added as a short connection to avoid the disappearance of gradients caused by the superposition of convolutional layers and speed up the convergence speed; at the same time, additional convolutional layers on the residual connection can adapt to changes in different feature channels .
  • the convolution block performance structure can also be customized according to network training requirements, such as deleting redundant convolution layers or adding batch normalization layers before activation layers. In essence, the convolution block is the convolution layer and activation layer that can achieve better training results.
  • Another aspect of the embodiment of the present invention also provides a terahertz single-pixel imaging system, including:
  • the training unit selects a corresponding sampling scheme, constructs an image reconstruction network model, and trains the image reconstruction network model to obtain a trained image reconstruction network model;
  • a generation unit generates a corresponding mask pattern according to the sampling scheme
  • the acquisition unit modulates the terahertz wave through the mask.
  • the modulated terahertz wave interacts with the target object, and the modulated signal formed is received by the detector, and undersampled one-dimensional data is obtained;
  • the reconstruction unit imports the undersampled one-dimensional data into the trained image reconstruction network model to obtain a reconstructed image.
  • the flow chart of the imaging method of the present invention is shown in Figure 2.
  • a sampling scheme is selected for single-pixel imaging, such as Hadamard coding, and a training set is generated for this coding method.
  • the handwritten digit recognition data set MNIST is used in the simulation.
  • Model pre-training after training, encoded mask patterns equal to the number of image undersampling and full sampling are sent to the digital micromirror device respectively; the 808nm laser is irradiated on the digital micromirror device to reflect the mask pattern to the target pattern On the modulator of the area; after using the terahertz laser to emit, the 808nm laser light intensity is modulated by the terahertz modulator, and at the same time, the terahertz laser is received through a terahertz single-point detector to obtain a set of one-dimensional under-sampled light intensity signals.
  • the undersampled one-dimensional data is passed into the image reconstruction network model SIDL, restored to a 32x32 pixel image, and inversely transformed with the fully sampled Image comparison and image quality evaluation.
  • the handwritten digit recognition data set MNIST is used for experimental simulation.
  • the training set is 60,000 images of 28x28 pixels
  • the verification set is 10,000 images of 28x28 pixels. After being enlarged to 32x32 pixels, it is used as the original image, and Gaussian noise is added and Hada is performed.
  • the one-dimensional data encoded by MM is used as the training set.
  • Set the training parameters as follows: the number of training rounds epoch is 25, the batch size is batch_size is 25, the learning rate learning_grate is 10-5, and the loss function Loss is the L1 function.
  • the model was trained with image sampling rates of 0.8% and 2.5%, and the partial renderings of the verification set are shown in Figure 5.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Biochemistry (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Analytical Chemistry (AREA)
  • Toxicology (AREA)
  • Image Processing (AREA)

Abstract

一种太赫兹单像素成像方法和系统，涉及光学成像和深度学习领域，包括：S1.选择采样方案，构建图像重建网络模型，并对图像重建网络模型进行训练，得到训练好的图像重建网络模型；S2.根据采样方案，生成相应的掩膜图案；S3.通过掩膜调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；S4.将欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像；把端到端卷积深度神经网络结合到太赫兹单像素成像上，同时将网络模型中残差密集连接和深度压缩相结合成为三角密集块，能够减少系统的算法冗余性，并且能够适应目前采用的大多数单像素成像算法。

Description

一种太赫兹单像素成像方法与系统 技术领域
本申请涉及光学成像和深度学习领域，特别涉及一种太赫兹单像素成像方法与系统。
背景技术
太赫兹波具有光子能量低、穿透非极性物质等优点，在太赫兹成像、光谱分析和高速通信等方面有着巨大的潜力。其中，太赫兹成像已经开始应用于国防安全和生物影像等领域。但由于缺乏合适的材料，太赫兹像素化探测器阵列的研发进度缓慢。目前大多数的多像素太赫兹探测器阵列都是窄带的，或者需要工作在低温制冷环境；而传统的太赫兹单点扫描成像无法满足快速成像的要求，大大制约了太赫兹成像技术的实际推广。
另一种解决太赫兹成像的方案是采用太赫兹单像素成像系统，它相比面阵型太赫兹成像系统不仅节约了硬件成本，而且为太赫兹小型化商用化带来新的可能性。目前太赫兹单像素成像系统通过压缩感知、哈达玛基和傅里叶基等方式实现，其中，She等人利用220um硅基石墨烯调制器和傅里叶条纹实现亚波长太赫兹图像重建，利用图像稀疏特点，用采集低频系数和逆傅里叶变换实现10％调制掩膜下图像重建；Rayko等人利用硅全内反射棱镜和哈达玛掩膜实现近乎实时的太赫兹单像素视频，利用哈达玛域稀疏特点采用欠采样技术减少采样时间。
但是，由于太赫兹波波长的特殊性与单像素成像的局限性，太赫兹单像素成像技术有以下问题有待解决：(1)成像速度慢。由于单像素成像通过调制器发送单个探测器接受若干个掩膜图案的光强，而成像速度取决于调制时间、投影的数量、投影的速度和探测器的响应 时间等；在图像分辨率增大的情况下，采样数量增多也会使成像速度变慢。(2)成像质量差：太赫兹波在传输过程中极易受相干光干扰，同时由于硬件热噪声带来的探测误差，极大程度影响了成像的质量。此外，虽然欠采样的方式可以大幅度缩短成像时间，但会导致输出图像的高频信息丢失而恶化图像；而从可观测的角度而言，当采样率低于一定比例时，重建图案会出现严重的失真甚至扭曲，因此采样率不适合低于某一个下限。(3)目前采用的传统深度学习算法应用在单像素成像方案无法低于某一个采样率，特别是复杂图案，同时算法相互拼接还会带来算法的冗余性，对成像系统的要求大大增加。
发明内容
本申请实施例提供一种太赫兹单像素成像方法与系统，以解决相关技术中成像速度慢、成像质量差、系统算法具有冗余性的问题。
本发明的一个方面提供了太赫兹单像素成像方法，包括以下步骤：
S1.构建图像重建网络模型，选择采样方案，基于所述采样方案对所述图像重建网络模型进行训练；
S2.根据所述采样方案，生成相应的掩膜图案；
S3.通过所述掩膜图案调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
S4.将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像；
所述图像重建网络模型包括全连接块和三角密集块，其中所述三角密集块包括卷积块、下采样层、上采样层和残差密集连接。
进一步地，所述卷积块包括带有激活层的卷积层、普通卷积层和残差连接。
进一步地，所述采样方案为哈达玛编码、傅里叶编码和小波变换编码。
进一步地，所述步骤S3包括：将所述掩膜图案发送到数字微镜器件，用激光照射所述数字微镜器件，并把所述掩膜图案反射到目标图案区域的调制器上，之后发射太赫兹激光并利用太赫兹调制器调制激光光强，同时通过太赫兹单点探测器进行太赫兹激光接收，获得欠采样和全采样的一维数据。
进一步地，所述步骤S4还包括：将所述全采样的一维数据做逆变换的图像处理，将其与所述重建图像对比，进行图像质量评价。
进一步地，所述步骤S1还包括：
对所述图像重建网络模型进行预训练之前，根据重建图像的场景选择相应的数据集，并且给输入图像数据加入一定程度的随机噪声或额外的图像处理。
进一步地，所述步骤S1包括：
根据所述采样方案生成训练集，所述训练集包括原始图像，以及根据所述采样方案对所述图像进行逆变换转换成的一维信号。
本发明的另一方面提供了一种端到端网络的太赫兹单像素成像系统，包括：
训练单元，构建图像重建网络模型，选择采样方案，基于所述采样方案对所述图像重建网络模型进行训练；
生成单元，根据所述采样方案，生成相应的掩膜图案；
获取单元，通过所述掩膜图案调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
重建单元，将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像；
所述图像重建网络模型包括全连接块和三角密集块组成，其中所述三角密集块包括卷积块、下采样层、上采样层和残差密集连接。
进一步地，所述卷积块包括带有激活层的卷积层、普通卷积层和残差连接。
进一步地，所述采样方案为哈达玛编码、傅里叶编码和小波变换编码。
本申请提供的技术方案带来的有益效果包括：
(1)减少了算法冗余。目前主流单像素成像的深度学习策略是通过传统成像算法重建后，放入到训练好的图像增强网络，两个算法是相互独立的。而端到端卷积神经网络能够训练单像素成像重建算法和图像增强算法，不仅节约了额外的信息存储空间，并且直接完成从一维信号到二维图像的算法重建。
(2)提高成像速度。本发明提出了基于三角密集块的深度网络模型SIDL，通过残差密集连接和深度下采样方式，把浅层信息和深层信息相结合从而较大程度把空间分辨率较低的欠采样图案恢复成高质量图像，大大降低了对复杂图案采样率的要求，实现测量率远低于奈奎斯特定理采样率来提高成像速度。
(3)本发明所提出的基于三角密集块的图像重建网络模型，使得整体的网络有记忆性，网络的每一层都有之前输入的全部信息，即使网络再深也不会发生梯度爆炸；同时深度压缩能够挖掘欠采样图像的深度信息，使得每一次下采样训练之后获得都是有效信息，从而提高了图像的质量并且更加接近于真实图像，还可以自适应不同特征通道的变化。
(4)本发明属于自适应性算法。端到端卷积神经网络在输入之 前的编码方式是不唯一的，可以选择已有或人为设定的编码方式采集一维测量信号，训练过程中会根据原始图像自适应性训练出重建需要的网络参数；相比只能训练已有的单像素成像方式的传统SIDL，该模型降低了对于系统的要求。
本发明提供了一种太赫兹单像素成像方法与系统，由于把端到端卷积深度神经网络结合到太赫兹单像素成像上，同时将网络模型中残差密集连接和深度压缩相结合成为三角密集块，因此，能够减少系统的算法冗余性，并且能够适应目前采用的大多数单像素成像算法，同时网络训练不容易发生梯度消失，同时更加适合单像素成像。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例中太赫兹单像素成像系统示意图；
图2为本发明实施例中太赫兹单像素成像流程图；
图3为本发明实施例中网络重建网络模型结构示意图；
图4为本发明实施例中卷积块的结构示意图；
图5为本发明实施例中部分手写数字识别原始图，0.8％采样率和2.5％采样率的图像传统算法重建图与SIDL算法的重建图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本申请 保护的范围。
首先，搭建太赫兹单像素成像装置，如图1所示。成像装置的主要光路由太赫兹激光器和探测器组成，调制部分是由激光、数字微镜阵列(Digital Mirrors Device，DMD)、投影镜头和本征半导体组成。将激光用DMD掩膜化Ф后经过镜头穿透氧化铟锡(Indium tin oxide，ITO)玻璃投影在调制器上，ITO对可见光可透过，对太赫兹光反射。太赫兹光经过透镜准直穿透调制器和目标物X被透镜聚焦在探测器上，太赫兹光经过掩膜调制，在探测器上输出调制后的强度信号I。
任何一个二维图像都可以看作由一组完备的正交掩膜图案的加权得到的，每个掩膜图案对应该变换域上的一个频率点，目标图案与变换域函数的关系由公式(1)给出。
Figure PCTCN2022137703-appb-000001
式中，I(x,y)为物体目标,M,N为物体目标的长与宽，u,v为变换域上频率的点坐标，f为二维矩阵函数，大小由(x,y,u,v)确定，a uv为权重大小，由(u，v)唯一确定。对全部正交基底图案进行加权，得到的原始图案的过程称为全采样，其中测量个数等于K＝M x N；为了减少测量个数，采集部分变换域上系数进行加权求逆，得到具有信息缺失图案的采样方式，称为欠采样。在一个实施例中，根据欠采样采用哈达玛编码获得的数据，是变换域中的不完备数据；对该数据进行逆变换时，会由于信息缺失导致空间分辨率下降，使得整体的图像质量变得模糊。
本发明实施例提供了一种太赫兹单像素成像方法，其包括以下步骤：
S1.构建图像重建网络模型，选择采样方案，基于所述采样方案对所述图像重建网络模型进行训练，得到训练好的图像重建网络模型；
采样方案可以为哈达玛编码、傅里叶编码和小波变换编码。
本发明提出一种新型深度卷积神经网络SIDL，可以训练各种单像素成像算法乃至太赫兹单像素成像，实现从一维数据直接转换成为二维图像，同时保证图像的高质量重建。其中图像重建网络模型结构示意图如图3所示。
如图3所示，图像重建网络模型SIDL包括全连接块和三角密集块组成。其中，三角密集块包括卷积块、下采样层、上采样层和残差密集连接组成。首先输入数据通过全连接层还原成特征图0，此时特征图0与重建输出尺寸一致。进入第一个卷积块后进入第一个特征通道1，并且获得特征图1，特征通道数为64，依次通过N-1个卷积块获得N-1个特征图；特征图经过下采样层对尺度进行压缩，而特征通道数增加128，获得特征图11，并且依次进入n-1个卷积层获得n-1个特征图，其中n＝N/2，即进入下一个特征通道卷积块个数为上一层个数的一半，特征通道数为上一层的两倍。同理，特征图11通过下采样层获得特征图21。每个特征通道的特征图都会进入融合层获得总特征图，并且通过上采样层进入上一层特征通道。最后通过第一层的融合层把全部特征图进行融合，并通过一个卷积层和残差连接恢复输出图。
该网络模型把残差密集连接和深度压缩(即下采样)相融合，同时每一次压缩都会减少相应层的网络深度，在保证不会出现由于感受野过小导致的梯度消失的情况下提高训练的时间。其优势在于，第一，整体的网络都有记忆性，使得网络的每一层都有之前输入的全部信息，使得网络再深也不会发生梯度爆炸；第二，深度压缩能够挖掘欠采样图像的深度信息，使得每一次下采样训练之后获得都是有效信息，从而提高了图像的质量并且更加接近于真实图像。
卷积块示意图如图4所示，其包括三个带有激活层的卷积层、一 个卷积层和残差连接组成，采用卷积块代替卷积层作为网络的基本单元，能够提高网络的层数实现特征的学习；加入残差连接作为短连接，避免由于卷积层的叠加造成的梯度消失，加快收敛速度；同时残差连接上额外的卷积层可以自适应不同特征通道的变化。除了上述提出的卷积块结构，还可以根据网络训练要求自定义卷积块表现结构，如删除多余的卷积层或者在激活层之前均加入批归一化层。本质来说，卷积块就是能够实现更优训练效果的卷积层和激活层。
对网络预训练之前，根据重建图像的场景选择相应的数据集，并且给输入图像数据加入一定程度的随机噪声或额外的图像处理，根据编码方式对处理图像进行逆变换转换成为一维信号作为输入源，与原始图像组成训练集。
S2.根据所述采样方案，生成相应的掩膜图案；
S3.通过掩膜调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
S4.将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像。
本发明实施例的另一方面还提供了一种太赫兹单像素成像系统，包括：
训练单元，选择相应的采样方案，构建图像重建网络模型，并对所述图像重建网络模型进行训练，得到训练好的图像重建网络模型；
生成单元，根据所述采样方案，生成相应的掩膜图案；
获取单元，通过掩膜调制太赫兹波，调制后的太赫兹波和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
重建单元，将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像。
上述各单元的功能参见相应的方法，在此不再赘述。
本发明的成像方法流程图如图2所示，首先，为单像素成像选择一种采样方案，比如哈达玛编码，同时针对该编码方式生成训练集，在仿真中采用手写数字识别数据集MNIST进行模型预训练；训练完毕后，把等于图像欠采样和全采样数量的编码掩膜图案分别发送到数字微镜器件上；把808nm激光照射在数字微镜器件上，使掩膜图案反射到目标图案区域的调制器上；用太赫兹激光发射后，通过太赫兹调制器调制808nm激光光强，同时通过一个太赫兹单点探测器进行太赫兹激光接收，获得一组一维的欠采样光强信号和一组一维全采样光强信号，即变换域数据；最后把欠采样的一维数据传入到图像重建网络模型SIDL中，恢复成32x32像素的图像，并与全采样后作逆变换的图像对比，进行图像质量评价。
本发明中，采用手写数字识别数据集MNIST进行实验仿真，训练集为60000张28x28像素的图像，验证集为10000张28x28像素图像，放大到32x32像素后作为原始图像，与加入高斯噪声并进行哈达玛编码的一维数据作为训练集。设置训练的参数为：训练轮数epoch为25，批次尺寸为batch_size为25，学习率learning_grate为10-5，损失函数Loss为L1函数。模拟实验中，针对图像采样率为0.8％和2.5％进行模型训练，验证集部分效果图如图5所示。
由于信息的大量缺失，在采样率为2.5％的图像经过哈达玛逆变换重建后基本无法辨认出数字内容；而采样率到了0.8％时，基本无法辨认出图像内容为数字；而经过SIDL模型重建后，通过输入信息的预测能够把0.8％的数据还原成较为相似的数字，而2.5％的数据能够极大程度重建成原始图像。图5的图像质量评价指标如表1所示。
表1
Figure PCTCN2022137703-appb-000002
表格中，分别对十张不同的图像两种采样率下分别进行哈达玛逆变换重建和SIDL算法重建，在传统算法中，2.5％的采样率下只能得到最高SSIM＝0.33和PSNR＝13的恢复效果，而0.8％下SSIM只在0.03-0.13之间，PSNR不超过13；而通过SIDL重建后，0.8％下SSIM在0.3-0.6之间，PSNR在12-21之间，而2.5％更是获得较好恢复，SSIM在0.75-0.92之间，PSNR在17-32之间。因此，在模型仿真中，我们认为在2.5％采样率下SIDL算法重建的效果最佳。
需要说明的是，在本申请中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本申请的具体实施方式，使本领域技术人员能够理解或实现本申请。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所申请的原理和新颖特点相一致的最宽的范围。

Claims (10)

  1. 一种太赫兹单像素成像方法，其特征在于，包括以下步骤：
    S1.构建图像重建网络模型，选择采样方案，基于所述采样方案对所述图像重建网络模型进行训练；
    S2.根据所述采样方案，生成相应的掩膜图案；
    S3.通过所述掩膜图案调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
    S4.将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像；
    所述图像重建网络模型包括全连接块和三角密集块，其中所述三角密集块包括卷积块、下采样层、上采样层和残差密集连接。
  2. 如权利要求1所述的太赫兹单像素成像方法，其特征在于，所述卷积块包括带有激活层的卷积层、普通卷积层和残差连接。
  3. 如权利要求1所述的太赫兹单像素成像方法，其特征在于，所述采样方案为哈达玛编码、傅里叶编码和小波变换编码。
  4. 如权利要求1所述的太赫兹单像素成像方法，其特征在于，所述步骤S3包括：将所述掩膜图案发送到数字微镜器件，用激光照射所述数字微镜器件，并把所述掩膜图案反射到目标图案区域的调制器上，之后发射太赫兹激光并利用太赫兹调制器调制激光光强，同时通过太赫兹单点探测器进行太赫兹激光接收，获得欠采样和全采样的一维数据。
  5. 如权利要求4所述的太赫兹单像素成像方法，其特征在于，所述步骤S4还包括：将所述全采样的一维数据做逆变换的图像处理， 将其与所述重建图像对比，进行图像质量评价。
  6. 如权利要求1所述的太赫兹单像素成像方法，其特征在于，所述步骤S1还包括：
    对所述图像重建网络模型进行预训练之前，根据重建图像的场景选择相应的数据集，并且给输入图像数据加入一定程度的随机噪声或额外的图像处理。
  7. 如权利要求1所述的太赫兹单像素成像方法，其特征在于，所述步骤S1包括：
    根据所述采样方案生成训练集，所述训练集包括原始图像，以及根据所述采样方案对所述图像进行逆变换转换成的一维信号。
  8. 一种端到端网络的太赫兹单像素成像系统，其特征在于，包括：
    训练单元，构建图像重建网络模型，选择采样方案，基于所述采样方案对所述图像重建网络模型进行训练；
    生成单元，根据所述采样方案，生成相应的掩膜图案；
    获取单元，通过所述掩膜图案调制太赫兹激光，调制后的太赫兹激光和目标物相互作用，形成的调制信号被探测器接收，并获得欠采样的一维数据；
    重建单元，将所述欠采样的一维数据导入到训练好的图像重建网络模型中，得到重建图像；
    所述图像重建网络模型包括全连接块和三角密集块组成，其中所述三角密集块包括卷积块、下采样层、上采样层和残差密集连接。
  9. 如权利要求8所述的太赫兹单像素成像系统，其特征在于，所述卷积块包括带有激活层的卷积层、普通卷积层和残差连接。
  10. 如权利要求8所述的太赫兹单像素成像系统，其特征在于，所述采样方案为哈达玛编码、傅里叶编码和小波变换编码。
PCT/CN2022/137703 2022-04-06 2022-12-08 一种太赫兹单像素成像方法与系统 WO2023193461A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210356149.8 2022-04-06
CN202210356149.8A CN114894740A (zh) 2022-04-06 2022-04-06 一种太赫兹单像素成像方法与系统

Publications (1)

Publication Number Publication Date
WO2023193461A1 true WO2023193461A1 (zh) 2023-10-12

Family

ID=82716271

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137703 WO2023193461A1 (zh) 2022-04-06 2022-12-08 一种太赫兹单像素成像方法与系统

Country Status (2)

Country Link
CN (1) CN114894740A (zh)
WO (1) WO2023193461A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117523095A (zh) * 2023-11-09 2024-02-06 深圳技术大学 一种基于深度学习的稀疏角度THz-CT图像重建方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114894740A (zh) * 2022-04-06 2022-08-12 深圳先进技术研究院 一种太赫兹单像素成像方法与系统
CN116245965A (zh) * 2022-11-25 2023-06-09 中国科学院深圳先进技术研究院 基于物理模型的太赫兹单像素实时成像方法及系统
CN116843788B (zh) * 2023-08-31 2023-11-28 清华大学 一种有限角层析成像方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110068268A1 (en) * 2009-09-18 2011-03-24 T-Ray Science Inc. Terahertz imaging methods and apparatus using compressed sensing
CN109297925A (zh) * 2018-10-09 2019-02-01 天津大学 一种基于分块压缩感知的太赫兹高分辨率快速成像装置
CN110231292A (zh) * 2019-06-28 2019-09-13 深圳先进技术研究院 一种单像素太赫兹波成像方法和系统
CN111736171A (zh) * 2020-07-28 2020-10-02 广东工业大学 一种单像素太赫兹检测系统及图像获取方法
CN112525851A (zh) * 2020-12-10 2021-03-19 深圳先进技术研究院 一种太赫兹单像素成像方法及其系统
CN114219843A (zh) * 2021-12-16 2022-03-22 河南工业大学 太赫兹光谱图像重构模型的构建方法及系统和应用
CN114894740A (zh) * 2022-04-06 2022-08-12 深圳先进技术研究院 一种太赫兹单像素成像方法与系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110068268A1 (en) * 2009-09-18 2011-03-24 T-Ray Science Inc. Terahertz imaging methods and apparatus using compressed sensing
CN109297925A (zh) * 2018-10-09 2019-02-01 天津大学 一种基于分块压缩感知的太赫兹高分辨率快速成像装置
CN110231292A (zh) * 2019-06-28 2019-09-13 深圳先进技术研究院 一种单像素太赫兹波成像方法和系统
CN111736171A (zh) * 2020-07-28 2020-10-02 广东工业大学 一种单像素太赫兹检测系统及图像获取方法
CN112525851A (zh) * 2020-12-10 2021-03-19 深圳先进技术研究院 一种太赫兹单像素成像方法及其系统
CN114219843A (zh) * 2021-12-16 2022-03-22 河南工业大学 太赫兹光谱图像重构模型的构建方法及系统和应用
CN114894740A (zh) * 2022-04-06 2022-08-12 深圳先进技术研究院 一种太赫兹单像素成像方法与系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117523095A (zh) * 2023-11-09 2024-02-06 深圳技术大学 一种基于深度学习的稀疏角度THz-CT图像重建方法

Also Published As

Publication number Publication date
CN114894740A (zh) 2022-08-12

Similar Documents

Publication Publication Date Title
WO2023193461A1 (zh) 一种太赫兹单像素成像方法与系统
WO2023109719A1 (zh) 一种太赫兹单像素超分辨成像方法和系统
CN112488924B (zh) 一种图像超分辨率模型训练方法、重建方法及装置
CN107817641B (zh) 基于多模光纤的单像素相机系统
CN112525851A (zh) 一种太赫兹单像素成像方法及其系统
CN110751700B (zh) 用于单像素成像的采样和重建集成深度学习网络的训练方法
CN113762460B (zh) 基于数值散斑的多模光纤传输图像迁移重构算法
Yang et al. License plate image super-resolution based on convolutional neural network
CN116957931A (zh) 一种基于神经辐射场的相机图像画质提升方法
WO2024109842A1 (zh) 基于物理模型的太赫兹单像素实时成像方法及系统
Pang et al. Lightweight multi-scale aggregated residual attention networks for image super-resolution
WO2024109836A1 (zh) 基于深度学习网络的水下单像素成像方法及系统
CN112113661B (zh) 一种深度学习型快照式光谱成像装置及其探测方法
CN113705340A (zh) 一种基于雷达遥感数据的深度学习变化检测方法
Qu et al. A demosaicing method for compressive color single-pixel imaging based on a generative adversarial network
Zou et al. Target recognition based on pre-processing in computational ghost imaging with deep learning
CN116091640A (zh) 一种基于光谱自注意力机制的遥感高光谱重建方法及系统
CN110599423A (zh) 一种基于深度学习CycleGAN模型处理SAR图像亮度补偿方法
CN115824412A (zh) 一种全偏振光谱成像装置及探测方法
CN115861749A (zh) 一种基于窗口交叉注意力的遥感图像融合方法
CN112907444A (zh) 一种基于复数域零样本学习的太赫兹图像超分辨重建方法
CN110044474B (zh) 一种基于插值算法的关联成像优化方法
CN112435200A (zh) 一种应用于目标检测的红外图像数据增强方法
Li et al. Deep-learning-based single-photon-counting compressive imaging via jointly trained subpixel convolution sampling
Jiang et al. Infrared Single Pixel Imaging Based on Generative Adversarial Network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22936386

Country of ref document: EP

Kind code of ref document: A1