CN111106836A

CN111106836A - Image reconstruction method and device

Info

Publication number: CN111106836A
Application number: CN201911059788.2A
Authority: CN
Inventors: 王萌萌; 陈晓康
Original assignee: Goertek Inc
Current assignee: Goertek Inc
Priority date: 2019-10-30
Filing date: 2019-10-30
Publication date: 2020-05-05

Abstract

The invention discloses an image reconstruction method and device. The method comprises the following steps: performing two-dimensional compression on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix to obtain a compressed sensing image, wherein each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image; constructing a deep convolutional neural network, and training the deep convolutional neural network by using a training image set to obtain a trained deep convolutional neural network; and inputting the compressed sensing image into a trained deep convolution neural network, and carrying out image reconstruction on the compressed sensing image by using the trained deep convolution neural network to obtain a reconstructed image of the image.

Description

Image reconstruction method and device

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image reconstruction method and an image reconstruction device.

Background

Compressed Sensing (CS) theory is an emerging sampling theory that has been proposed in recent years for sparse signals, and the simultaneous sampling and compression of signals is successfully achieved at a speed much lower than the nyquist frequency. The compressed sensing technology is applied to an image reconstruction system, and the problems of compression, transmission, reconstruction and the like at the receiving and transmitting ends of the system can be solved by sampling a small amount of data, so that the bandwidth resource waste and the hardware equipment cost in the transmission and storage processes are reduced. Therefore, the research of image compressed sensing has great significance.

The theory has been increasingly applied in the fields of information source coding, sensor network, signal detection, medical image processing, radar remote sensing, biosensing, mode recognition, blind source separation, spectrum analysis and the like.

A large number of excellent reconstruction algorithms appear in the field of compressed sensing images, but the traditional reconstruction algorithms are poor in real-time performance, deep learning is trained under a line, the compressed sensing images are reconstructed by using trained parameters, and good real-time performance is achieved. The prior art provides a plurality of depth learning based compressed sensing image reconstruction methods, such as a reconstruction network Reconnet method, which utilizes an observation matrix to perform one-dimensional compression on an image and does not utilize spatial structure information of the image to influence the reconstruction quality of the image.

Disclosure of Invention

The invention aims to provide a new technical scheme for image reconstruction.

According to a first aspect of the present invention, there is provided an image reconstruction method comprising: performing two-dimensional compression on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix to obtain a compressed sensing image, wherein each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image; constructing a deep convolutional neural network, and training the deep convolutional neural network by using a training image set to obtain a trained deep convolutional neural network; and inputting the compressed sensing image into a trained deep convolution neural network, and carrying out image reconstruction on the compressed sensing image by using the trained deep convolution neural network to obtain a reconstructed image of the image.

Optionally, performing two-dimensional compression on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix, including: constructing an observation matrix, wherein the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1; calculating a transpose matrix of the observation matrix; and performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix and the transposed matrix.

Optionally, the observation matrix is a number in a matrix elementThe value 1 distributes the row ladder matrix according with the preset rule, and the preset rule comprises: element a in the observation matrix_0,0＝a_0,1＝1，a_1,2＝a_1,3＝1，a_i,2i＝a_2,2i+1The parameter a is an element of the observation matrix, the subscript i of the parameter is a natural number greater than 1 and less than N-1, and N is the number of rows of the observation matrix.

Optionally, performing two-dimensional compression on the horizontal pixel direction and the vertical pixel direction of the original image by using the observation matrix to obtain a compressed sensing image, including: dividing an original image into a plurality of image blocks with uniform sizes; performing two-dimensional compression on the plurality of image blocks in the horizontal pixel direction and the vertical pixel direction by using the observation matrix to obtain a plurality of compressed image block data; and rearranging the compressed image block data into a plurality of column vectors respectively, wherein the plurality of column vectors are the compressed sensing image data.

Optionally, constructing a deep convolutional neural network comprises: constructing a deep convolutional neural network comprising an input layer, a full-link layer, a neuron reconstruction layer and a predetermined number of convolutional layers in series; and setting an activation function corresponding to each convolution layer, and connecting the corresponding activation function with the nonlinear characteristic after each convolution layer.

Optionally, constructing a deep convolutional neural network specifically includes: the method comprises the steps that column vector data with the same dimensionality as input data and output data of an input layer are constructed, the output data of a full connection layer are column vectors with the dimensionality higher than the dimensionality of the input data, and the column vector dimensionality of the output data of the input layer is the same as the column vector dimensionality of the input data of the full connection layer; constructing a column vector with the same dimension as that of output data of the full-connection layer as input data of the neuron reconstruction layer, wherein the output data of the neuron reconstruction layer is a one-dimensional matrix; constructing a serial three-layer convolutional layer, wherein input data of the first layer of convolutional layer is a one-dimensional matrix with the same matrix size as output data of the neuron recombination layer, and output data of the first layer of convolutional layer is a multi-dimensional matrix with the same matrix size as input data of the first layer of convolutional layer; the input data and the output data of the activation function connected with the first layer of convolution layer are both multidimensional matrixes which are the same as the output data of the first layer of convolution layer in matrix size and matrix dimension; the input data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size and matrix dimension with the output data of the first layer of convolution layer, and the output data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size with the input data and matrix dimension smaller than the input data; the input data and the output data of the activation function connected with the second layer of convolution layer are both multidimensional matrixes which are the same as the output data of the second layer of convolution layer in matrix size and matrix dimension; the input data of the third layer of the convolution layer is a multi-dimensional matrix with the same matrix size and matrix dimension as the output data of the second layer of the convolution layer, and the output data of the third layer of the convolution layer is a one-dimensional matrix with the same matrix size as the input data of the third layer of the convolution layer; the input data and the output data of the activation function connected with the convolution layer of the third layer are both one-dimensional matrixes with the same matrix size as the output data of the convolution layer of the third layer.

Preferably, constructing a deep convolutional neural network further comprises: the linear correction unit is used as an activation function connected with each convolution layer, and the maximum value of output data of the activation function connected with the third layer of convolution layers is set to be not more than 1.

Preferably, the image reconstruction of the compressed sensing image by using the trained deep convolutional neural network comprises: receiving the compressed sensing image data through an input layer, and inputting the compressed sensing image data to a full connection layer; performing dimensionality-up processing on the compressed sensing image data through a full connection layer, and inputting the compressed sensing image data subjected to dimensionality-up processing into a neuron reconstruction layer; and carrying out neuron recombination operation on the compressed sensing image data through a neuron recombination layer, and inputting the recombined compressed sensing image data into a serial convolution layer for reconstruction.

According to a second aspect of the present invention, there is also provided an image reconstruction apparatus comprising: the compression processing unit is used for performing two-dimensional compression on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix to obtain a compressed sensing image, and each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image; the preprocessing unit is used for constructing a deep convolutional neural network and training the deep convolutional neural network by utilizing a training image set to obtain a trained deep convolutional neural network; and the reconstruction unit is used for inputting the compressed sensing image into the trained deep convolution neural network, and performing image reconstruction on the compressed sensing image by using the trained deep convolution neural network to obtain a reconstructed image of the picture.

Optionally, the compression processing unit includes a matrix calculation module, a segmentation module, a compression module, and a rearrangement module; the matrix calculation module is to: constructing an observation matrix, wherein the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1; calculating a transpose matrix of the observation matrix; performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix and the transposed matrix; the segmentation module is used for segmenting the original image into a plurality of image blocks with uniform sizes; the compression module is used for performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the plurality of image blocks by using the observation matrix to obtain a plurality of compressed image block data; the rearrangement module is used for rearranging the plurality of compressed image block data into a plurality of column vectors respectively, wherein the plurality of column vectors are compressed sensing image data.

One beneficial effect of the invention is that: according to the method and the device provided by the embodiment of the invention, the traditional compressed sensing image reconstruction scheme is improved, the observation matrix is utilized to carry out two-dimensional compression on the original image, the spatial information of the original image is fully utilized, and the effect of improving the image reconstruction quality is achieved by improving the PSNR of the reconstructed image.

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly described below. It is appreciated that the following drawings depict only certain embodiments of the invention and are therefore not to be considered limiting of its scope. For a person skilled in the art, it is possible to derive other relevant figures from these figures without inventive effort.

FIG. 1 is a diagram of a hardware configuration of an image reconstruction system according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of an image reconstruction method according to an embodiment of the present invention;

FIG. 3 is an image reconstruction process according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a deep convolutional neural network framework according to an embodiment of the present invention;

FIG. 5 is a graphical illustration of a first activation function and a second activation function according to an embodiment of the invention;

FIG. 6 is a graphical illustration of a third activation function according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of an original image according to an embodiment of the invention;

FIG. 8 is a schematic diagram of a reconstructed image for image reconstruction using a contrast method according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a reconstructed image using an improved method for image reconstruction according to an embodiment of the invention;

fig. 10 is a block diagram of an image reconstruction apparatus according to an embodiment of the present invention.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

< example one >

Fig. 1 is a block diagram of a hardware configuration of an image reconstruction system 100 according to an embodiment of the present invention.

As shown in fig. 1, the image reconstruction system 100 includes an image acquisition apparatus 1000 and an image reconstruction apparatus 2000.

The image capturing apparatus 1000 is used to capture an image and provide the captured image to the image reconstructing apparatus 2000.

The image capturing apparatus 1000 may be any imaging device capable of taking pictures, such as a camera.

The image reconstruction apparatus 2000 may be any electronic device, such as a PC, a notebook computer, a server, or the like.

In the present embodiment, referring to fig. 1, the image reconstruction device 2000 may include a processor 2100, a memory 2200, an interface device 2300, a communication device 2400, a display device 2500, an input device 2600, a speaker 2700, a microphone 2800, and the like.

The processor 2100 may be a mobile version processor. The memory 2200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. The interface device 2300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 2400 can perform wired or wireless communication, for example, the communication device 2400 may include a short-range communication device, such as any device that performs short-range wireless communication based on a short-range wireless communication protocol, such as a Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, and the like, and the communication device 2400 may also include a remote communication device, such as any device that performs WLAN, GPRS, 2G/3G/4G/5G remote communication. The display device 2500 is, for example, a liquid crystal display, a touch display, or the like, and the display device 2500 is used to display the target image acquired by the image acquisition device. The input device 2600 may include, for example, a touch screen, a keyboard, and the like. A user can input/output voice information through the speaker 2700 and the microphone 2800.

In this embodiment, the memory 2200 of the image reconstruction device 2000 is configured to store instructions for controlling the processor 2100 to operate at least to perform an image reconstruction method according to any embodiment of the present invention. The skilled person can design the instructions according to the disclosed solution. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.

Although a plurality of devices of the image reconstruction device 2000 are illustrated in fig. 1, the present invention may relate only to some of the devices, for example, the image reconstruction device 2000 relates only to the memory 2200, the processor 2100, and the display device 2500.

In the present embodiment, the image capturing device 1000 is used for capturing an image and providing the image to the image reconstructing device 2000, and the image reconstructing device 2000 implements the method according to any embodiment of the present invention based on the image.

It should be understood that although fig. 1 shows only one image capturing device 1000 and one image reconstructing device 2000, the respective numbers are not meant to be limiting, and a plurality of image capturing devices 1000 and/or image reconstructing devices 2000 may be included in the image reconstructing system 100.

< example two >

An embodiment of the present invention further provides an image reconstruction method, and as shown in fig. 2, the image reconstruction method of the embodiment specifically includes the following steps S2100 to S2300:

in step S2100, two-dimensional compression is performed on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix, so as to obtain a compressed sensing image, where each pixel observation value of the compressed sensing image indicates a spatial correlation between adjacent elements of the original image.

The original image is an image obtained by photographing according to business requirements, and can be a medical image for medical monitoring.

In one example, the original image may be a digital image, which may be composed of an infinite number of pixel points. Here, the digital image may be displayed based on different color modes, such as, but not limited to, RGB (Red, Green, Blue) color mode, CMYK (Cyan, Magenta, Black) color mode, hsb (huesource bright) color mode, bitmap mode, and the like. In another example, the original image may also be an analog image.

In this embodiment, a compressed sensing observation technology may be adopted to perform compression processing on the original image, and the compressed sensing observation process is a process in which an observation matrix performs linear weighted summation on pixel values of the original image. The original image is subjected to compressed sensing observation processing to obtain feature data including spatial information of the original image, where the feature data may be feature vector data of 256 dimensions, and of course, may also be feature vector data of other dimensions, which is not limited herein.

At present, in a depth learning-based compressed sensing image reconstruction network, an observation matrix is mostly adopted to perform one-dimensional compression on an image, so that each pixel observation value in a compressed sensing image contains all information of the image, the spatial structure information of the image is not distinguished, that is, the spatial structure information of the image is not utilized in the reconstruction process, and the image reconstruction quality is poor.

Since a certain spatial correlation exists locally in the picture, the observation matrix which performs one-dimensional compression on the picture does not take the spatial correlation into consideration. In order to utilize the spatial correlation of the picture, in the present embodiment, the original image is observed by using an observation matrix, and each pixel observation value only contains pixel information of a local neighborhood (for example, 4 pixel values of 2 × 2) of the original image, but does not contain other pixel information far away from the local neighborhood. Therefore, in the embodiment, when reconstructing, the spatial correlation information between adjacent pixels can be utilized, the interference of a farther pixel to the current local adjacent pixel is also eliminated, and the reconstruction effect is improved. In other words, in this embodiment, the original image is two-dimensionally compressed by using the observation matrix in a special form, and spatial structure information of the original image is fully used, so as to achieve the purposes of improving Peak Signal to Noise Ratio (PSNR) of the reconstructed image and improving image reconstruction quality.

After performing step S2100, two-dimensional compression in the horizontal pixel direction and the vertical pixel direction is performed on the original image using the observation matrix to obtain a compressed sensing image, the process proceeds to step S2200.

Step S2200 is to construct a deep convolutional neural network, and train the deep convolutional neural network by using the training image set to obtain a trained deep convolutional neural network.

In this embodiment, the construction of the convolutional layer is mainly considered when constructing the deep convolutional neural network. When building convolutional layers, the number of convolutional layers may be equalized based on reconstruction performance and reconstruction efficiency. As shown in fig. 3, in an embodiment of the present invention, the deep convolutional neural network uses three convolutional layers, which may achieve a good reconstruction performance, and may also increase the depth of the deep convolutional neural network, that is, add more convolutional layers, but increase the amount of computation during network reconstruction, thereby reducing real-time performance. Therefore, the number of the buildup layers in the present embodiment is preferably three.

After step S2200 is executed to construct a deep convolutional neural network and the deep convolutional neural network is trained by using the training image set to obtain a trained deep convolutional neural network, the process proceeds to step S2300.

And S2300, inputting the compressed sensing image into a trained deep convolutional neural network, and carrying out image reconstruction on the compressed sensing image by using the trained deep convolutional neural network to obtain a reconstructed image of the picture.

According to the method provided by the embodiment of the invention, the traditional compressed sensing image reconstruction scheme is improved, the observation matrix is utilized to perform two-dimensional compression on the original image in two dimensions of the horizontal pixel direction and the vertical pixel direction, the spatial correlation information between adjacent pixels of the original image is fully utilized, and the effect of improving the image reconstruction quality is achieved by improving the PSNR of the reconstructed image.

< example three >

In this embodiment, the two-dimensional compression in the horizontal pixel direction and the vertical pixel direction is performed on the original image by using the observation matrix in the step S2100, and the obtaining of the compressed sensing image may further include the following steps S2110 to S2130:

step S2110, an observation matrix is constructed, the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1.

In this embodiment, the scale of the observation matrix is smaller than that of the prior art. For example, in the prior art, the size of the observation matrix used for one-dimensional compression is generally 272 × 1089-296208, while the size of the observation matrix in the present embodiment is 16 × 32-512, and the size of the observation matrix is much smaller than that of the prior art observation matrix.

And because the observation matrix of this embodiment is a sparse matrix, its elements are only 0 and 1 and the number of 1 is very small, so that the number of bits of storage is reduced, and storage is convenient, and the storage amount is 1024/296208 ═ 0.35% in the prior art.

In a specific example, the observation matrix is a row ladder matrix in which the distribution of the values 1 in the matrix elements conforms to a preset rule, and the preset rule includes: element a in the observation matrix_0,0＝a_0,1＝1，a_1,2＝a_1,3＝1，a_i,2i＝a_2,2i+11, the other elements in the observation matrix are all zero; wherein, the parameter a is an element of the observation matrix, the lower corner mark i of the parameter is a natural number which is more than 1 and less than N-1, and N is the row number of the observation matrix.

Illustratively, the matrix size of the observation matrix is 16 × 32, the matrix elements in the observation matrix include 32 1 s, and the distribution of the values 1 in the matrix elements of the observation matrix conforms to the above a_0,0＝a_0,1＝1，a_1,2＝a_1,3＝1，a_i,2i＝a_2,2i+1 Rule 1.

The row ladder observation matrix is as follows:

in step S2120, a transposed matrix of the observation matrix is calculated.

For the 16 × 32 scale row ladder observation matrix of the above example, the scale of its transpose is 32 × 16.

In step S2130, the original image is subjected to two-dimensional compression in the horizontal pixel direction and the vertical pixel direction using the observation matrix and the transposed matrix.

In the prior art, the image observation performed by using an observation matrix with the size of 272 × 1089 ═ 296208 is as follows: and using a product result y ' obtained by multiplying the observation matrix phi ' by the image data x ' as compressed sensing image data, thereby realizing one-dimensional image observation. The specific image observation process is shown by reference to formula (1), the parameter phi' in formula (1) is an observation matrix,

for elements of the observation matrix, the parameter x ' is the column vector data, x ' of the image '₁,x′₂,x′₃...x′₁₀₈₈,x′₁₀₈₉Is a vector element of a column vector, y 'is compressed perceptual image data, y'₁...y′₂₇₂Are pixel observations in compressed perceptual image data.

In the present embodiment, as shown in fig. 3, the process of observing an image by using an observation matrix with a size of 16 × 32 to 512 is as follows: the product result phi x between the observation matrix phi and the original image x is utilized to be further mixed with the transposition matrix phi of the original image phi^TAnd multiplying, and taking the obtained product result y as compressed sensing image data, thereby realizing two-dimensional image observation. The specific image observation process is shown by reference to formula (2), the parameter phi in formula (2) is an observation matrix,

for observing the elements of the matrix, the parameter x is the matrix data of the original image, and the parameter phi^TIs a transposed matrix, x_1,1,x_1,2,x_1,3...x_32,31,x_32,32Is matrix element of matrix data of original image, y is matrix data of compressed sensing image, y_1,1...y_16,16Are pixel observations in matrix data of a compressed perceptual image.

It can be seen that, when the observation matrix is used to perform image observation, the number of multiplication and the number of addition in the prior art are 272 × 1089 × 1 ═ 296208 and 272 × 1088 × 1 ═ 295936, respectively, while the number of multiplication and the number of addition in the present embodiment are 16 × 32 × 32+16 × 32 × 16 ═ 24576 and 16 × 3 × 32+16 × 3 × 16 ═ 23808, respectively, and the total number of multiplication and addition is 8.3% in the prior art, the compressed sensing observation process in the present embodiment can speed up the speed of compressing the sensing image and significantly reduce the required hardware storage resources.

< example four >

In this embodiment, the step S2100 of performing two-dimensional compression on the original image in the horizontal pixel direction and the vertical pixel direction by using the observation matrix to obtain the compressed sensing image may further include the following steps S2140 to S2160:

step S2140, the original image is divided into a plurality of image blocks of uniform size.

The original image may be segmented into a plurality of image blocks of uniform size, for example, the original image may be segmented into 32 x 32 small image blocks, depending on the image size.

Step S2150, performing two-dimensional compression on the plurality of image blocks in the horizontal pixel direction and the vertical pixel direction using the observation matrix, to obtain a plurality of compressed image block data.

In this embodiment, the image compression method in the third embodiment may be used to perform compression processing on each image block, for example, the pixel values of each image block are subjected to linear weighted summation processing by using the 16 × 32 row gradient observation matrix and the transposed matrix of 32 × 16, so as to obtain a compressed 16 × 16 image block.

Step S2160 rearranges the compressed image block data into a plurality of column vectors, respectively, where the plurality of column vectors are compressed sensing image data.

After the compressed 16 × 16 image blocks are obtained, the compressed 16 × 16 image blocks are rearranged into column vectors with the size of 256, and after a plurality of 16 × 16 image blocks are obtained, each image block is re-arranged into column vectors with the size of 256, so that a plurality of 256 column vectors are obtained.

According to the present embodiment, the compressed sensing process for an image is completed, and a compressed image including image space information is obtained.

< example five >

In this embodiment, the step S2200 is to construct a deep convolutional neural network, and train the deep convolutional neural network by using a neural network training image set, so as to obtain a trained deep convolutional neural network, which may further include the following steps S2210 to S2220:

step S2210, a deep convolutional neural network is constructed that includes an input layer, a fully connected layer, a neuron reconstruction layer, and a predetermined number of convolutional layers in series.

The deep convolutional neural network can be constructed on the basis of a keras deep learning framework, and the time consumption of image reconstruction is reduced.

Step S2220, set up the activation function corresponding to each convolution layer, connect the corresponding activation function with nonlinear characteristic after each convolution layer.

In one embodiment, referring to fig. 4, the construction of the deep convolutional neural network comprises the following steps S2211 to S2213:

step S2211, constructing column vector data in which the input data and the output data of the input layer have the same dimension, the output data of the full connection layer has a dimension higher than that of the input data, and the column vector dimension of the output data of the input layer is the same as that of the input data of the full connection layer.

As shown in fig. 4, the deep convolutional neural network includes an Input Layer (Input Layer), which is "Input" shown in fig. 4, and the Input data and the output data of the Input Layer are column vectors of 256 in one dimension, that is, the Input size of the Input Layer is 256, and the output size is 256.

The deep convolutional neural network further includes a Fully Connected Layer (Fully Connected Layer) Connected to the input Layer, that is, "dense _ 1" shown in fig. 4, the dimension of the output data of the Fully Connected Layer is higher than the dimension of the input data of the Fully Connected Layer, the output data and the input data of the Fully Connected Layer are column vectors, the dimension of the column vectors of the input data of the Fully Connected Layer is the same as the dimension of the column vectors of the output data of the input Layer, that is, the input size of the Fully Connected Layer is 256, and the output size is 1024.

Step S2212, the input data of the neuron reconstruction layer is constructed as a column vector with the same dimension as the output data of the full connection layer, and the output data of the neuron reconstruction layer is a one-dimensional matrix.

As shown in fig. 4, the deep convolutional neural network includes a neuron reconstruction layer connected to the fully-connected layer, i.e., "reshape _ 1" shown in fig. 3, and performs a neuron reconstruction operation using the neuron reconstruction layer to change the shape of the feature map, thereby facilitating subsequent convolutional layer processing. Referring to fig. 4, the input data of the neuron reconstruction layer is a column vector having the same dimension as the output data of the fully connected layer, that is, the output data of the neuron reconstruction layer is a column vector having a size of 1024, the output data of the neuron reconstruction layer is a one-dimensional matrix having a size of 32 × 32, and obviously, the matrix size may be other sizes, for example, 16 × 16.

Step S2213, a serial three-layer convolutional layer is constructed, in which,

the input data of the first layer of convolutional layer is a one-dimensional matrix with the same matrix size as the output data of the neuron recombination layer, and the output data of the first layer of convolutional layer is a multi-dimensional matrix with the same matrix size as the input data of the first layer of convolutional layer; the input data and the output data of the activation function connected with the first layer of convolution layer are both multidimensional matrixes which are the same as the output data of the first layer of convolution layer in matrix size and matrix dimension.

As shown in fig. 4, in the deep convolutional neural network shown in fig. 4 and including three convolutional layers, "conv _ 1" is the first convolutional layer, the convolutional kernel size is 11 × 11, and the number of feature maps is set to 64. The input data of the first convolutional layer is a one-dimensional matrix with the same matrix size as the output data of the neuron recombination layer, namely the input size of the first convolutional layer is a one-dimensional matrix of 32 × 32, and the output data of the first convolutional layer is a multi-dimensional matrix with the same matrix size as the input data of the first convolutional layer, namely the output size of the first convolutional layer is a multi-dimensional matrix of 32 × 32, the dimension of the multi-dimensional matrix is the same as the number of the characteristic maps, and the dimension of the multi-dimensional matrix is 64.

As shown in fig. 4, the input data and the output data of the first activation function relu _1 connected to the first layer convolution layer are both multidimensional matrices having the same matrix size and matrix dimension as the first layer convolution layer output data. The input and output sizes of the first activation function in this embodiment are each a 32 x 32 64-dimensional matrix.

The input data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size and matrix dimension with the output data of the first layer of convolution layer, and the output data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size with the input data and matrix dimension smaller than the input data; the input data and the output data of the activation function connected with the second layer of convolution layer are both multidimensional matrixes which are the same as the output data of the second layer of convolution layer in matrix size and matrix dimension.

As shown in fig. 4, in the deep convolutional neural network shown in fig. 4 and including three convolutional layers, "conv _ 2" is the second convolutional layer, the convolutional kernel size is 1 × 1, and the number of feature maps is set to 32. The input data of the second layer of convolution layer is a multi-dimensional matrix with the same matrix size and matrix dimension as the output data of the first layer of convolution layer, namely, the input size of the second layer of convolution layer is a 64-dimensional matrix with 32 x 32, and the output data of the second layer of convolution layer is a multi-dimensional matrix with the same matrix size as the input data of the second layer of convolution layer and smaller matrix dimension than the input data, namely, the output size of the second layer of convolution layer is a 32-dimensional matrix with 32 x 32, and the dimension of the 32-dimensional matrix is the same as the number of characteristic graphs and is 32.

As shown in fig. 4, the input data and the output data of the second activation function relu _2 connected to the second layer convolution layer are both multidimensional matrices having the same matrix size and matrix dimension as the second layer convolution layer output data. I.e. a 32-dimensional matrix with 32 x 32 input and output sizes for the second activation function in this embodiment.

The input data of the third layer of the convolution layer is a multi-dimensional matrix with the same matrix size and matrix dimension as the output data of the second layer of the convolution layer, and the output data of the third layer of the convolution layer is a one-dimensional matrix with the same matrix size as the input data of the third layer of the convolution layer; the input data and the output data of the activation function connected with the convolution layer of the third layer are both one-dimensional matrixes with the same matrix size as the output data of the convolution layer of the third layer.

As shown in fig. 4, in the deep convolutional neural network shown in fig. 4 and including three convolutional layers, "conv _ 3" is the third convolutional layer, the convolutional kernel size is 7 × 7, and the number of feature maps is set to 1. The input data of the third layer of convolution layer is a multi-dimensional matrix with the same matrix size and matrix dimension as the output data of the second layer of convolution layer, namely, the input data of the third layer of convolution layer is a 32-dimensional matrix with the matrix size of 32 x 32, and the output data of the third layer of convolution layer is a one-dimensional matrix with the same matrix size as the input data of the third layer of convolution layer and the matrix dimension smaller than the input data, namely, the output data of the third layer of convolution layer is a one-dimensional matrix with the output size of 32 x 32.

As shown in fig. 4, the input data and the output data of the third activation function relu _3 connected to the third convolutional layer are both one-dimensional matrices having the same matrix size and matrix dimension as the output data of the third convolutional layer. I.e. the input and output sizes of the third activation function in this embodiment are 32 x 32 one-dimensional matrices.

In one embodiment, the linear correction unit is used as an activation function connected to each convolutional layer, and the maximum value of the output data of the activation function connected to the convolutional layer of the third layer is set to be not more than 1, so as to ensure that the normalized pixel value is between 0 and 1.

The number of feature maps of the first layer of convolutional layer, the second layer of convolutional layer and the third layer of convolutional layer can be set optionally, and 1, 32 and 64 are all hyper-parameters. In addition, the first, second, and third activation functions in this embodiment do not change the shape of the tensor, for example, if the input size is (323264), the output size is (323264).

In this embodiment, the first activation function, the second activation function, and the third activation function are all functions having nonlinear characteristics, as shown in fig. 5, the first activation function is the same as the second activation function, the function curve is as shown in fig. 5, the third activation function is different from the first activation function, the function curve is as shown in fig. 6, and by the third activation function, it is ensured that the maximum value of data is not greater than 1, and quality analysis of reconstructed image data is facilitated.

The deep convolutional neural network is constructed through the above steps S2210 to S2220, and after the deep convolutional neural network is constructed, a training image set is established, for example, a training set used in the prior art, which includes 91 images, may be used. Extracting 32 x 32 image blocks from the image sets, sliding by 14 to generate 22227 32 x 32 image blocks, compressing the image blocks by using a row ladder observation matrix to generate 22227 16 x 16 image blocks, recombining each 16 x 16 image block to obtain a column vector with the size of 256, finally generating 22227 column vectors with the size of 256 as a training set, training the deep convolutional neural network by using the training set, and training by using an L1 type loss function in the training process to obtain a trained network.

After the trained deep convolutional neural network is obtained, the image reconstruction of the compressed sensing image by using the trained deep convolutional neural network in step S2300 includes steps S2310 to S2330:

step S2310, receiving the compressed sensing image data through the input layer, and inputting the compressed sensing image data to the full connection layer.

Step S2320, the compressed sensing image data is subjected to dimension-increasing processing through the full connection layer, and the compressed sensing image data subjected to dimension-increasing processing is input into the neuron reconstruction layer.

Step S2330, a neuron reorganization operation is carried out on the compressed sensing image data through a neuron reorganization layer, and the reorganized compressed sensing image data are input into a serial convolution layer to be reorganized.

Therefore, the reconstruction of the original image can be completed through the steps, and the reconstructed image has better image quality.

< example six >

To illustrate the effect of image reconstruction in this embodiment, the present embodiment respectively performs image reconstruction on a plurality of classical test images by using a compressed sensing image reconstruction method (hereinafter, referred to as a contrast method) in the prior art and an image reconstruction method (hereinafter, referred to as an improved method) provided by this embodiment.

In this embodiment, the main differences between the improved method and the comparative method include:

the contrast technology adopts an observation matrix with the size of 272 x 1089 to carry out one-dimensional compression on an original image, and adopts a convolution neural network to respectively carry out image reconstruction on 12 classical images in the table 1; the improved method adopts a row ladder observation matrix with the size of 16 x 32 and a transposed matrix with the size of 32 x 16 to carry out two-dimensional compression on an original image, and adopts a depth convolution neural network to respectively carry out image reconstruction on 12 classic images in the table 1.

Table 1 shows the results of reconstruction (mainly on PSNR) for various classical test images using the improved method and the comparative method.

Table 1:

name of test image	Prior Art	This example
			Monarch	24.31	28.12
Fingerprint	25.57	30.23
			Flintstones	22.45	26.60
House	28.46	32.55
			Parrot	25.59	29.96
Barbara	23.25	25.78
			Boats	27.30	30.47
Cameraman	23.15	26.19
			Foreman	29.47	35.29
Lena	26.54	30.43
			Peppers	24.77	27.26
Mean PSNR	25.53	29.35

Based on the disclosure of table 1, the PSNR values of the reconstructed images obtained by reconstructing the 12 classical test images in table 1 by using the comparison method are shown in the third column of table 1. Meanwhile, the PSNR values of the reconstructed images obtained by reconstructing the 12 classical test images in table 1 by the improved method are shown in the second column of table 1.

Compared with the contrast method, the improved method relates to parameters which are about one hundred thousand orders of magnitude less than the contrast method, the average value of the reconstruction result of 12 classical test images in the contrast method on the PSNR is 25.53, the improved method is 29.35, and the image reconstruction quality obtained by the improved method is obviously better than that obtained by the contrast method. Wherein a larger PSNR value indicates a higher image reconstruction quality.

To intuitively give the difference in quality of the reconstructed image between the improved method and the comparative method, the present embodiment is illustrated by fig. 7 to 9. Fig. 7 is an original image of a test image, fig. 8 is a reconstruction result obtained by image reconstruction using a contrast method, fig. 9 is a reconstruction result obtained by image reconstruction using an improved method, and for the convenience of observing contrast, a central block region of the reconstruction result of fig. 8 and 9 is displayed in an enlarged manner in a block region at the lower right corner of the image, and as can be seen from comparison with fig. 7-9, details of an enlarged schematic diagram of the central block region of the reconstructed image in fig. 9 are clearly clearer and are closer to details of the original image in fig. 7.

< example six >

Fig. 10 is a schematic block diagram of an image reconstruction apparatus 7000 according to an embodiment of the present invention.

As shown in fig. 10, the image reconstruction apparatus 7000 of the present embodiment may include: a compression processing unit 7100, a pre-processing unit 7200, and a reconstruction unit 7300.

A compression processing unit 7100, which performs two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix to obtain a compressed sensing image, wherein each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image;

the preprocessing unit 7200 is configured to construct a deep convolutional neural network, and train the deep convolutional neural network by using a training image set to obtain a trained deep convolutional neural network;

the reconstruction unit 7300 inputs the compressed sensing image into the trained deep convolutional neural network, and performs image reconstruction on the compressed sensing image by using the trained deep convolutional neural network to obtain a reconstructed image of the picture.

In one embodiment, the compression processing unit 7100 includes a matrix calculation module, a segmentation module, a compression module, and a rearrangement module;

the matrix calculation module is to: constructing an observation matrix, wherein the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1; calculating a transpose matrix of the observation matrix; performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix and the transposed matrix;

the segmentation module is used for segmenting the original image into a plurality of image blocks with uniform sizes;

the compression module is used for performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the plurality of image blocks by using the observation matrix to obtain a plurality of compressed image block data;

the rearrangement module is used for rearranging the plurality of compressed image block data into a plurality of column vectors respectively, wherein the plurality of column vectors are compressed sensing image data.

In one embodiment, the pre-processing unit 7200 is configured to: constructing a deep convolutional neural network comprising an input layer, a full-link layer, a neuron reconstruction layer and a predetermined number of convolutional layers in series; and setting an activation function corresponding to each convolution layer, and connecting the corresponding activation function with the nonlinear characteristic after each convolution layer.

The pretreatment unit 7200 is specifically: the method comprises the steps that column vector data with the same dimensionality as input data and output data of an input layer are constructed, the output data of a full connection layer are column vectors with the dimensionality higher than the dimensionality of the input data, and the column vector dimensionality of the output data of the input layer is the same as the column vector dimensionality of the input data of the full connection layer;

constructing a column vector with the same dimension as that of output data of the full-connection layer as input data of the neuron reconstruction layer, wherein the output data of the neuron reconstruction layer is a one-dimensional matrix;

a serial, three-layer convolutional layer is constructed, in which,

the input data of the first layer of convolutional layer is a one-dimensional matrix with the same matrix size as the output data of the neuron recombination layer, and the output data of the first layer of convolutional layer is a multi-dimensional matrix with the same matrix size as the input data of the first layer of convolutional layer; the input data and the output data of the activation function connected with the first layer of convolution layer are both multidimensional matrixes which are the same as the output data of the first layer of convolution layer in matrix size and matrix dimension;

the input data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size and matrix dimension with the output data of the first layer of convolution layer, and the output data of the second layer of convolution layer is a multidimensional matrix which has the same matrix size with the input data and matrix dimension smaller than the input data; the input data and the output data of the activation function connected with the second layer of convolution layer are both multidimensional matrixes which are the same as the output data of the second layer of convolution layer in matrix size and matrix dimension;

the input data of the third layer of the convolution layer is a multi-dimensional matrix with the same matrix size and matrix dimension as the output data of the second layer of the convolution layer, and the output data of the third layer of the convolution layer is a one-dimensional matrix with the same matrix size as the input data of the third layer of the convolution layer; the input data and the output data of the activation function connected with the convolution layer of the third layer are both one-dimensional matrixes with the same matrix size as the output data of the convolution layer of the third layer. The linear correction unit is used as an activation function connected with each convolution layer, and the maximum value of output data of the activation function connected with the convolution layer of the third layer is not more than 1.

In one embodiment, the reconstruction unit 7300 is configured to:

receiving compressed sensing image data through an input layer, and inputting the compressed sensing image data to a full connection layer; performing dimensionality-up processing on the compressed sensing image data through a full connection layer, and inputting the compressed sensing image data subjected to dimensionality-up processing into a neuron reconstruction layer; and carrying out neuron recombination operation on the compressed sensing image data through a neuron recombination layer, and inputting the recombined compressed sensing image data into a serial convolution layer for reconstruction.

The specific implementation manner of each module in the apparatus embodiment of the present invention may refer to the related content in the method embodiment of the present invention, and is not described herein again.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. An image reconstruction method, comprising:

performing two-dimensional compression on an original image in a horizontal pixel direction and a vertical pixel direction by using an observation matrix to obtain a compressed sensing image, wherein each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image;

constructing a deep convolutional neural network, and training the deep convolutional neural network by using a training image set to obtain a trained deep convolutional neural network;

and inputting the compressed sensing image into the trained deep convolution neural network, and performing image reconstruction on the compressed sensing image by using the trained deep convolution neural network to obtain a reconstructed image of the picture.

2. The method of claim 1, wherein the two-dimensional compression of the original image in horizontal and vertical pixel directions using the observation matrix comprises:

constructing an observation matrix, wherein the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1;

calculating a transpose of the observation matrix;

and performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix and the transposed matrix.

3. The method of claim 2, wherein the observation matrix is a row ladder matrix with a distribution of values 1 in matrix elements according to a preset law, the preset law comprising: element a in the observation matrix_0,0＝a_0,1＝1，a_1,2＝a_1,3＝1，a_i,2i＝a_2,2i+1The parameter a is an element of the observation matrix, the subscript i of the parameter is a natural number which is greater than 1 and less than N-1, and N is the number of rows of the observation matrix.

4. The method according to claim 1, wherein the two-dimensional compression of the horizontal pixel direction and the vertical pixel direction of the original image by using the observation matrix to obtain the compressed sensing image comprises:

dividing the original image into a plurality of image blocks of uniform size;

performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the plurality of image blocks by using the observation matrix to obtain a plurality of compressed image block data;

and rearranging the compressed image block data into a plurality of column vectors respectively, wherein the column vectors are the compressed sensing image data.

5. The method of claim 1, wherein the constructing a deep convolutional neural network comprises:

constructing a deep convolutional neural network comprising an input layer, a full-link layer, a neuron reconstruction layer and a predetermined number of convolutional layers in series;

and setting an activation function corresponding to each convolution layer, and connecting the corresponding activation function with the nonlinear characteristic after each convolution layer.

6. The method according to claim 5, wherein the constructing the deep convolutional neural network specifically comprises:

constructing column vector data with the same dimensionality of input data and output data of the input layer, wherein the output data of the fully-connected layer is a column vector with the dimensionality higher than that of the input data of the fully-connected layer, and the column vector dimensionality of the output data of the input layer is the same as that of the input data of the fully-connected layer;

constructing input data of the neuron reconstruction layer as column vectors with the same dimension as that of output data of the full connection layer, wherein the output data of the neuron reconstruction layer is a one-dimensional matrix;

a serial, three-layer convolutional layer is constructed, in which,

the input data of the second layer of convolutional layer is a multidimensional matrix which has the same matrix size and matrix dimension as the output data of the first layer of convolutional layer, and the output data of the second layer of convolutional layer is a multidimensional matrix which has the same matrix size as the input data and matrix dimension smaller than the input data; the input data and the output data of the activation function connected with the second layer of convolution layer are both multidimensional matrixes which are the same as the output data of the second layer of convolution layer in matrix size and matrix dimension;

the input data of the third layer of convolutional layer is a multi-dimensional matrix which has the same matrix size and matrix dimension with the output data of the second layer of convolutional layer, and the output data of the third layer of convolutional layer is a one-dimensional matrix which has the same matrix size with the input data of the third layer of convolutional layer; and the input data and the output data of the activation function connected with the third layer of convolution layer are both one-dimensional matrixes with the same matrix size as the output data of the third layer of convolution layer.

7. The method of claim 6, wherein the constructing a deep convolutional neural network further comprises:

and taking a linear correction unit as the activation function connected with each convolution layer, and setting the maximum value of the output data of the activation function connected with the third layer of convolution layers to be not more than 1.

8. The method of claim 5, wherein the image reconstructing the compressed perceptual image using the trained deep convolutional neural network comprises:

receiving, by the input layer, the compressed sensing image data and inputting the compressed sensing image data to the fully-connected layer;

performing dimensionality-up processing on the compressed sensing image data through the full connection layer, and inputting the compressed sensing image data subjected to dimensionality-up processing into the neuron reconstruction layer;

and carrying out neuron recombination operation on the compressed sensing image data through the neuron recombination layer, and inputting the recombined compressed sensing image data into the serial convolution layer for reconstruction.

9. An image reconstruction apparatus, comprising:

the compression processing unit is used for performing two-dimensional compression on an original image in the horizontal pixel direction and the vertical pixel direction by utilizing an observation matrix to obtain a compressed sensing image, and each pixel observation value of the compressed sensing image indicates the spatial correlation between adjacent elements of the original image;

the preprocessing unit is used for constructing a deep convolutional neural network and training the deep convolutional neural network by utilizing a training image set to obtain a trained deep convolutional neural network;

and the reconstruction unit is used for inputting the compressed sensing image into the trained deep convolution neural network, and carrying out image reconstruction on the compressed sensing image by using the trained deep convolution neural network to obtain a reconstructed image of the image.

10. The apparatus of claim 9, wherein the compression processing unit comprises a matrix calculation module, a segmentation module, a compression module, and a reordering module;

the matrix calculation module is configured to: constructing an observation matrix, wherein the observation matrix is a sparse matrix, and matrix elements of the observation matrix only comprise 0 and 1; calculating a transpose of the observation matrix; performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the original image by using the observation matrix and the transposed matrix;

the compression module is used for performing two-dimensional compression in the horizontal pixel direction and the vertical pixel direction on the image blocks by using the observation matrix to obtain a plurality of compressed image block data;

the rearrangement module is configured to rearrange the plurality of compressed image block data into a plurality of column vectors respectively, where the plurality of column vectors are the compressed sensing image data.