CN108171663B - Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks - Google Patents

Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks Download PDF

Info

Publication number
CN108171663B
CN108171663B CN201711416650.4A CN201711416650A CN108171663B CN 108171663 B CN108171663 B CN 108171663B CN 201711416650 A CN201711416650 A CN 201711416650A CN 108171663 B CN108171663 B CN 108171663B
Authority
CN
China
Prior art keywords
layer
image
feature map
deconvolution
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711416650.4A
Other languages
Chinese (zh)
Other versions
CN108171663A (en
Inventor
左旺孟
颜肇义
李晓明
山世光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology Shenzhen
Original Assignee
Harbin Institute of Technology Shenzhen
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology Shenzhen filed Critical Harbin Institute of Technology Shenzhen
Priority to CN201711416650.4A priority Critical patent/CN108171663B/en
Publication of CN108171663A publication Critical patent/CN108171663A/en
Application granted granted Critical
Publication of CN108171663B publication Critical patent/CN108171663B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

基于特征图最近邻替换的卷积神经网络的图像填充系统，属于图像填充技术领域，解决了现有图像填充方法无法快速地获得整体语义一致且具有良好清晰度的填充图像的问题。所述系统：生成网络对待填充图像先编码后解码，得到已填充图像。生成网络的解码器包括N个反卷积层，对于第一反卷积层～第N‑1反卷积层中的任意M个反卷积层，生成网络基于每个反卷积层的输出结果和该反卷积层对应的卷积层的输出结果，并采用特征图最近邻替换的方式得到附加特征图，并将每个反卷积层的输出结果、该反卷积层对应的卷积层的输出结果和附加特征图共同作为下一反卷积层的输入对象。判别网络用于判断已填充图像是否为待填充图像对应的真实图像。

Figure 201711416650

An image filling system based on a convolutional neural network with feature map nearest neighbor replacement belongs to the technical field of image filling, and solves the problem that the existing image filling method cannot quickly obtain a filling image with consistent overall semantics and good definition. The system: the generation network encodes and then decodes the image to be filled to obtain the filled image. The decoder of the generation network includes N deconvolution layers. For any M deconvolution layers in the first deconvolution layer to the N-1th deconvolution layer, the generation network is based on the output of each deconvolution layer. The result is the output result of the convolution layer corresponding to the deconvolution layer, and the additional feature map is obtained by replacing the nearest neighbor of the feature map, and the output result of each deconvolution layer, the volume corresponding to the deconvolution layer The output of the convolutional layer and the additional feature map are jointly used as the input object of the next deconvolutional layer. The discriminant network is used to judge whether the filled image is the real image corresponding to the image to be filled.

Figure 201711416650

Description

Image filling system of convolutional neural network based on feature map nearest neighbor replacement
Technical Field
The invention relates to an image filling system, and belongs to the technical field of image filling.
Background
Image filling is a fundamental problem in the field of computer vision and image processing, and is mainly used for performing restoration reconstruction on damaged images or removing unnecessary objects in the images.
The existing image filling methods mainly include a diffusion-based image filling method, a sample-based image filling method and a depth learning-based image filling method.
The basic idea of the diffusion-based image filling method is as follows: and diffusing the image information at the edge of the region to be filled into the inner part of the region to be filled by taking the pixel points as units. When the area of the area to be filled is small, the structure is simple, and the texture is single, the image filling method can well complete the image filling task. However, when the area of the region to be filled is large, the definition of the filled image obtained by the image filling method is poor.
The basic idea of the sample-based image filling method is as follows: and gradually filling the image blocks from the known area of the image to the area to be filled by taking the image blocks as units. And filling the area to be filled with the image blocks which are most similar to the image blocks at the edge of the area to be filled in the known area of the image each time the image blocks are filled. Compared with the image filling method based on diffusion, the filled image obtained by the image filling method based on the sample has better texture and higher definition. However, since the sample-based image filling method gradually replaces the unknown image blocks in the region to be filled with similar image blocks in the known region of the image, a filled image with uniform overall semantics cannot be obtained by using the image filling method.
The image filling method based on deep learning mainly refers to the application of a deep neural network to the field of image filling. Currently, it is proposed to use an encoder-decoder network to perform image filling on an image with missing intermediate regions. However, this image filling method is only applicable to 128 × 128 RGB images. Although the filled image obtained by the image filling method can meet the requirement of uniform overall semantics, the definition of the filled image is poor. In response to this problem, some researchers have attempted to perform a clear filling of large graphs using multi-scale iterative updating. However, although such image filling methods result in filled images with overall semantic consistency and good sharpness, they are extremely slow. In the Titan X display running environment, it takes tens of seconds to several minutes to fill a 256 × 256 RGB image.
Disclosure of Invention
The invention provides an image filling system of a convolutional neural network based on nearest neighbor replacement of a feature map, which aims to solve the problem that the existing image filling method cannot quickly obtain a filled image with consistent integral semantics and good definition.
The image filling system of the convolutional neural network based on feature map nearest neighbor replacement comprises a generating network and a judging network;
the generation network comprises an encoder and a decoder, wherein the encoder comprises N convolution layers, the decoder comprises N deconvolution layers, and N is more than or equal to 2;
the generation network obtains the filled image by a mode of firstly encoding and then decoding the image to be filled;
for any M deconvolution layers from the first deconvolution layer to the N-1 th deconvolution layer, generating a network based on an output result of each deconvolution layer and an output result of a convolution layer corresponding to the deconvolution layer, obtaining an additional feature map by adopting a feature map nearest neighbor replacement mode, and taking the output result of each deconvolution layer, the output result of the convolution layer corresponding to the deconvolution layer and the obtained additional feature map as input objects of the next deconvolution layer;
1≤M≤N-1；
the judgment network is used for judging whether the filled image is a real image corresponding to the image to be filled, and further restricting the weight learning of the generated network.
Preferably, the encoder includes a convolutional layer E1-convolutional layer E8The decoder includes an deconvolution layer D1Inverse convolution layer D8；
The image to be filled is a convolution layer E1The input object of (1);
for convolution layer E1-convolutional layer E8The output result of the former is used as the input object of the latter after being sequentially subjected to batch normalization and activation of a Leaky ReLU function;
convolution layer E8The output result of (A) is used as an deconvolution layer D after being sequentially subjected to batch normalization and activation of a Leaky ReLU function1The input object of (1);
deconvolution layer D1The output result of (A) is used as a deconvolution layer D after being activated by a ReLU function2The first input object of (1);
for deconvolution layer D2Inverse convolution layer D8The output result of the former is used as a first input object of the latter after being sequentially activated by a ReLU function and normalized in batches;
deconvolution layer D2Inverse convolution layer D8The second input object of (2) is a convolutional layer in turnE7-convolutional layer E1The output result is sequentially subjected to batch normalization and Leaky ReLU function activation;
deconvolution layer D after Tanh function activation8The output of (1) is a filled image;
convolution layer E1The convolution operation is used for performing 64 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E2The convolution operation is used for performing 128 4-by-4 convolution operations with the step size of 2 on the input object;
convolution layer E3The convolution operation is used for carrying out 256 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E4-convolutional layer E8All used for carrying out 512 convolution operations with 4 × 4 and 2 steps on the input object;
deconvolution layer D1Inverse convolution layer D4All used for carrying out 512 deconvolution operations with 4 × 4 and step length of 2 on the input object;
deconvolution layer D5The deconvolution operation is used for carrying out 256 operations with 4 × 4 and the step size of 2 on the input object;
deconvolution layer D6The deconvolution operation is carried out on the input objects with 128 4 × 4 steps of 2;
deconvolution layer D7The deconvolution device is used for carrying out 64 deconvolution operations with 4 × 4 and the step size of 2 on the input object;
deconvolution layer D8The deconvolution operation is performed on the input object by 3 times 4 with the step size of 2;
generating networks based on deconvolution layer D5Output result of (2) and convolutional layer E3And obtaining an additional feature map by adopting a feature map nearest neighbor replacement mode, and taking the additional feature map as a deconvolution layer D6The third input object of (1).
Preferably, the generation network is based on the deconvolution layer D5Output result of (2) and convolutional layer E3The specific process of obtaining the additional feature map by adopting the feature map nearest neighbor replacement mode is as follows:
selecting a feature map to be assigned with feature values of 0The feature map and the deconvolution layer D5Output feature map of (2) and convolutional layer E3The output feature maps of (a) have equal channel numbers and the same space size;
calculating to obtain the deconvolution layer D5Mask region of the output feature map of (1) and convolution layer E3And simultaneously cutting the masked areas and the unmasked areas into a plurality of feature blocks;
the characteristic blocks are cuboids with the size of C h w, wherein C, h and w are deconvolution layers D respectively5The number of channels of the output characteristic diagram, the length of the cuboid and the width of the cuboid;
for each feature block p in the masked area1Selecting a feature block p from the plurality of feature blocks of the non-mask region1Nearest feature block p2；
Selecting a region to be assigned in the feature map to be assigned, wherein the region to be assigned and the feature block p1At the deconvolution layer D5The positions in the output feature map of (a) are consistent;
will the characteristic block p2The characteristic value of (2) is given to the area to be assigned.
Preferably, the feature block p2And feature block p1The cosine of (c) is closest.
Preferably, the calculation method of the masked area and the unmasked area of the output feature map is as follows:
giving a mask image to replace an image to be filled, wherein the mask image and the image to be filled have the same size, the number of channels is 1, and the characteristic value is 0 or 1;
0 represents that the corresponding position of the characteristic point on the image to be filled is a non-filling point;
1 represents that the corresponding position of the characteristic point on the image to be filled is a point to be filled;
calculating a mask region and a non-mask region of a feature map of a mask image through a convolution network, wherein the convolution network comprises a first convolution layer to a third convolution layer;
the mask image is an input object of the first convolution layer;
for the first convolution layer to the third convolution layer, the output result of the former is the input object of the latter;
the first convolution layer to the third convolution layer are all used for carrying out 1 convolution operation with 4 x 4 and step length of 2 on the input object;
the output result of the third convolution layer is a feature map of the mask image, the size of the feature map is 32 × 32, and the channel is 1;
for the feature map of the mask image, when one feature value of the feature map is larger than a set threshold value, judging that the feature point is a mask point, otherwise, judging that the feature point is a non-mask point;
the mask area of the feature map of the mask image is a set of mask points, and the unmasked area of the feature map of the mask image is a set of unmasked points;
the mask area of the output characteristic diagram is equal to the mask area of the characteristic diagram of the mask image, and the unmasked area of the output characteristic diagram is equal to the unmasked area of the characteristic diagram of the mask image.
Preferably, the generated network is trained in a guidance loss constraint mode, wherein the specific guidance loss constraint mode is to perform feature similarity constraint on the real image and the input image in any convolution layer or deconvolution layer in the network generation training process;
the input image is a real image subjected to the masking operation.
Preferably, the specific way of generating the network for training is as follows:
the target image IgtInputting the data into a generation network, calculating a mask region of a characteristic diagram of the l-th layer, and obtaining (phi)l(Igt))yInformation;
inputting the image I to be filled into a generation network, calculating a mask region of a characteristic diagram of the L-L layer, and obtaining (phi)L-l(I))yInformation;
at this point a guidance loss constraint L is definedg：
Figure BDA0001520819100000041
Where Ω is the mask area, L is the total number of layers to generate the network, y is any coordinate point within the mask area, ΦL-l(I) When the input object is an image to be filled, a characteristic diagram output by the network at the L-L level is generated, (phi)L-l(I))yInformation of y in the masked region of the output feature map for the L-L-th layer, Φl(Igt) When the input object is a target image, generating a characteristic diagram output by the network at the l-th layer (phi)l(Igt))yAnd the information of y in the mask area of the output characteristic diagram of the l-th layer.
Preferably, the discriminating network comprises a convolutional layer E9-convolutional layer E13；
Convolution layer E9The input object of (1) is a filled image;
convolution layer E9The output result of (A) is activated by a Leaky ReLU function and then used as a convolution layer E10The input object of (1);
for convolution layer E10-convolutional layer E13The output result of the former is used as the input object of the latter after being sequentially subjected to batch normalization and activation of a Leaky ReLU function;
convolutional layer E sequentially subjected to batch normalization and Sigmoid function activation13The output result of (1) is the output result of the discrimination network;
convolution layer E9The convolution operation is used for performing 64 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E10The convolution operation is used for performing 128 4-by-4 convolution operations with the step size of 2 on the input object;
convolution layer E11The convolution operation is used for carrying out 256 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E12The convolution operation is used for carrying out 512 convolution operations with 4 x 4 and step size of 1 on the input object;
convolution layer E13For performing 1 convolution operation with 4 × 4 and step size of 1 on the input object.
Preferably, the filled image is an RGB image of 256 × 256, and the convolution layer E is formed13The space of the output result is largeSmall 64 x 64, channel 1.
Preferably, the image population system is trained end-to-end using an Adam optimization algorithm.
The image filling system of the convolutional neural network based on feature map nearest neighbor replacement takes an image to be filled as an input object of the image, and performs feature map nearest neighbor replacement through intermediate output of a network decoding part, so that the filled image with integral semantic consistency and good definition can be obtained through one-time forward propagation. Compared with the existing image filling method, the image filling system can obtain the filled image more quickly because only one forward propagation is needed.
Drawings
The image filling system of the convolutional neural network based on feature map nearest neighbor replacement according to the present invention will be described in more detail below based on embodiments and with reference to the accompanying drawings, in which:
FIG. 1 is a block diagram of a network according to an embodiment;
FIG. 2 is a block diagram of a discrimination network according to an embodiment;
FIG. 3 is an arbitrarily missing image to be filled;
FIG. 4 is a filled image obtained after inputting any missing image to be filled into the generation network;
FIG. 5 is a missing-centered image to be filled;
fig. 6 is a filled image obtained by inputting an image to be filled with a missing center into the generation network.
Detailed Description
The image filling system based on the convolutional neural network with feature map nearest neighbor replacement according to the present invention will be further described with reference to the accompanying drawings.
Example (b): the present embodiment will be described in detail with reference to fig. 1 to 6.
The image filling system of the convolutional neural network based on feature map nearest neighbor replacement described in this embodiment includes a generation network and a discrimination network;
the generation network comprises an encoder and a decoder, wherein the encoder comprises N convolution layers, the decoder comprises N deconvolution layers, and N is more than or equal to 2;
the generation network obtains the filled image by a mode of firstly encoding and then decoding the image to be filled;
for any M deconvolution layers from the first deconvolution layer to the N-1 th deconvolution layer, generating a network based on an output result of each deconvolution layer and an output result of a convolution layer corresponding to the deconvolution layer, obtaining an additional feature map by adopting a feature map nearest neighbor replacement mode, and taking the output result of each deconvolution layer, the output result of the convolution layer corresponding to the deconvolution layer and the obtained additional feature map as input objects of the next deconvolution layer;
1≤M≤N-1；
the judgment network is used for judging whether the filled image is a real image corresponding to the image to be filled, and further restricting the weight learning of the generated network.
The encoder of the present embodiment includes a convolution layer E1-convolutional layer E8The decoder includes an deconvolution layer D1Inverse convolution layer D8；
The image to be filled is a convolution layer E1The input object of (1);
for convolution layer E1-convolutional layer E8The output result of the former is used as the input object of the latter after being sequentially subjected to batch normalization and activation of a Leaky ReLU function;
convolution layer E8The output result of (A) is used as an deconvolution layer D after being sequentially subjected to batch normalization and activation of a Leaky ReLU function1The input object of (1);
deconvolution layer D1The output result of (A) is used as a deconvolution layer D after being activated by a ReLU function2The first input object of (1);
for deconvolution layer D2Inverse convolution layer D8The output result of the former is used as a first input object of the latter after being sequentially activated by a ReLU function and normalized in batches;
deconvolution layer D2Inverse convolution layer D8The second input object of (2) is sequentially a convolutional layer E7-convolutional layer E1The output result is sequentially subjected to batch normalization and Leaky ReLU function activation;
deconvolution layer D after Tanh function activation8The output of (1) is a filled image;
convolution layer E1The convolution operation is used for performing 64 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E2The convolution operation is used for performing 128 4-by-4 convolution operations with the step size of 2 on the input object;
convolution layer E3The convolution operation is used for carrying out 256 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E4-convolutional layer E8All used for carrying out 512 convolution operations with 4 × 4 and 2 steps on the input object;
deconvolution layer D1Inverse convolution layer D4All used for carrying out 512 deconvolution operations with 4 × 4 and step length of 2 on the input object;
deconvolution layer D5The deconvolution operation is used for carrying out 256 operations with 4 × 4 and the step size of 2 on the input object;
deconvolution layer D6The deconvolution operation is carried out on the input objects with 128 4 × 4 steps of 2;
deconvolution layer D7The deconvolution device is used for carrying out 64 deconvolution operations with 4 × 4 and the step size of 2 on the input object;
deconvolution layer D8The deconvolution operation is performed on the input object by 3 times 4 with the step size of 2;
generating networks based on deconvolution layer D5Output result of (2) and convolutional layer E3And obtaining an additional feature map by adopting a feature map nearest neighbor replacement mode, and taking the additional feature map as a deconvolution layer D6The third input object of (1).
The generation network of this embodiment is based on deconvolution layer D5Output result of (2) and convolutional layer E3The specific process of obtaining the additional feature map by adopting the feature map nearest neighbor replacement mode is as follows:
selecting a feature map to be assigned with feature values of 0, and comparing the feature map with a deconvolution layer D5Output feature map of (2) and convolutional layer E3The output feature maps of (a) have equal channel numbers and the same space size;
calculating to obtain the deconvolution layer D5Mask region of the output feature map of (1) and convolution layer E3And simultaneously cutting the masked areas and the unmasked areas into a plurality of feature blocks;
the characteristic blocks are cuboids with the size of C h w, wherein C, h and w are deconvolution layers D respectively5The number of channels of the output characteristic diagram, the length of the cuboid and the width of the cuboid;
for each feature block p in the masked area1Selecting a feature block p from the plurality of feature blocks of the non-mask region1Nearest feature block p2；
Selecting a region to be assigned in the feature map to be assigned, wherein the region to be assigned and the feature block p1At the deconvolution layer D5The positions in the output feature map of (a) are consistent;
will the characteristic block p2The characteristic value of (2) is given to the area to be assigned.
The calculation mode of the mask region and the non-mask region of the output characteristic diagram is as follows:
giving a mask image to replace an image to be filled, wherein the mask image and the image to be filled have the same size, the number of channels is 1, and the characteristic value is 0 or 1;
0 represents that the corresponding position of the characteristic point on the image to be filled is a non-filling point;
1 represents that the corresponding position of the characteristic point on the image to be filled is a point to be filled;
calculating a mask region and a non-mask region of a feature map of a mask image through a convolution network, wherein the convolution network comprises a first convolution layer to a third convolution layer;
the mask image is an input object of the first convolution layer;
for the first convolution layer to the third convolution layer, the output result of the former is the input object of the latter;
the first convolution layer to the third convolution layer are all used for carrying out 1 convolution operation with 4 x 4 and step length of 2 on the input object;
the output result of the third convolution layer is a feature map of the mask image, the size of the feature map is 32 × 32, and the channel is 1;
for the feature map of the mask image, when one feature value of the feature map is larger than a set threshold value, judging that the feature point is a mask point, otherwise, judging that the feature point is a non-mask point;
the mask area of the feature map of the mask image is a set of mask points, and the unmasked area of the feature map of the mask image is a set of unmasked points;
the mask area of the output characteristic diagram is equal to the mask area of the characteristic diagram of the mask image, and the unmasked area of the output characteristic diagram is equal to the unmasked area of the characteristic diagram of the mask image.
The generation network of the embodiment is trained in a guidance loss constraint mode, wherein the specific guidance loss constraint mode is to perform feature similarity constraint on a real image and an input image in any convolutional layer or deconvolution layer in the network generation training process;
the input image is a real image subjected to the masking operation.
The specific way of training the generated network of this embodiment is as follows:
inputting the target image Igt into a generation network, calculating a mask region of a characteristic diagram of the l-th layer, and obtaining (phi)l(Igt))yInformation;
inputting the image I to be filled into a generation network, calculating a mask region of a characteristic diagram of the L-L layer, and obtaining (phi)L-l(I))yInformation;
at this point a guidance loss constraint L is definedg：
Figure BDA0001520819100000081
Where Ω is the mask area, L is the total number of layers to generate the network, y is any coordinate point within the mask area, ΦL-l(I) When the input object is an image to be filled, generatingCharacteristic diagram of network output at L-L layer (phi)L-l(I))yInformation of y in the masked region of the output feature map for the L-L-th layer, Φl(Igt) When the input object is a target image, generating a characteristic diagram output by the network at the l-th layer (phi)l(Igt))yAnd the information of y in the mask area of the output characteristic diagram of the l-th layer.
In addition, the image I to be filled is recorded as phi (I; W) after passing through the generation network, wherein W is a parameter for generating a network model. Defining reconstruction loss
Figure BDA0001520819100000082
Figure BDA0001520819100000083
For each (phi)L-l(I))yOf which is in contact with (phi)l(I))xThe distance of (d) is calculated as follows:
Figure BDA0001520819100000084
x is any coordinate point in the non-mask region (phi)l(I))xIs the information of x in the unmasked region of the output feature map of the l-th layer,
Figure BDA0001520819100000085
non-masked areas.
Wherein the distance metric is formulated as follows:
Figure BDA0001520819100000091
find the closest point x*After (y), using x*(y) substitution
Figure BDA0001520819100000092
In the same plane as y in the area
Figure BDA0001520819100000093
Is an additional feature map to be input into the next deconvolution layer.
Namely, the method comprises the following steps:
Figure BDA0001520819100000094
the discriminating network of this embodiment includes a convolutional layer E9-convolutional layer E13；
Convolution layer E9The input object of (1) is a filled image;
convolution layer E9The output result of (A) is activated by a Leaky ReLU function and then used as a convolution layer E10The input object of (1);
for convolution layer E10-convolutional layer E13The output result of the former is used as the input object of the latter after being sequentially subjected to batch normalization and activation of a Leaky ReLU function;
convolutional layer E sequentially subjected to batch normalization and Sigmoid function activation13The output result of (1) is the output result of the discrimination network;
convolution layer E9The convolution operation is used for performing 64 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E10The convolution operation is used for performing 128 4-by-4 convolution operations with the step size of 2 on the input object;
convolution layer E11The convolution operation is used for carrying out 256 convolution operations with 4 × 4 and the step size of 2 on the input object;
convolution layer E12The convolution operation is used for carrying out 512 convolution operations with 4 x 4 and step size of 1 on the input object;
convolution layer E13For performing 1 convolution operation with 4 × 4 and step size of 1 on the input object.
256 × 256 RGB image as filled image, convolution layer E13The spatial size of the output result of (1) is 64 × 64, and the channel is 1.
Determining if the network input is phi (I; W) or I generating the output of the networkgtGenerating a network and judging the network to carry out the confrontation training, and generating the confrontation loss L at the momentadv：
Figure BDA0001520819100000095
In the formula, pdata(Igt) For distribution of real images, pmiss(I) For the distribution of the input image, D (-) means that the image of the discrimination network input into the discrimination network is from pdata(Igt) Log is a logarithmic function, IgtIs the target image and I is the image to be filled.
Thus, when training the generating network, the total loss is L:
Figure BDA0001520819100000101
wherein λgAnd λadvAre all hyper-parameters.
Fig. 3 is an image to be filled which is arbitrarily missing, and fig. 4 is a filled image obtained after the image to be filled which is arbitrarily missing is input into the generation network. Comparing fig. 3 with fig. 4, it can be seen that: the image filling system of the convolutional neural network based on feature map nearest neighbor replacement is suitable for filling any missing image to be filled, and can obtain a good filling effect.
Fig. 5 is an image to be filled with a missing center, and fig. 6 is a filled image obtained by inputting the image to be filled with a missing center into the generation network. Comparing fig. 5 with fig. 6, it can be seen that: the image filling system of the convolutional neural network based on feature map nearest neighbor replacement is suitable for filling images to be filled with missing centers, and can obtain a good filling effect.
Through simulation experiments, the image filling system based on the convolutional neural network with feature map nearest neighbor replacement described in this embodiment takes about 80ms for a 256 × 256 RGB image. Compared with the existing image filling method which takes tens of seconds to several minutes, the image filling system of the embodiment has a significant improvement in filling speed.
The image filling system of the convolutional neural network based on feature map nearest neighbor replacement described in this embodiment performs end-to-end training by using an Adam optimization algorithm.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.

Claims (8)

1.基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，所述图像填充系统包括生成网络和判别网络；1. based on the image filling system of the convolutional neural network replaced by the nearest neighbor of feature map, it is characterized in that, described image filling system comprises generation network and discriminating network; 生成网络包括编码器和解码器，编码器包括N个卷积层，解码器包括N个反卷积层，N≥2；The generation network includes an encoder and a decoder, the encoder includes N convolutional layers, and the decoder includes N deconvolutional layers, N≥2; 生成网络通过对待填充图像先编码后解码的方式，得到已填充图像；The generation network obtains the filled image by encoding and then decoding the filled image; 对于第一反卷积层～第N-1反卷积层中的任意M个反卷积层，生成网络基于每个反卷积层的输出结果和该反卷积层对应的卷积层的输出结果，并采用特征图最近邻替换的方式得到附加特征图，并将每个反卷积层的输出结果、该反卷积层对应的卷积层的输出结果和得到的附加特征图共同作为下一反卷积层的输入对象；For any M deconvolution layers from the first deconvolution layer to the N-1th deconvolution layer, the generation network is based on the output result of each deconvolution layer and the corresponding convolution layer of the deconvolution layer. Output the results, and obtain additional feature maps by replacing the nearest neighbors of the feature maps, and use the output results of each deconvolution layer, the output results of the convolution layers corresponding to the deconvolution layer, and the obtained additional feature maps as the input object for the next deconvolution layer; 判别网络用于判断已填充图像是否为待填充图像对应的真实图像，进而对生成网络的权重学习进行约束；The discriminant network is used to judge whether the filled image is the real image corresponding to the image to be filled, so as to constrain the weight learning of the generation network; 编码器包括卷积层E1～卷积层E8，解码器包括反卷积层D1～反卷积层D8；The encoder includes convolution layers E 1 to convolution layers E 8 , and the decoder includes deconvolution layers D 1 to deconvolution layers D 8 ; 待填充图像为卷积层E1的输入对象；The image to be filled is the input object of the convolutional layer E1; 对于卷积层E1～卷积层E8，前者的输出结果在依次经批规范化和Leaky ReLU函数激活后，作为后者的输入对象；For the convolutional layers E 1 to E 8 , the output of the former is used as the input object of the latter after batch normalization and Leaky ReLU function activation in turn; 卷积层E8的输出结果在依次经批规范化和Leaky ReLU函数激活后，作为反卷积层D1的输入对象；The output result of the convolution layer E 8 is used as the input object of the deconvolution layer D 1 after batch normalization and Leaky ReLU function activation in turn; 反卷积层D1的输出结果在经ReLU函数激活后作为反卷积层D2的第一输入对象； The output result of the deconvolution layer D1 is used as the first input object of the deconvolution layer D2 after being activated by the ReLU function; 对于反卷积层D2～反卷积层D8，前者的输出结果在依次经ReLU函数激活和批规范化后，作为后者的第一输入对象；For the deconvolution layer D 2 to the deconvolution layer D 8 , the output result of the former is used as the first input object of the latter after being activated by the ReLU function and batch normalized in turn; 反卷积层D2～反卷积层D8的第二输入对象依次为卷积层E7～卷积层E1的依次经批规范化和Leaky ReLU函数激活后的输出结果；The second input objects of the deconvolution layer D 2 to the deconvolution layer D 8 are sequentially the output results of the convolution layer E 7 to the convolution layer E 1 after batch normalization and activation of the Leaky ReLU function; 经Tanh函数激活后的反卷积层D8的输出结果为已填充图像；The output result of the deconvolution layer D 8 activated by the Tanh function is a filled image; 卷积层E1用于对输入对象进行64个4*4、步长为2的卷积操作；The convolution layer E 1 is used to perform 64 convolution operations of 4*4 and stride 2 on the input object; 卷积层E2用于对输入对象进行128个4*4、步长为2的卷积操作；The convolution layer E 2 is used to perform 128 convolution operations of 4*4 with a stride of 2 on the input object; 卷积层E3用于对输入对象进行256个4*4、步长为2的卷积操作；The convolutional layer E3 is used to perform 256 4*4 convolution operations with a stride of 2 on the input object; 卷积层E4～卷积层E8均用于对输入对象进行512个4*4、步长为2的卷积操作；The convolutional layers E 4 to E 8 are all used to perform 512 4*4 convolution operations on the input object with a stride of 2; 反卷积层D1～反卷积层D4均用于对输入对象进行512个4*4、步长为2的反卷积操作；The deconvolution layer D 1 to the deconvolution layer D 4 are all used to perform 512 deconvolution operations of 4*4 and a step size of 2 on the input object; 反卷积层D5用于对输入对象进行256个4*4、步长为2的反卷积操作；The deconvolution layer D 5 is used to perform 256 4*4 deconvolution operations on the input object with a stride of 2; 反卷积层D6用于对输入对象进行128个4*4、步长为2的反卷积操作；The deconvolution layer D 6 is used to perform 128 4*4 deconvolution operations on the input object with a stride of 2; 反卷积层D7用于对输入对象进行64个4*4、步长为2的反卷积操作；The deconvolution layer D 7 is used to perform 64 deconvolution operations of 4*4 and stride 2 on the input object; 反卷积层D8用于对输入对象进行3个4*4、步长为2的反卷积操作；The deconvolution layer D 8 is used to perform three 4*4 deconvolution operations on the input object with a stride of 2; 生成网络基于反卷积层D5的输出结果和卷积层E3的输出结果，并采用特征图最近邻替换的方式得到附加特征图，并将该附加特征图作为反卷积层D6的第三输入对象；The generation network is based on the output results of the deconvolution layer D 5 and the output results of the convolution layer E 3 , and uses the feature map nearest neighbor replacement method to obtain an additional feature map, and uses the additional feature map as the deconvolution layer D 6 . the third input object; 生成网络基于反卷积层D5的输出结果和卷积层E3的输出结果，并采用特征图最近邻替换的方式得到附加特征图的具体过程为：The specific process of generating an additional feature map by using the nearest neighbor replacement method of feature map based on the output results of the deconvolution layer D 5 and the output results of the convolution layer E 3 is as follows: 选取一个特征值均为0的待赋值特征图，该特征图与反卷积层D5的输出特征图和卷积层E3的输出特征图具有相等的通道数和相同的空间大小；Select a feature map to be assigned whose feature value is 0, and the feature map and the output feature map of the deconvolution layer D 5 and the output feature map of the convolution layer E 3 have the same number of channels and the same space size; 计算得到反卷积层D5的输出特征图的掩膜区域和卷积层E3的输出特征图的非掩膜区域，并同时将所述掩膜区域和所述非掩膜区域切割为多个特征块；Calculate the mask area of the output feature map of the deconvolution layer D 5 and the non-mask area of the output feature map of the convolution layer E 3 , and simultaneously cut the mask area and the non-mask area into multiple feature block; 多个特征块均为长方体，其尺寸为C*h*w，其中，C、h和w分别为反卷积层D5的输出特征图的通道数、长方体的长度和长方体的宽度；A plurality of feature blocks are cuboid, and its size is C*h*w, wherein C, h and w are the number of channels of the output feature map of the deconvolution layer D5 , the length of the cuboid, and the width of the cuboid; 对于所述掩膜区域中的每个特征块p1，选取所述非掩膜区域的多个特征块中与特征块p1距离最近的特征块p2；For each feature block p 1 in the mask area, select a feature block p 2 that is closest to the feature block p 1 among the multiple feature blocks in the non-mask area; 选取待赋值特征图中的待赋值区域，该待赋值区域与特征块p1在反卷积层D5的输出特征图中的位置一致；Select the to-be-assigned area in the to-be-assigned feature map, and the to-be-assigned area is consistent with the position of the feature block p 1 in the output feature map of the deconvolution layer D5; 将特征块p2的特征值赋予所述待赋值区域。The feature value of feature block p 2 is assigned to the to-be-assigned area. 2.如权利要求1所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，特征块p2与特征块p1的余弦距离最近。2 . The image filling system of convolutional neural network based on feature map nearest neighbor replacement according to claim 1 , wherein the cosine distance between the feature block p 2 and the feature block p 1 is the closest. 3 . 3.如权利要求2所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，输出特征图的掩膜区域和非掩膜区域的计算方式为：3. the image filling system of the convolutional neural network based on the nearest neighbor replacement of feature map as claimed in claim 2, it is characterized in that, the calculation mode of the mask area of output feature map and non-mask area is: 给定一幅掩码图像来替代待填充图像，掩码图像与待填充图像的尺寸相同，通道数为1，特征值为0或1；Given a mask image to replace the image to be filled, the mask image has the same size as the image to be filled, the number of channels is 1, and the feature value is 0 or 1; 0表示该特征点在待填充图像上的相应位置为非待填充点；0 indicates that the corresponding position of the feature point on the image to be filled is not a point to be filled; 1表示该特征点在待填充图像上的相应位置为待填充点；1 indicates that the corresponding position of the feature point on the image to be filled is the point to be filled; 通过卷积网络来计算掩码图像的特征图的掩膜区域和非掩膜区域，该卷积网络包括第一卷积层～第三卷积层；The mask area and the non-mask area of the feature map of the mask image are calculated by a convolution network, the convolution network including the first convolution layer to the third convolution layer; 掩码图像为第一卷积层的输入对象；The mask image is the input object of the first convolutional layer; 对于第一卷积层～第三卷积层，前者的输出结果为后者的输入对象；For the first convolutional layer to the third convolutional layer, the output of the former is the input object of the latter; 第一卷积层～第三卷积层均用于对输入对象进行1个4*4、步长为2的卷积操作；The first convolutional layer to the third convolutional layer are all used to perform a 4*4 convolution operation on the input object with a stride of 2; 第三卷积层的输出结果为掩码图像的特征图，其尺寸为32*32，通道为1；The output of the third convolutional layer is the feature map of the mask image, its size is 32*32, and the channel is 1; 对于掩码图像的特征图，当其一个特征值大于设定的阈值时，判定该特征点为掩膜点，否则，判定该特征点为非掩膜点；For the feature map of the mask image, when one of its feature values is greater than the set threshold, it is determined that the feature point is a mask point, otherwise, the feature point is determined to be a non-mask point; 掩码图像的特征图的掩膜区域为掩膜点的集合，掩码图像的特征图的非掩膜区域为非掩膜点的集合；The mask area of the feature map of the mask image is a set of mask points, and the non-mask area of the feature map of the mask image is a set of non-mask points; 输出特征图的掩膜区域与掩码图像的特征图的掩膜区域相等，输出特征图的非掩膜区域与掩码图像的特征图的非掩膜区域相等。The masked area of the output feature map is equal to the masked area of the feature map of the mask image, and the non-masked area of the output feature map is equal to the non-masked area of the feature map of the mask image. 4.如权利要求3所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，生成网络采用引导损失约束的方式进行训练，引导损失约束的具体方式为在生成网络训练的过程中，在任意卷积层或反卷积层中对真实图像和输入图像进行特征相似约束；4. the image filling system of the convolutional neural network based on feature map nearest neighbor replacement as claimed in claim 3, it is characterized in that, the generation network adopts the mode of guiding loss constraint to carry out training, and the specific mode of guiding loss constraint is that in the generation network During the training process, the feature similarity constraints are imposed on the real image and the input image in any convolutional layer or deconvolutional layer; 输入图像为经掩膜操作的真实图像。The input image is the masked real image. 5.如权利要求4所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，生成网络进行训练的具体方式为：5. the image filling system of the convolutional neural network based on feature map nearest neighbor replacement as claimed in claim 4, is characterized in that, the concrete mode that generation network carries out training is: 将目标图像Igt输入至生成网络，计算第l层的特征图的掩膜区域，并得到(Φl(Igt))y信息；Input the target image I gt to the generation network, calculate the mask area of the feature map of the lth layer, and obtain (Φ l (I gt )) y information; 将待填充图像I输入至生成网络，计算第L-l层的特征图的掩膜区域，并得到(ΦL-l(I))y信息；Input the image I to be filled into the generation network, calculate the mask area of the feature map of the L1 layer, and obtain (Φ L1 (1)) y information; 此时定义引导损失约束Lg：The bootstrap loss constraint L g is now defined:
Figure FDA0002996341810000031
Figure FDA0002996341810000031
 式中，Ω是掩模区域，L为生成网络的总层数，y为掩模区域内的任一坐标点，ΦL-l(I)为当输入对象为待填充图像时，生成网络在第L-l层输出的特征图，(ΦL-l(I))y为第L-l层的输出特征图的掩膜区域中y的信息，Φl(Igt)为输入对象为目标图像时，生成网络在第l层输出的特征图，(Φl(Igt))y为第l层的输出特征图的掩膜区域中y的信息。In the formula, Ω is the mask area, L is the total number of layers of the generation network, y is any coordinate point in the mask area, Φ Ll (I) is when the input object is the image to be filled, the generation network in the Ll The feature map output by the layer, (Φ Ll (I)) y is the information of y in the mask area of the output feature map of the Ll layer, and Φ l (I gt ) is the input object when the target image is generated. The feature map output by the layer, (Φ l (I gt )) y is the information of y in the mask area of the output feature map of the lth layer. 6.如权利要求5所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，判别网络包括卷积层E9～卷积层E13；6. The image filling system based on the convolutional neural network replaced by the nearest neighbor of the feature map as claimed in claim 5, wherein the discriminating network comprises a convolutional layer E 9 to a convolutional layer E 13 ; 卷积层E9的输入对象为已填充图像；The input object of the convolutional layer E 9 is the filled image; 卷积层E9的输出结果经Leaky ReLU函数激活后，作为卷积层E10的输入对象；After the output result of the convolutional layer E9 is activated by the Leaky ReLU function, it is used as the input object of the convolutional layer E10 ; 对于卷积层E10～卷积层E13，前者的输出结果依次经批规范化和Leaky ReLU函数激活后，作为后者的输入对象；For the convolutional layer E 10 to the convolutional layer E 13 , the output result of the former is used as the input object of the latter after batch normalization and Leaky ReLU function activation in turn; 依次经批规范化和Sigmoid函数激活后的卷积层E13的输出结果为判别网络的输出结果；The output result of the convolutional layer E 13 after batch normalization and sigmoid function activation in turn is the output result of the discriminant network; 卷积层E9用于对输入对象进行64个4*4、步长为2的卷积操作；The convolution layer E 9 is used to perform 64 convolution operations of 4*4 and stride 2 on the input object; 卷积层E10用于对输入对象进行128个4*4、步长为2的卷积操作；The convolution layer E 10 is used to perform 128 convolution operations of 4*4 with a stride of 2 on the input object; 卷积层E11用于对输入对象进行256个4*4、步长为2的卷积操作；The convolutional layer E11 is used to perform 256 4*4 convolution operations on the input object with a stride of 2; 卷积层E12用于对输入对象进行512个4*4、步长为1的卷积操作；The convolution layer E 12 is used to perform 512 4*4 convolution operations with a stride of 1 on the input object; 卷积层E13用于对输入对象进行1个4*4、步长为1的卷积操作。The convolutional layer E 13 is used to perform a 4*4 convolution operation with a stride of 1 on the input object. 7.如权利要求6所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，已填充图像为256*256的RGB图像，卷积层E13的输出结果的空间大小为64*64，通道为1。7. the image filling system of the convolutional neural network based on feature map nearest neighbor replacement as claimed in claim 6, is characterized in that, the filled image is the RGB image of 256*256, the space of the output result of convolutional layer E 13 The size is 64*64, and the channel is 1. 8.如权利要求7所述的基于特征图最近邻替换的卷积神经网络的图像填充系统，其特征在于，所述图像填充系统采用Adam优化算法进行端对端的训练。8 . The image filling system based on convolutional neural network with feature map nearest neighbor replacement according to claim 7 , wherein the image filling system adopts Adam optimization algorithm to perform end-to-end training. 9 .
CN201711416650.4A 2017-12-22 2017-12-22 Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks Active CN108171663B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711416650.4A CN108171663B (en) 2017-12-22 2017-12-22 Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711416650.4A CN108171663B (en) 2017-12-22 2017-12-22 Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks

Publications (2)

Publication Number Publication Date
CN108171663A CN108171663A (en) 2018-06-15
CN108171663B true CN108171663B (en) 2021-05-25

Family

ID=62520202

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711416650.4A Active CN108171663B (en) 2017-12-22 2017-12-22 Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks

Country Status (1)

Country Link
CN (1) CN108171663B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109087375B (en) * 2018-06-22 2023-06-23 华东师范大学 Image Hole Filling Method Based on Deep Learning
CN108898647A (en) * 2018-06-27 2018-11-27 Oppo（重庆）智能科技有限公司 Image processing method, device, mobile terminal and storage medium
JP7202087B2 (en) * 2018-06-29 2023-01-11 日本放送協会 Video processing device
CN109300128B (en) * 2018-09-29 2022-08-26 聚时科技(上海)有限公司 Transfer learning image processing method based on convolution neural network hidden structure
WO2020098360A1 (en) * 2018-11-15 2020-05-22 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method, system, and computer-readable medium for processing images using cross-stage skip connections
DE102019201702A1 (en) * 2019-02-11 2020-08-13 Conti Temic Microelectronic Gmbh Modular inpainting process
CN110490203B (en) * 2019-07-05 2023-11-03 平安科技（深圳）有限公司 Image segmentation method and device, electronic equipment and computer readable storage medium
CN111242874B (en) * 2020-02-11 2023-08-29 北京百度网讯科技有限公司 Image restoration method, device, electronic equipment and storage medium
CN111614974B (en) * 2020-04-07 2021-11-30 上海推乐信息技术服务有限公司 Video image restoration method and system
CN112184566B (en) * 2020-08-27 2023-09-01 北京大学 An image processing method and system for removing attached water mist and water droplets

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104025588A (en) * 2011-10-28 2014-09-03 三星电子株式会社 Method and device for intra prediction of video
CN106952239A (en) * 2017-03-28 2017-07-14 厦门幻世网络科技有限公司 image generating method and device
CN107133934A (en) * 2017-05-18 2017-09-05 北京小米移动软件有限公司 Image completion method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10319076B2 (en) * 2016-06-16 2019-06-11 Facebook, Inc. Producing higher-quality samples of natural images

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104025588A (en) * 2011-10-28 2014-09-03 三星电子株式会社 Method and device for intra prediction of video
CN106952239A (en) * 2017-03-28 2017-07-14 厦门幻世网络科技有限公司 image generating method and device
CN107133934A (en) * 2017-05-18 2017-09-05 北京小米移动软件有限公司 Image completion method and device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
生成对抗映射网络下的图像多层感知去雾算法;李策等;《计算机辅助设计与图形学学报》;20171031;第29卷(第10期);全文 *
生成对抗网络理论模型和应用综述;徐一峰;《金华职业技术学院学报》;20170630;全文 *

Also Published As

Publication number Publication date
CN108171663A (en) 2018-06-15

Similar Documents

Publication Publication Date Title
CN108171663B (en) Image Filling System Based on Feature Map Nearest Neighbor Replacement with Convolutional Neural Networks
CN112784954B (en) Method and device for determining neural network
CN107529650B (en) Closed loop detection method and device and computer equipment
CN115204183B (en) Knowledge enhancement-based two-channel emotion analysis method, device and equipment
CN112446461B (en) A neural network model training method and device
CN111274999B (en) Data processing method, image processing device and electronic equipment
CN111105017A (en) Neural network quantization method and device and electronic equipment
WO2018068421A1 (en) Method and device for optimizing neural network
CN113065525A (en) Age recognition model training method, face age recognition method and related device
CN113971732A (en) Small target detection method and device, readable storage medium and electronic equipment
CN114266894B (en) Image segmentation method, device, electronic device and storage medium
CN114037893A (en) High-resolution remote sensing image building extraction method based on convolutional neural network
CN112101364B (en) Semantic segmentation method based on incremental learning of parameter importance
CN117274744B (en) Small target detection method based on graph attention network
CN112214775A (en) Injection type attack method and device for graph data, medium and electronic equipment
CN116704217B (en) Model training method, device and storage medium based on difficult sample mining
CN114462490B (en) Image target retrieval methods, retrieval devices, electronic devices and storage media
WO2024027068A1 (en) Attack method and device for evaluating robustness of object detection model
TWI803243B (en) Method for expanding images, computer device and storage medium
WO2022100607A1 (en) Method for determining neural network structure and apparatus thereof
CN119599946A (en) A road crack detection method, device and electronic equipment
CN111199507A (en) A kind of image steganalysis method, intelligent terminal and storage medium
CN120495285B (en) A stereo matching method for high-resolution stereo satellite panchromatic image pairs
WO2023231796A1 (en) Visual task processing method and related device thereof
CN106446844A (en) Pose estimation method, pose estimation device and computer system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant