CN114511644B - Adaptive digital camouflage method based on deep learning - Google Patents
Adaptive digital camouflage method based on deep learning Download PDFInfo
- Publication number
- CN114511644B CN114511644B CN202210069098.0A CN202210069098A CN114511644B CN 114511644 B CN114511644 B CN 114511644B CN 202210069098 A CN202210069098 A CN 202210069098A CN 114511644 B CN114511644 B CN 114511644B
- Authority
- CN
- China
- Prior art keywords
- convolution
- camouflage
- gating
- deep learning
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a self-adaptive digital camouflage method based on deep learning, and belongs to the field of image processing. According to the invention, the surrounding environment background is continuously scanned by devices such as a camera, and the environment background state corresponding to the position of the camouflage target is rapidly analyzed and calculated by utilizing the relation between the space information and the front and rear frames of the video, so that the real-time pixel level camouflage is carried out on the camouflage target, and the effect of visual stealth is achieved. The technology can realize good camouflage of stationary objects and moving objects, and has great significance for natural exploration, latent investigation, national security and scientific research.
Description
Technical Field
The invention belongs to the field of image processing, in particular to a technology for calculating the optimal camouflage color of a camouflage target through the color information of the surrounding environment, which can ensure that the camouflage target has great concealment.
Background
The conventional digital camouflage generation method is complicated in implementation steps, generally needs to generate digital camouflage patches through background dominant color extraction and patch extraction, and finally color filling is carried out on the generated patches to complete the whole digital camouflage generation process. Although the digital camouflage generated by the traditional method is widely applied, due to the visual persistence effect of human eyes, the method cannot realize good camouflage in the motion process of the camouflage object, and the self-adaptive digital camouflage technology can make up for the defects, so that the camouflage object can also realize good camouflage effect in the motion process. The self-adaptive digital camouflage algorithm based on deep learning is provided, and the generation of the end-to-end self-adaptive digital camouflage is realized. The method provided by the invention utilizes the deep neural network to extract time and space information from the reference frame, and can realize good camouflage in the motion process of the camouflage object.
Disclosure of Invention
The invention aims to solve the problem that the digital camouflage cannot ensure good camouflage of the moving object to be camouflaged in the moving process of the object to be camouflaged, and provides a solution for the problem, environmental information around the object to be camouflaged is acquired through equipment such as a camera, the camouflage color most similar to the surrounding environment is calculated for the moving object to be camouflaged at any time, the purpose of high speed and high efficiency is achieved, and a solid foundation is laid for further research of the self-adaptive digital camouflage technology.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: an adaptive digital camouflage method based on deep learning, the method comprising:
step 1: shooting an environmental information video around a current target to be camouflaged through a camera, and obtaining a video image closest to the current time;
step 2: making mask information on the video images of the six frames, so that a camouflage target is black in the mask video image, and environment information is white in the mask video image;
step 3: the color information obtained in the step 1 and the mask information obtained in the step 2 are used as inputs and sent to an adaptive camouflage network based on deep learning;
step 4: calculating the input obtained in the step 3 by utilizing the weight of the deep learning neural network trained in advance to obtain a result of the self-adaptive camouflage algorithm based on the deep learning;
step 5: extracting the result to obtain camouflage color information of the current frame, and performing color camouflage by utilizing the variable color metamaterial of the target to be camouflaged and the flexible OLED;
step 6: and (3) after the camouflage at the moment is finished, repeating the steps 1-5 until the camouflage task is finished;
the adaptive camouflage network for deep learning comprises an encoder and a decoder, and has 17 layers; the encoder has 12 layers, namely a 6-layer serial space-time gating convolution module with different convolution kernels, a 4-layer serial cavity convolution module and a 2-layer serial space-time gating convolution module; the decoder has 5 layers, and is sequentially provided with a transposition convolution module, a gating convolution module, a transposition convolution module, a gating convolution module and a gating convolution module;
the space-time gating convolution module is used for obtaining a result by adopting a mode of combining gating convolution and space-time convolution simultaneously for continuous frame images input by the piece; the gating convolution is to obtain the result of one branch by using convolution operation on the original input, then to obtain the result of the other branch by using the same convolution mode and sigmoid activation function to perform feature selection on the original input, and then to multiply the results of the two branches to obtain the output of the time-space gating convolution; the space-time convolution is to move the front quarter of the channel number in the continuous multi-frame image back one frame, the first frame left is filled with 0, then move one frame forward for the next quarter, the information of the last frame left is filled with zero, and the rest half part is kept unchanged; then, carrying out feature extraction on the processed continuous multi-frame images by using two-dimensional convolution, and then merging feature information;
the cavity convolution module uses a convolution kernel to sample input continuous frame images at equal intervals;
and when the transposition convolution module is used, four sides of an input image are subjected to zero padding, then the convolution kernel is rotated by 180 degrees, and convolution is performed.
Further, in the step 1, color information of the surrounding environment of the sixty frames closest to the current time is obtained, the video images of the sixty frames are sampled at equal intervals, the sampling interval is ten frames, finally, 6 frames of video images are obtained, and the 6 frames of images are adopted for subsequent processing.
The invention aims to solve the problem that the digital camouflage cannot ensure good camouflage of the moving object to be camouflaged in the moving process of the object to be camouflaged, and provides a solution for the problem, environmental information around the object to be camouflaged is acquired through equipment such as a camera, the camouflage color most similar to the surrounding environment is calculated for the moving object to be camouflaged at any time, the purpose of high speed and high efficiency is achieved, and a solid foundation is laid for further research of the self-adaptive digital camouflage technology.
In order to meet the requirement of real-time performance, the invention adopts time-space gating convolution to replace 3D convolution, and can effectively reduce operation parameters; in the latter half of the neural network, a 2D convolution module of transpose convolution and gate control convolution is adopted, and only the image of the current frame is decoded to generate, so that the parameter number of the neural network can be reduced to the greatest extent, and the operation time is reduced. According to the experiment, under the computer condition that the CPU is AMD ryzen-5 3600CPU@3.60GHz, the display card is NVIDIA GeTorce GTX 1660Ti GPU, and the memory is 16G RAM, the operation speed can reach more than 12 frames/second.
Drawings
Fig. 1 is a flow chart of an adaptive digital camouflage technique.
Fig. 2 is a deep learning network structure of the present invention.
FIG. 3 is a graph of camouflage results from one experiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples, but embodiments of the invention are not limited thereto.
The algorithm flow chart of the invention is shown in fig. 1, and the self-adaptive digital camouflage method based on deep learning specifically comprises the following steps:
step 1: shooting an environmental information video around a current target to be camouflaged through a camera, acquiring color information of the surrounding environment of sixty frames closest to the current time, sampling the video images of the sixty frames at equal intervals, wherein the sampling interval is ten frames, and finally obtaining six frames of video images;
step 2: making mask information on the video images of the six frames so that a camouflage target is black (the pixel is 0) in the mask video image, and the environment information is white (the pixel is 255) in the mask video image;
step 3: the color information obtained in the step 1 and the mask information obtained in the step 2 are used as inputs and sent to an adaptive camouflage algorithm based on deep learning;
step 4: calculating the input obtained in the step 3 by utilizing the weight of the deep learning neural network trained in advance to obtain a result of the self-adaptive camouflage algorithm based on the deep learning;
step 5: and extracting the result to obtain camouflage color information of the current frame, and performing color camouflage by utilizing the variable color metamaterial of the target to be camouflaged and the flexible OLED.
Step 6: and (5) after the camouflage is finished at the moment, repeating the steps 1-5 until the camouflage task is finished.
For the adaptive camouflage algorithm based on deep learning of step 3 in the specific embodiment, the algorithm needs to be described in detail:
a. introduction to the overall structure of the algorithm. The algorithm is an end-to-end convolutional neural network and mainly comprises cavity convolution, time-space gating convolution and transposed convolution. The algorithm structure adopts a coder decoder structure, and has 17 layers, wherein the coder has 12 layers, namely a 6-layer serial space-time gating convolution module with different convolution kernels, a 2-layer serial cavity convolution module and a 2-layer serial space-time gating convolution module, and the decoder has 5 layers, namely a transposition convolution module, a gating convolution module, a transposition convolution module, a gating convolution module and a gating convolution module.
b. Introduction to hole convolution
Hole convolution is a convolution manner in which an input matrix is sampled at equal intervals using a convolution kernel. The cavity convolution has two main effects, namely, the first is that the receptive field can be enlarged while the spatial resolution is not reduced. Second is the ability to capture multi-scale context information.
c. Introduction to space-time gated convolution
The space-time gating convolution consists of a space-time convolution product and a gating part. Space-time convolution is to move the front quarter of the channel information back one frame, the first frame is filled with 0; and then moving one quarter forward by one frame, zero padding the information of the last frame, keeping the last half part unchanged, then carrying out feature extraction by using two-dimensional convolution, and then merging the feature information. The gating convolution is to divide the input into two parts, wherein one part adopts a convolution mode such as a space-time convolution mode, the other part uses a sigmoid activation function to perform feature selection after using the convolution, and then the result of the two parts is multiplied to obtain the output of the space-time gating convolution.
Gating y,x ＝∑∑W g *I
TsmFeature y,x ＝∑∑W f *I
Where Γ is a sigmoid function, outputting only 0 and 1,0 representing an invalid position, and 1 representing a valid position.
Is an activation function and may be a ReLU or an ELU, etc. W (W) f And W is g Representing two different spatio-temporal convolution kernels.
d. Introduction to transpose convolution
Transpose convolution is a special way of convolution that can output information that is larger than the input size, often used to upsample the image. The operation of transpose convolution is to zero-fill the four sides of the input, then rotate the convolution kernel by 180 degrees, and then perform normal convolution to obtain the final output.
Claims (2)
1. An adaptive digital camouflage method based on deep learning, the method comprising:
step 1: shooting an environmental information video around a current target to be camouflaged through a camera, and obtaining a video image closest to the current time;
step 2: making mask information on the video image so that the camouflage target is black in the mask video image and the environment information is white in the mask video image;
step 3: taking the video image obtained in the step 1 and the mask information obtained in the step 2 as inputs, and sending the inputs to a self-adaptive camouflage network based on deep learning;
step 4: calculating the input obtained in the step 3 by utilizing the weight of the deep learning neural network trained in advance to obtain a result of the self-adaptive camouflage algorithm based on the deep learning;
step 5: extracting the result to obtain camouflage color information of the current frame, and performing color camouflage by utilizing the variable color metamaterial of the target to be camouflaged and the flexible OLED;
step 6: and (3) after the camouflage at the moment is finished, repeating the steps 1-5 until the camouflage task is finished;
the adaptive camouflage network for deep learning comprises an encoder and a decoder, and has 17 layers; the encoder has 12 layers, namely a 6-layer serial space-time gating convolution module with different convolution kernels, a 4-layer serial cavity convolution module and a 2-layer serial space-time gating convolution module; the decoder has 5 layers, and is sequentially provided with a transposition convolution module, a gating convolution module, a transposition convolution module, a gating convolution module and a gating convolution module;
the space-time gating convolution module is used for simultaneously adopting a gating convolution and space-time convolution combined mode to obtain a result for the input continuous frame images; the gating convolution is to obtain the result of one branch by using convolution operation on the original input, then to obtain the result of the other branch by using the same convolution mode and sigmoid activation function to perform feature selection on the original input, and then to multiply the results of the two branches to obtain the output of the time-space gating convolution; the space-time convolution is to move the front quarter of the channel number in the continuous multi-frame image back one frame, the first frame left is filled with 0, then move one frame forward for the next quarter, the information of the last frame left is filled with zero, and the rest half part is kept unchanged; then, carrying out feature extraction on the processed continuous multi-frame images by using two-dimensional convolution, and then merging feature information;
the cavity convolution module uses a convolution kernel to sample input continuous frame images at equal intervals;
the transpose convolution module is used for carrying out zero padding on four sides of an input image, rotating a convolution kernel by 180 degrees and then carrying out convolution.
2. The adaptive digital camouflage method based on deep learning as claimed in claim 1, wherein the color information of the surrounding environment of the sixty frames closest to the current time is obtained in the step 1, the video images of the sixty frames are sampled at equal intervals, the sampling interval is ten frames, finally, 6 frames of video images are obtained, and the 6 frames of images are used for subsequent processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210069098.0A CN114511644B (en) | 2022-01-21 | 2022-01-21 | Adaptive digital camouflage method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210069098.0A CN114511644B (en) | 2022-01-21 | 2022-01-21 | Adaptive digital camouflage method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114511644A CN114511644A (en) | 2022-05-17 |
| CN114511644B true CN114511644B (en) | 2023-06-09 |
Family
ID=81549781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210069098.0A Active CN114511644B (en) | 2022-01-21 | 2022-01-21 | Adaptive digital camouflage method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114511644B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376236A (en) * | 2014-12-02 | 2015-02-25 | 上海出版印刷高等专科学校 | Scheme self-adaptive digital watermark embedding and extracting method based on camouflage technology |
| CN110245598A (en) * | 2019-06-06 | 2019-09-17 | 北京瑞莱智慧科技有限公司 | It fights sample generating method, device, medium and calculates equipment |
| WO2020037965A1 (en) * | 2018-08-21 | 2020-02-27 | 北京大学深圳研究生院 | Method for multi-motion flow deep convolutional network model for video prediction |
| CN112733744A (en) * | 2021-01-14 | 2021-04-30 | 北京航空航天大学 | Camouflage object detection model based on edge cooperative supervision and multi-level constraint |
| CN113538505A (en) * | 2021-06-17 | 2021-10-22 | 杭州电子科技大学 | Motion estimation system and method of single picture based on deep learning |
| WO2021244079A1 (en) * | 2020-06-02 | 2021-12-09 | 苏州科技大学 | Method for detecting image target in smart home environment |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4143655B2 (en) * | 2005-11-24 | 2008-09-03 | キヤノン株式会社 | Image processing apparatus, image processing method, program, storage medium, and system |
| CN104835104B (en) * | 2014-02-07 | 2017-11-03 | 湖南第一师范学院 | Digital picture active Camouflage project based on point shape |
| GB2532458B (en) * | 2014-11-19 | 2018-04-18 | Folium Optics Ltd | Adaptive camouflage device, system and method |
| US10282628B1 (en) * | 2016-04-08 | 2019-05-07 | Joshua LaRue | System for producing camouflage pattern based upon perception machine vision |
| US10776923B2 (en) * | 2018-06-21 | 2020-09-15 | International Business Machines Corporation | Segmenting irregular shapes in images using deep region growing |
| WO2020215236A1 (en) * | 2019-04-24 | 2020-10-29 | 哈尔滨工业大学（深圳） | Image semantic segmentation method and system |
| CN110348319B (en) * | 2019-06-18 | 2021-05-04 | 武汉大学 | Face anti-counterfeiting method based on face depth information and edge image fusion |
| CN111292220B (en) * | 2020-01-19 | 2021-09-28 | 西北工业大学 | Target camouflage image generation method for target image recognition of synthetic aperture radar |
| CN111142081B (en) * | 2020-04-07 | 2021-02-26 | 湖南赛博诺格电子科技有限公司 | Radar self-adaptive camouflage method based on digital coding super surface |
| CN112288647B (en) * | 2020-10-13 | 2022-06-03 | 武汉大学 | Remote sensing image cloud and shadow restoration method based on gating convolution |
| CN112446357B (en) * | 2020-12-15 | 2022-05-03 | 电子科技大学 | SAR automatic target recognition method based on capsule network |
| CN112801895B (en) * | 2021-01-15 | 2022-10-14 | 南京邮电大学 | Two-stage attention mechanism-based GAN network image restoration algorithm |
| CN112801914A (en) * | 2021-02-09 | 2021-05-14 | 北京工业大学 | Two-stage image restoration method based on texture structure perception |
| CN113450561B (en) * | 2021-05-06 | 2022-05-24 | 浙江工业大学 | Traffic speed prediction method based on space-time graph convolution-generation countermeasure network |
| CN113507625B (en) * | 2021-06-22 | 2022-08-19 | 西安理工大学 | Self-adaptive video restoration method |
| CN113298736B (en) * | 2021-06-24 | 2022-03-04 | 河北工业大学 | Face image restoration method based on face pattern |
-
2022
- 2022-01-21 CN CN202210069098.0A patent/CN114511644B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376236A (en) * | 2014-12-02 | 2015-02-25 | 上海出版印刷高等专科学校 | Scheme self-adaptive digital watermark embedding and extracting method based on camouflage technology |
| WO2020037965A1 (en) * | 2018-08-21 | 2020-02-27 | 北京大学深圳研究生院 | Method for multi-motion flow deep convolutional network model for video prediction |
| CN110245598A (en) * | 2019-06-06 | 2019-09-17 | 北京瑞莱智慧科技有限公司 | It fights sample generating method, device, medium and calculates equipment |
| WO2021244079A1 (en) * | 2020-06-02 | 2021-12-09 | 苏州科技大学 | Method for detecting image target in smart home environment |
| CN112733744A (en) * | 2021-01-14 | 2021-04-30 | 北京航空航天大学 | Camouflage object detection model based on edge cooperative supervision and multi-level constraint |
| CN113538505A (en) * | 2021-06-17 | 2021-10-22 | 杭州电子科技大学 | Motion estimation system and method of single picture based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114511644A (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106683067B (en) | Deep learning super-resolution reconstruction method based on residual sub-images | |
| CN109544448B (en) | Group network super-resolution image reconstruction method of Laplacian pyramid structure | |
| Zhao et al. | Invertible image decolorization | |
| CN113284051B (en) | Face super-resolution method based on frequency decomposition multi-attention machine system | |
| CN113096017A (en) | Image super-resolution reconstruction method based on depth coordinate attention network model | |
| CN110717868A (en) | Video high dynamic range inverse tone mapping model construction and mapping method and device | |
| CN113658057A (en) | Swin transform low-light-level image enhancement method | |
| CN112288630A (en) | Super-resolution image reconstruction method and system based on improved wide-depth neural network | |
| Wang et al. | Underwater image super-resolution and enhancement via progressive frequency-interleaved network | |
| CN116188274A (en) | Image super-resolution reconstruction method | |
| CN113379606B (en) | Face super-resolution method based on pre-training generation model | |
| CN114511644B (en) | Adaptive digital camouflage method based on deep learning | |
| CN112435165A (en) | Two-stage video super-resolution reconstruction method based on generation countermeasure network | |
| CN113313133A (en) | Training method for generating countermeasure network and animation image generation method | |
| CN111861877A (en) | Method and apparatus for video hyper-resolution | |
| Zhang et al. | Research on image super-resolution reconstruction based on deep learning | |
| CN115512039A (en) | 3D face construction method | |
| CN116266336A (en) | Video super-resolution reconstruction method, device, computing equipment and storage medium | |
| CN115294199A (en) | Underwater image enhancement and depth estimation method, device and storage medium | |
| CN115375540A (en) | Terahertz image super-resolution method based on deep learning algorithm | |
| CN114582002A (en) | Facial expression recognition method combining attention module and second-order pooling mechanism | |
| Xu et al. | Joint super-resolution and inverse tone-mapping: A feature decomposition aggregation network and a new benchmark | |
| CN117635478B (en) | Low-light image enhancement method based on spatial channel attention | |
| Terai et al. | Comparative study of Video/Image denoising algorithms based on Convolutional neural network CNN | |
| CN117834899A (en) | Video snapshot compression imaging method and system based on reconstruction difficulty perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |