CN110852424B - Processing method and device for countermeasure generation network - Google Patents
Processing method and device for countermeasure generation network Download PDFInfo
- Publication number
- CN110852424B CN110852424B CN201911122032.8A CN201911122032A CN110852424B CN 110852424 B CN110852424 B CN 110852424B CN 201911122032 A CN201911122032 A CN 201911122032A CN 110852424 B CN110852424 B CN 110852424B
- Authority
- CN
- China
- Prior art keywords
- matrix
- tensors
- generation network
- countermeasure generation
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a processing method and a device for an countermeasure generation network, wherein the method comprises the following steps: converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors; performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors; converting the kernel tensor into a matrix to obtain a kernel matrix; the nuclear matrix is replaced with a parameter matrix of a full-connection layer in a preset countermeasure generation network to obtain a new countermeasure generation network, and the technical problem that the existing countermeasure generation network has a plurality of parameters and high calculation amount, so that the requirement on hardware equipment is high is solved.
Description
Technical Field
The present disclosure relates to the field of deep learning technologies, and in particular, to a method and an apparatus for processing an countermeasure generation network.
Background
In recent years, convolutional neural networks are widely applied in the fields of image classification, face recognition, target detection and the like, along with the increase of the number of layers of the convolutional neural networks, the calculation complexity is higher and the calculation amount is larger, and in order to avoid the problem of network overfitting, more training data are needed when training the deep convolutional neural network. While collecting a large amount of training data requires a lot of manpower, material resources and time, and a large amount of training data cannot be obtained quickly, in order to solve these problems, an countermeasure generation network (Generative Adversarial Networks, GAN) is used in the prior art to perform data expansion.
The main task of the countermeasure generation network belongs to the application of deep learning in unsupervised, the countermeasure generation network is to automatically generate images or videos and the like which cannot be distinguished by human beings, the GAN mainly comprises two parts, one part is a generator, the other part is a discriminator, the generator is used for generating images or videos which are spurious, the discriminator is used for distinguishing whether the generated images or videos are real images or videos, the generator and the discriminator are mutually game, in the game process, the data generated by the generator are more and more real, the capability of the discriminator for distinguishing the data from the spurious is higher and higher, and balance is finally achieved. However, the generator and the arbiter are substantially composed of a convolutional layer and a fully-connected layer of the deep convolutional neural network, so that there are problems of more parameters and large calculation amount, and high requirements on hardware equipment are also caused.
Disclosure of Invention
The application provides a processing method and a processing device of an countermeasure generation network, which are used for solving the technical problem of high requirements on hardware equipment caused by multiple parameters and large calculation amount of the existing countermeasure generation network.
In view of this, a first aspect of the present application provides a method for processing an countermeasure generation network, including:
converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors;
performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors;
converting the kernel tensor into a matrix to obtain a kernel matrix;
and replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network.
Preferably, the converting the kernel tensor into a matrix to obtain a kernel matrix specifically includes:
selecting one target kernel tensor from all the kernel tensors to perform classical mode 2 expansion to obtain a first target matrix;
combining the first target matrix with non-target tensors in all the tensors to obtain a combined tensor;
performing classical mode 2 expansion on the combined tensors to obtain a second target matrix;
and carrying out reshape operation on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full connection layer.
Preferably, the merging operation of the first target matrix and all the kernel tensors except the target kernel tensor to obtain a merged tensor specifically includes:
multiplying the first target matrix by non-target ones of all the kernel tensors to obtain a combined tensor.
Preferably, the converting the parameter matrix of the full connection layer in the preset countermeasure generation network into tensors to obtain parameter tensors specifically includes:
and converting the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors based on bijection to obtain parameter tensors.
Preferably, the converting the parameter matrix of the full connection layer in the preset countermeasure generation network into tensors to obtain parameter tensors further includes:
and acquiring the preset countermeasure generation network.
A second aspect of the present application provides a processing apparatus for an countermeasure generation network, comprising:
the first conversion module is used for converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors;
the decomposition module is used for performing tensor ring decomposition on the parameter tensors to obtain a plurality of kernel tensors;
the second conversion module is used for converting the kernel tensor into a matrix to obtain a kernel matrix;
and the replacing module is used for replacing the parameter matrix of the full-connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network.
Preferably, the second conversion module specifically includes:
the first unfolding sub-module is used for selecting one target nuclear tensor from all the nuclear tensors to conduct classical mode 2 unfolding to obtain a first target matrix;
the merging submodule is used for merging the first target matrix with non-target nuclear tensors in all the nuclear tensors to obtain a merged tensor;
the second unfolding sub-module is used for carrying out classical mode 2 unfolding on the combined tensors to obtain a second target matrix;
and the remolding sub-module is used for carrying out reshape operation on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full-connection layer.
Preferably, the merging sub-module is specifically configured to:
multiplying the first target matrix by non-target ones of all the kernel tensors to obtain a combined tensor.
Preferably, the first conversion module is specifically configured to:
and converting the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors based on bijection to obtain parameter tensors.
Preferably, the method further comprises:
and the acquisition module is used for acquiring the preset countermeasure generation network.
From the above technical scheme, the application has the following advantages:
the application provides a processing method of an countermeasure generation network, which comprises the following steps: converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors; performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors; converting the kernel tensor into a matrix to obtain a kernel matrix; and replacing the parameter matrix of the full-connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network. According to the processing method of the countermeasure generation network, the parameter tensor is obtained by carrying out high-order processing on the parameter matrix of the full-connection layer; performing tensor loop decomposition on the obtained parameter tensors to obtain a plurality of kernel tensors; the method comprises the steps that a kernel matrix is obtained based on kernel tensor conversion, the kernel matrix is replaced with a parameter matrix of a full-connection layer in a preset countermeasure generation network, a new countermeasure generation network is obtained, when the obtained new countermeasure generation network is trained or tested, hardware equipment only needs to store a small number of kernel tensors and does not need to store the whole parameter matrix of the full-connection layer, occupied memory of the hardware equipment is greatly reduced, the running speed of the equipment is improved, the new countermeasure generation network can be embedded into low-end equipment, and the technical problem that the hardware equipment is required to be high due to the fact that the existing countermeasure generation network has a large number of parameters and large calculation amount is solved; and the parameter matrix of the full-connection layer is replaced by the nuclear matrix, so that the structure of the original full-connection layer is not changed, and the performance of the network is ensured.
Drawings
FIG. 1 is a flow chart of one embodiment of a method of processing an countermeasure generation network provided herein;
FIG. 2 is a flow chart of another embodiment of a method of processing an countermeasure generation network provided herein;
fig. 3 is a schematic structural diagram of an embodiment of a processing device for an countermeasure generation network provided in the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
For ease of understanding, referring to fig. 1, an embodiment of a processing method for an countermeasure generation network provided in the present application includes:
and step 101, converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors.
Step 102, performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors.
In order to solve the technical problem that the requirement on hardware equipment is high due to the fact that the existing countermeasure generation network is multiple in parameters and large in calculation amount, in the embodiment of the invention, a tensor ring decomposition method is adopted to compress the preset countermeasure generation network, most of parameters of the countermeasure generation network are in a full-connection layer, tensor ring decomposition is more effective for tensors of a high order, therefore, higher-order processing is carried out on a parameter matrix of the full-connection layer, the parameter matrix is converted into higher-order tensors, the parameter tensors are obtained, tensor ring decomposition is carried out on the obtained parameter tensors, a plurality of low-rank nuclear tensors are obtained, and the number of the nuclear tensors is preset.
Step 103, converting the kernel tensor into a matrix to obtain a kernel matrix.
And 104, replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network.
In the prior art, a tensor algebra method is adopted to compress the convolutional neural network, the operation of the convolutional neural network is regarded as a tensor operation to compress network parameters, and tensor algebra operation is used to replace the operation of a full-connection layer in the convolutional neural network, so that the original full-connection layer structure of the convolutional neural network is damaged, and the network performance is reduced. In order not to change the structure of the full connection layer, the obtained kernel tensor is converted to obtain a kernel matrix, and the kernel matrix is used for replacing a parameter matrix of the full connection layer in the preset countermeasure generation network, so that the structure of the original full connection layer is not changed, and the performance of the network is guaranteed.
According to the processing method of the countermeasure generation network, which is provided by the embodiment of the application, the parameter tensor is obtained by carrying out high-order processing on the parameter matrix of the full-connection layer; performing tensor loop decomposition on the obtained parameter tensors to obtain a plurality of kernel tensors; the method comprises the steps that a kernel matrix is obtained based on kernel tensor conversion, the kernel matrix is replaced with a parameter matrix of a full-connection layer in a preset countermeasure generation network, a new countermeasure generation network is obtained, when the obtained new countermeasure generation network is trained or tested, hardware equipment only needs to store a small number of kernel tensors and does not need to store the whole parameter matrix of the full-connection layer, occupied memory of the hardware equipment is greatly reduced, the running speed of the equipment is improved, the new countermeasure generation network can be embedded into low-end equipment, and the technical problem that the hardware equipment is required to be high due to the fact that the existing countermeasure generation network has a large number of parameters and large calculation amount is solved; and the parameter matrix of the full-connection layer is replaced by the nuclear matrix, so that the structure of the original full-connection layer is not changed, and the performance of the network is ensured.
For ease of understanding, referring to fig. 2, another embodiment of a processing method for generating a network for countering is provided herein, including:
step 201, acquiring a preset countermeasure generation network.
It should be noted that one of a plurality of types of countermeasure generation networks disclosed in the prior art may be acquired as a preset countermeasure generation network, or the countermeasure generation network may be constructed by a plurality of convolution layers and full connection layers as a preset countermeasure generation network, and the present invention is not limited in detail.
Step 202, converting a parameter matrix of a full connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors.
In order to solve the technical problem of high requirements on hardware equipment caused by multiple parameters and large calculation amount of the existing countermeasure generation network, in the embodiment of the present application, a tensor ring decomposition method is adopted to perform compression processing on the preset countermeasure generation network, and because most of parameters of the countermeasure generation network are in a full connection layer, tensor ring decomposition is more effective on tensors of a high order, higher order processing is performed on a parameter matrix of the full connection layer, and the parameter matrix is converted into a high order tensor, so as to obtain the parameter tensor.
In the embodiment of the application, the dual-shot is adopted to convert the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors to obtain parameter tensors, and the parameter matrix of the full-connection layer is assumed to be W epsilon R m×n ，W (t, l) is the value of the parameter matrix W at the position (t, l), t E [1, m]，l∈[1,n]Defining bijective ((v) 1 (t),…,v d (t)),(u 1 (l),…,u d (l) The bijection can map the values of the indexes t, l onto a vector, respectively, the number of elements in the vector and the formula +.>As many as the number of terms, i.e., d, elements at a certain position of the parameter matrix can be mapped to elements at a certain position of the tensor by bijection so that the parameter matrix is converted into a parameter tensor, e.g., assuming that the parameter matrix W ε R 8×8 The behavior of the parameter matrix is 8=2x2x2, the column is 8=2x2x2, and the parameter matrix can be transformed by bijectionChanging to a third-order tensor, defining a bijection of t, l for matrix element W (t, l), respectively, i.e., when t=1, a vector (1, 1); t=2, corresponds to one vector (1, 2); t=3, corresponds to one vector (1, 2, 1); t=4, corresponds to one vector (1, 2); t=5, corresponds to a vector (2, 1); t=6, corresponds to one vector (2, 1, 2); t=7, corresponds to one vector (2, 1); t=8, corresponds to one vector (2, 2); similarly, when l=1, a vector (1, 1) is corresponding; l=2, corresponds to one vector (1, 2); l=3, corresponds to one vector (1, 2, 1); l=4, corresponds to one vector (1, 2); l=5, corresponds to one vector (2, 1); l=6, corresponds to one vector (2, 1, 2); l=7, corresponds to one vector (2, 1); when l=8, a vector (2, 2) is corresponding, and when the parameter matrix W takes the element (2, 5), two vectors (1, 2) and (2, 1) can be obtained by the corresponding relation, i.e. W (t, l) =w ((v) 1 (t),u 1 (l)),…,(v d (t),u d (l) And) corresponding, W can be converted to a parameter tensor W ((v) 1 (t),u 1 (l)),…,(v k (t),u k (l),…,(v d (t),u d (l)))。
And 203, performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors.
The number of the tensor loop decomposed rank and the tensor number are preset, and the tensor loop decomposed parameter is decomposed to obtain a preset number of tensor loop, for example, the number of the tensor is set as d, and the decomposed rank is (r 1 ,r 2 ,…,r d+1 ) The kernel tensor can be expressed asSince the tensor ring decomposition is performed in the prior art, a detailed description of the tensor ring decomposition process is omitted here.
And 204, selecting a target kernel tensor from all the kernel tensors to perform classical mode 2 expansion to obtain a first target matrix.
It should be noted that, one target tensor may be randomly selected from all the tensors to perform classical mode 2 expansion to obtain the first target matrix, where the classical mode 2 expansion method belongs to the prior art, and detailed description of a specific classical mode 2 expansion process is omitted here.
Step 205, merging the first target matrix with non-target tensors in all the tensors to obtain a merged tensor.
It should be noted that, multiplying the first target matrix by the non-target tensors in all the tensors may be used to obtain the combined tensors.
And 206, performing classical mode 2 expansion on the combined tensors to obtain a second target matrix.
And 207, performing reshape operation on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full connection layer.
It should be noted that, according to the size of the parameter matrix of the full connection layer in the preset countermeasure generation network, the reshape operation is performed on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full connection layer, so that the parameter matrix of the full connection layer in the preset countermeasure generation network is replaced by the core matrix.
And step 208, replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network.
It should be noted that the mathematical expression of the full connection layer is:
y＝σ(Wx+b)；
wherein W.epsilon.R m×n ,x∈R n×1 ,b∈R m×1 ,y∈R m×1 。
Replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix, namely:
wherein Tr { Z 1 (v 1 (t),u 1 (l)),…Z k (v k (t),u k (l)),…,Z d (v d (t),u d (l) As (x)Nuclear matrix, Z k Is a nuclear tensor.
In the prior art, a tensor algebra method is adopted to compress the convolutional neural network, the operation of the convolutional neural network is regarded as a tensor operation to compress network parameters, and tensor algebra operation is used to replace the operation of a full-connection layer in the convolutional neural network, so that the original full-connection layer structure of the convolutional neural network is damaged, and the network performance is not ideal. In order not to change the structure of the full-connection layer, the obtained nuclear tensor is converted in the embodiment of the application, the obtained nuclear matrix is consistent with the parameter matrix of the original full-connection layer in size, the parameter matrix of the full-connection layer is replaced by the nuclear matrix, the structure of the original full-connection layer is not changed, and the performance of the network is guaranteed.
Wherein the resulting new challenge-generating network can be used to generate images or video to augment the data set. For example, noise and real data can be input into a new countermeasure generation network together, a generator and a discriminator in the new countermeasure generation network are trained until the new countermeasure generation network converges, a trained new countermeasure generation network is obtained, in the training process, hardware equipment only needs to store a few kernel tensors and does not need to store the whole parameter matrix of a full connection layer, occupied memory of the hardware equipment is greatly reduced, the running speed of the equipment is improved, the new countermeasure generation network can be embedded into low-end equipment, and the technical problem that the hardware equipment is required to be high due to the fact that the existing countermeasure generation network has a large number of parameters and large calculation amount is solved; and the obtained nuclear matrix is consistent with the parameter matrix of the original full-connection layer in size, and the parameter matrix of the full-connection layer is replaced by the nuclear matrix, so that the structure of the original full-connection layer is not changed, and the performance of the network is ensured.
For ease of understanding, referring to fig. 3, an embodiment of a processing apparatus for generating a network for countermeasure provided in the present application includes:
the first conversion module 301 is configured to convert a parameter matrix of a full connection layer in a preset challenge-generating network into a tensor, thereby obtaining a parameter tensor.
The decomposition module 302 is configured to perform tensor loop decomposition on the parameter tensor to obtain a plurality of kernel tensors.
A second conversion module 303, configured to convert the kernel tensor into a matrix, to obtain a kernel matrix.
And the replacing module 304 is configured to replace the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix, so as to obtain a new countermeasure generation network.
Further, the second conversion module 303 specifically includes:
the first expansion submodule 3031 is configured to select one target kernel tensor from all the kernel tensors to perform classical mode 2 expansion to obtain a first target matrix.
And the merging submodule 3032 is used for merging the first target matrix with non-target nuclear tensors in all the nuclear tensors to obtain merged tensors.
And a second expansion submodule 3033, configured to perform classical mode 2 expansion on the combined tensor to obtain a second target matrix.
And a remodeling submodule 3034, configured to perform reshape operation on the second target matrix to obtain a core matrix with the size identical to that of the parameter matrix of the full connection layer.
Further, the merging sub-module 3032 is specifically configured to:
multiplying the first target matrix by non-target ones of all the tensors to obtain a combined tensor.
Further, the first conversion module 301 is specifically configured to:
and converting the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors based on bijection to obtain parameter tensors.
Further, the method further comprises the following steps:
an acquisition module 305 is configured to acquire a preset countermeasure generation network.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to execute all or part of the steps of the methods described in the embodiments of the present application by a computer device (which may be a personal computer, a server, or a network device, etc.). And the aforementioned storage medium includes: u disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (8)
1. A method of processing an countermeasure generation network, comprising:
converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors;
performing tensor loop decomposition on the parameter tensors to obtain a plurality of kernel tensors;
converting the kernel tensor into a matrix to obtain a kernel matrix;
replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network, wherein the full connection layer of the new countermeasure generation network is as follows:
y＝σ(∑Tr(Z 1 (v 1 (t),u 1 (l)),…，Z k (v k (t),u k (l),…,Z d (v d (t),u d (l)))x+b)；
in Tr { Z 1 (v 1 (t),u 1 (l)),…Z k (v k (t),u k (l)),…,Z d (v d (t),u d (l) As a kernel matrix, Z) k As the nuclear tensor, x is the input of the full-connection layer, b is the bias term of the full-connection layer, and y is the output of the full-connection layer;
the noise and the real image data or video data are input into the new countermeasure generation network together so as to train a generator and a discriminator in the new countermeasure generation network until the new countermeasure generation network converges to obtain a trained new countermeasure generation network, wherein in the training process, the generator is used for generating images or videos with false and spurious, and the discriminator is used for discriminating whether the generated images or videos are real images or videos; the trained new countermeasure generation network is used for generating images or videos so as to expand a data set;
the step of converting the kernel tensor into a matrix to obtain a kernel matrix specifically comprises the following steps:
selecting one target kernel tensor from all the kernel tensors to perform classical mode 2 expansion to obtain a first target matrix;
combining the first target matrix with non-target tensors in all the tensors to obtain a combined tensor;
performing classical mode 2 expansion on the combined tensors to obtain a second target matrix;
and carrying out reshape operation on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full connection layer.
2. The method for processing the countermeasure generation network according to claim 1, wherein the merging operation is performed on the first target matrix and all the core tensors except the target core tensor to obtain a merged tensor, specifically including:
multiplying the first target matrix by non-target ones of all the kernel tensors to obtain a combined tensor.
3. The method for processing the challenge-generating network according to claim 1, wherein the step of converting the parameter matrix of the full-connection layer in the preset challenge-generating network into tensors to obtain parameter tensors specifically comprises:
and converting the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors based on bijection to obtain parameter tensors.
4. A method for processing an countermeasure generation network according to claim 1, wherein the converting the parameter matrix of the fully connected layer in the preset countermeasure generation network into tensors to obtain parameter tensors further includes:
and acquiring the preset countermeasure generation network.
5. A processing apparatus for an countermeasure generation network, comprising:
the first conversion module is used for converting a parameter matrix of a full-connection layer in a preset countermeasure generation network into tensors to obtain parameter tensors;
the decomposition module is used for performing tensor ring decomposition on the parameter tensors to obtain a plurality of kernel tensors;
the second conversion module is used for converting the kernel tensor into a matrix to obtain a kernel matrix;
the replacing module is used for replacing the parameter matrix of the full connection layer in the preset countermeasure generation network with the core matrix to obtain a new countermeasure generation network, and the full connection layer of the new countermeasure generation network is as follows:
y＝σ(∑Tr(Z 1 (v 1 (t),u 1 (l)),…，Z k (v k (t),u k (l),…,Z d (v d (t),u d (l)))x+b)；
in Tr { Z 1 (v 1 (t),u 1 (l)),…Z k (v k (t),u k (l)),…,Z d (v d (t),u d (l) As a kernel matrix, Z) k As the nuclear tensor, x is the input of the full-connection layer, b is the bias term of the full-connection layer, and y is the output of the full-connection layer;
the training module is used for inputting noise and real image data or video data into the new countermeasure generation network together so as to train a generator and a discriminator in the new countermeasure generation network until the new countermeasure generation network converges to obtain a trained new countermeasure generation network, and in the training process, the generator is used for generating a false and spurious image or video, and the discriminator is used for discriminating whether the generated image or video is a real image or video; the trained new countermeasure generation network is used for generating images or videos so as to expand a data set;
the second conversion module specifically includes:
the first unfolding sub-module is used for selecting one target nuclear tensor from all the nuclear tensors to conduct classical mode 2 unfolding to obtain a first target matrix;
the merging submodule is used for merging the first target matrix with non-target nuclear tensors in all the nuclear tensors to obtain a merged tensor;
the second unfolding sub-module is used for carrying out classical mode 2 unfolding on the combined tensors to obtain a second target matrix;
and the remolding sub-module is used for carrying out reshape operation on the second target matrix to obtain a core matrix with the same size as the parameter matrix of the full-connection layer.
6. The device for processing an countermeasure generation network of claim 5, wherein the merging submodule is specifically configured to:
multiplying the first target matrix by non-target ones of all the kernel tensors to obtain a combined tensor.
7. The device for processing an countermeasure generation network of claim 5, wherein the first conversion module is specifically configured to:
and converting the parameter matrix of the full-connection layer in the preset countermeasure generation network into tensors based on bijection to obtain parameter tensors.
8. The countermeasure generation network processing apparatus of claim 5, further comprising:
and the acquisition module is used for acquiring the preset countermeasure generation network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911122032.8A CN110852424B (en) | 2019-11-15 | 2019-11-15 | Processing method and device for countermeasure generation network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911122032.8A CN110852424B (en) | 2019-11-15 | 2019-11-15 | Processing method and device for countermeasure generation network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110852424A CN110852424A (en) | 2020-02-28 |
| CN110852424B true CN110852424B (en) | 2023-07-25 |
Family
ID=69601026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911122032.8A Active CN110852424B (en) | 2019-11-15 | 2019-11-15 | Processing method and device for countermeasure generation network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852424B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127297A (en) * | 2016-06-02 | 2016-11-16 | 中国科学院自动化研究所 | The acceleration of degree of depth convolutional neural networks based on resolution of tensor and compression method |
| CN109344966A (en) * | 2018-07-26 | 2019-02-15 | 广东工业大学 | A kind of method of the full Connection Neural Network of efficient tensorization |
| WO2019146189A1 (en) * | 2018-01-29 | 2019-08-01 | 日本電気株式会社 | Neural network rank optimization device and optimization method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10733505B2 (en) * | 2016-11-10 | 2020-08-04 | Google Llc | Performing kernel striding in hardware |
| CN108197525B (en) * | 2017-11-20 | 2020-08-11 | 中国科学院自动化研究所 | Face image generation method and device |
| CN110263913A (en) * | 2019-05-23 | 2019-09-20 | 深圳先进技术研究院 | A kind of deep neural network compression method and relevant device |
| CN110390647A (en) * | 2019-06-14 | 2019-10-29 | 平安科技（深圳）有限公司 | The OCT image denoising method and device for generating network are fought based on annular |
-
2019
- 2019-11-15 CN CN201911122032.8A patent/CN110852424B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127297A (en) * | 2016-06-02 | 2016-11-16 | 中国科学院自动化研究所 | The acceleration of degree of depth convolutional neural networks based on resolution of tensor and compression method |
| WO2019146189A1 (en) * | 2018-01-29 | 2019-08-01 | 日本電気株式会社 | Neural network rank optimization device and optimization method |
| CN109344966A (en) * | 2018-07-26 | 2019-02-15 | 广东工业大学 | A kind of method of the full Connection Neural Network of efficient tensorization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110852424A (en) | 2020-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gu et al. | Context-aware feature generation for zero-shot semantic segmentation | |
| Hartford et al. | Deep models of interactions across sets | |
| Bai et al. | Explainable deep learning for efficient and robust pattern recognition: A survey of recent developments | |
| Li et al. | MDLatLRR: A novel decomposition method for infrared and visible image fusion | |
| Xiao et al. | Attentional factorization machines: Learning the weight of feature interactions via attention networks | |
| Xu et al. | Dual-stream recurrent neural network for video captioning | |
| Giraldo et al. | Graph moving object segmentation | |
| Cheng et al. | Sparse representation and learning in visual recognition: Theory and applications | |
| Lin et al. | Deep hierarchical LSTM networks with attention for video summarization | |
| Liu et al. | Data-free knowledge transfer: A survey | |
| Wang et al. | Learning efficient binarized object detectors with information compression | |
| Ashir et al. | Facial expression recognition based on image pyramid and single-branch decision tree | |
| Wang et al. | TRC‐YOLO: A real‐time detection method for lightweight targets based on mobile devices | |
| Tian et al. | Ean: event adaptive network for enhanced action recognition | |
| Chen et al. | Unpaired deep image dehazing using contrastive disentanglement learning | |
| US20220101144A1 (en) | Training a latent-variable generative model with a noise contrastive prior | |
| KR20190125029A (en) | Methods and apparatuses for generating text to video based on time series adversarial neural network | |
| Chien et al. | Hierarchical and self-attended sequence autoencoder | |
| Wang et al. | Basketball shooting angle calculation and analysis by deeply-learned vision model | |
| Wang et al. | Optimization-based post-training quantization with bit-split and stitching | |
| Zhang et al. | Tensor graph convolutional neural network | |
| Zhao et al. | Removing adversarial noise via low-rank completion of high-sensitivity points | |
| Ward et al. | A practical tutorial on graph neural networks | |
| Garau et al. | CapsulePose: A variational CapsNet for real-time end-to-end 3D human pose estimation | |
| Su et al. | Lightweight pixel difference networks for efficient visual representation learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |