WO2022037295A1

WO2022037295A1 - Targeted attack method for deep hash retrieval and terminal device

Info

Publication number: WO2022037295A1
Application number: PCT/CN2021/104818
Authority: WO
Inventors: 夏树涛; 白家旺; 陈斌; 戴涛; 李清; 齐竹云
Original assignee: 鹏城实验室; 清华大学深圳国际研究生院
Priority date: 2020-08-20
Filing date: 2021-07-06
Publication date: 2022-02-24
Also published as: CN112115317A; CN112115317B

Abstract

Disclosed in the present invention are a targeted attack method for deep hash retrieval and a terminal device. The method comprises: providing a sample set with a tag t, inputting all samples in the sample set into a deep hash retrieval model, and generating a corresponding hash code; obtaining a representative hash code h_a by adopting a bit voting algorithm; specifying the size of a hyper-parameter α to be 0-1, and designing a loss function; calculating the gradient of x' by using a gradient descent method and updating the x' by using the gradient; projecting the generated adversarial sample x' such that the x' meets infinite constraints and image space; determining whether a preset number of updates is reached or not, and if yes, obtaining an adversarial sample x'; and inputting the adversarial sample x' into the deep hash retrieval model, and returning a sample of an expected category. When the deep hash retrieval model is designed, the attack method is adopted, the safety and robustness of the model can be improved, and the generated adversarial sample can enable the retrieval model to return to a category sample expected by an attacker.

Description

A Targeted Attack Method and Terminal Device for Deep Hash Retrieval

technical field

The invention relates to the technical field of hash retrieval, in particular to a targeted attack method and terminal device for deep hash retrieval.

Background technique

Large-scale data approximate nearest neighbor retrieval has the characteristics of high efficiency and high performance, and is used in many search engines to retrieve images or videos, such as Google and Bing. Among these approximate nearest neighbor search methods, hash-based retrieval in particular has received more attention, which can map data into a compact binary space, thereby using Hamming distance to measure similarity and improve computational efficiency.

Hash retrieval methods based on deep learning can achieve the best performance in current hash retrieval. However, many studies have shown that deep learning models are vulnerable to adversarial attacks, which affects the performance of deep learning models. According to the different attack purposes, adversarial sample generation can be divided into two types: untargeted attack and targeted attack. Untargeted attack refers to degrading the performance of the attacked model, while targeted attack refers to the attacker to achieve a specific goal (for example, in a classification task, the goal is to classify adversarial examples into a specified class). There are many approaches to these two attacks in classification tasks. However, there are few methods about adversarial attacks in retrieval tasks, and there is no targeted attack method for deep hash retrieval, which is not conducive to the research on the robustness and security of retrieval systems.

Therefore, the existing technology still needs to be improved and developed.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a targeted attack method and terminal device for deep hash retrieval in view of the deficiencies of the prior art, aiming to solve the lack of a targeted attack method for deep hash retrieval in the prior art , which is not conducive to the research on the robustness and security of the retrieval system.

In order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is as follows:

A targeted attack method for deep hash retrieval, comprising the steps of:

Provide a sample set with label t, input all samples in the sample set into the deep hash retrieval model, and generate corresponding hash codes

Wherein, the label t specifies the category expected to be returned by the attacker, and the label t is different from the category of the query image x;

The representative hash code _ha is obtained by adopting the bit voting algorithm;

Specify the size of the hyperparameter α from 0 to 1, and design the loss function as:

Among them, tanh is the hyperbolic tangent function, and x' is the adversarial sample;

Use the gradient descent method to calculate the gradient of x';

update x' with the computed gradient;

Project the generated adversarial sample x' so that x' satisfies the infinite constraints and the image space;

Determine whether the preset number of updates has been reached, and if so, get the adversarial sample x';

Input the adversarial sample x' into the deep hash retrieval model, and return samples of the desired class.

The targeted attack method for deep hash retrieval, wherein the deep hash retrieval model is F( ), the length of the hash code is K, and the generation formula of the hash code of the sample x _i is: h= F(x)=sign(f _θ (x)), where f _θ ( ) represents the deep neural network model, sign( ) is the sign function,

represents N datasets divided into C categories, and y _i ∈ {0, 1} ^C represents the label vector.

In the targeted attack method for deep hash retrieval, the sample _xi is a picture or a video.

The targeted attack method for deep hash retrieval, wherein the step of using _a bit voting algorithm to obtain the representative hash code ha includes:

Hash code for all samples in the sample set

According to the bit voting method, the representative hash code _ha is obtained.

The targeted attack method for deep hash retrieval, wherein the hash code of all samples in the sample set is

The steps of obtaining the representative hash code _ha include:

For j=1,2,...K, count the number of +1 and -1 at each position, expressed as

and

in,

Represents an indicator function;

According to the formula

Determine the jth position

The value of , thus returning the representative hash code _ha .

A computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors, so as to realize the depth-targeting described in the present invention. Steps in a targeted attack method for hash retrieval.

A terminal device, comprising: a processor, a memory and a communication bus; a computer-readable program executable by the processor is stored on the memory;

The communication bus implements connection communication between the processor and the memory;

When the processor executes the computer-readable program, the steps in the targeted attack method for deep hash retrieval of the present invention are implemented.

Beneficial effects: Compared with the prior art, the present invention provides a targeted attack method, storage medium and terminal device for deep hash retrieval. First, the targeted attack in retrieval is defined as a point-to-set optimization problem, that is, Minimize the average distance between the hash code of the adversarial sample and the set of hash codes of the desired category; then a bit-voting algorithm is designed to obtain the optimal representative hash code of the set of hash codes of the desired category; in order to ensure the invisibility of the adversarial samples It is further proposed to optimize the adversarial noise under infinite constraints, so that the distance between the hash code of the adversarial sample and the representative hash code is as small as possible. The method of the invention not only ensures the indistinguishability between the confrontation sample and the original sample, but also obtains a good target attack effect; the invention adopts this attack method when designing the deep hash retrieval model, which is beneficial to improve the security and robustness of the model. The adversarial examples generated can make the retrieval model return the class samples expected by the attacker.

Description of drawings

FIG. 1 is a flowchart of a preferred embodiment of a targeted attack method for deep hash retrieval provided by the present invention.

FIG. 2 is a schematic diagram of a targeted attack method for deep hash retrieval provided by the present invention.

FIG. 3 is a schematic structural diagram of a terminal device provided by the present invention.

detailed description

The present invention provides a targeted attack method, storage medium and terminal device for deep hash retrieval. In order to make the purpose, technical solution and effect of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. . It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

It will be understood by those skilled in the art that the singular forms "a", "an", "said" and "the" as used herein can also include the plural forms unless expressly stated otherwise. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of stated features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components and/or groups thereof. It will be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Furthermore, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combination of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms, such as those defined in a general dictionary, should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as herein, should not be interpreted in idealistic or overly formal meaning to explain.

In the following, the content of the invention will be further illustrated by describing the embodiments with reference to the accompanying drawings.

Existing targeted attacks are mainly aimed at classification tasks. In classification tasks, adversarial attacks have clear optimization goals due to the presence of class labels in images. In hash retrieval, samples are mapped into a binary space, making the target of targeted attacks unclear. Taking hash-based image retrieval as an example, for such large-scale data retrieval, approximate nearest neighbor search can achieve good results, it balances efficiency and accuracy, so that retrieval achieves good results. The main idea is to represent each picture with a relatively short 01 code, such as a code with a length of 64,128, which still approximately maintains the physical neighbor relationship in the picture space. When the user uploads a picture, use the hash function to convert it into 01 code, and then calculate the distance between this code and the codes of all pictures in the database (using Hamming distance calculation at this time), that is, the picture's Binary code, XOR operation with all binary codes in the database, the number of 1 is the distance, sort all distances, select the first 100 closest pictures as similar pictures, and then find the original picture by index and display it . Specifically, when doing hash-based image retrieval, the cifar-10 data set can be used. First, the gist feature is extracted from the data set, and each image is represented by a vector. For example, if 512 features are extracted, then each image will use A 512-dimensional vector representation, 10,000 pictures are finally formed: a 10000*512 matrix. Divide the data into a training set and a test set, and the training set is used to train the hash function. The test set is used to test the precision and recall. The hash function is trained from the training set. The training data is converted into a hash function code through a hash function, and the test data is converted into a hash code. Calculate the distance from the test data to the training data, sort, select the top 100 pictures with the smallest distance, and the 100 pictures found are the pictures of the approximate neighbors.

The hash retrieval method based on deep learning can achieve the best performance in the current hash retrieval, however, research shows that the deep learning model is vulnerable to adversarial attacks, which affects the performance of the deep learning model. According to the different attack purposes, adversarial sample generation can be divided into two types: untargeted attack and targeted attack. Untargeted attack refers to degrading the performance of the attacked model, while targeted attack refers to the attacker to achieve a specific goal (for example, in a classification task, the goal is to classify adversarial examples into a specified class). There are many approaches to these two attacks in classification tasks. However, due to the different nature of classification tasks and retrieval tasks, the targeted attack methods in classification cannot be directly transferred to retrieval. There are few methods for adversarial attacks in retrieval tasks in the prior art, and there is no method for deep hash retrieval. There are targeted attack methods, which are not conducive to the robustness and security of research retrieval systems. Therefore, it is necessary to propose an effective targeted attack technical scheme adapted to the characteristics of retrieval tasks.

Embodiments of the present invention provide a targeted attack method for deep hash retrieval, which includes the steps:

Use the gradient descent method to calculate the gradient of x';

update x' with the computed gradient;

Specifically, the deep hash retrieval model is F( ), the length of the hash code is K, and the generation formula of the hash code of the sample x _i is: h=F(x)=sign(f _θ (x )), where f _θ ( ) represents the deep neural network model, sign( ) is the sign function,

represents N datasets divided into C categories, and y _i ∈ {0, 1} ^C represents the label vector. When the deep hash retrieval model is not attacked, the retrieval process for query sample x is as follows: first, the model outputs the hash code F(x) of x, and then calculates the difference between the query hash code and all sample hash codes in the database. Hamming distance d _H (F(x), F(x _i )), and finally the retrieval system will sort the samples in the database according to the calculated distance and return the result.

The targeted attack method for deep hash retrieval provided by this embodiment first defines the targeted attack in deep hash retrieval as a point-to-set optimization problem, that is, minimizing the hash code of the adversarial sample and the expected class hash Then, a bit-voting algorithm is designed to obtain the optimal representative hash code method of the desired category hash code set; in order to ensure the invisibility of adversarial samples, it is further proposed to optimize the adversarial noise under infinite constraints, so that The distance between the hash code of the adversarial example and the representative hash code is as small as possible. The method of this embodiment not only ensures the indistinguishability of the adversarial sample from the original sample, but also obtains a good effect of targeted attack; this embodiment adopts this attack method when designing the deep hash retrieval model, which is beneficial to improve the security of the model and robustness, and the resulting adversarial examples enable the retrieval model to return samples of the class expected by the attacker.

In this embodiment, as shown in FIG. 1, for the query image x, the attacker specifies the desired category t to be returned, and t needs to be different from the real category of x; as an example, if the category of x is dog, the attacker specifies the desired category to be returned. The category t of can be cat, pig, fish, chicken, etc., but is not limited thereto. An attacker can provide a set of samples with label t

Generate hash codes for all samples in sample set X ^(t) using model F( )

Hash code for all samples in the sample set

According to the bit voting method, the representative hash code _ha is obtained; then the size of the hyperparameter α is specified as 0 to 1, and the loss function is designed as:

Among them, tanh is the hyperbolic tangent function, and x' is the adversarial sample; then use the gradient descent method to calculate the gradient of x', and use the calculated gradient to update x'; project the generated adversarial sample x' so that x' satisfies infinity Constraint and image space; judge whether the preset number of updates is reached, if so, get the adversarial sample x'; if not, continue to return to step S06 to continue updating x'; finally input the adversarial sample x' into the depth In the hope retrieval model, the samples of the desired category are returned.

As shown in Figure 2, the adversarial samples generated by this algorithm are first input into the hash model, that is, the adversarial query "dog" picture is input into the following feature extractor and fully connected layer to obtain the hash code of the adversarial samples. The hash code retrieves the neighbor samples in the database, and the obtained neighbor samples belong to the attack category preset by the attacker in the targeted attack, that is, the "cat" in the figure below.

In this embodiment, the size of the hyperparameter α is set from 0 to 1 to prevent the gradient disappearance problem during backpropagation and speed up the convergence speed of the adversarial sample generation algorithm; by designing the loss function

To denote that the infinite norm of the original query image and the generated adversarial sample is smaller than a given threshold ∈, that is, to make the hash code of the adversarial sample and the representative hash code _ha as close as possible to make the two samples indistinguishable.

In this embodiment, calculating the gradient of x' by using the gradient descent method refers to using the back-propagation algorithm, according to the loss function provided above, starting from the output layer, calculating the gradient layer by layer, and obtaining the gradient G of the input x' until. Then use the formula x'=x'-G to update x', where G is the gradient G obtained in the previous step.

In this embodiment, the step of projecting the generated adversarial sample x' so that x' satisfies the infinite constraint and the image space specifically includes: projecting the adversarial sample x' according to the formula x'=clamp(x'), wherein, clamp() is a projection function, set the value of x' greater than x+∈ as x+∈, and set the value of x' less than x+∈ as x', and ensure that x' satisfies the image space, that is, in the space represented by 0-255.

In this embodiment, the preset number of updates is a parameter set by the attacker, which can be set to 2000; reaching a certain preset number of times is to satisfy the success of the attack and at the same time, within an acceptable calculation time, the preset number of updates does not reach the preset value. The number of updates may cause the generated adversarial examples to attack poorly.

In some embodiments, the sample x' is a picture or a video.

In some embodiments, the hash code for all samples in the sample set

The steps of obtaining the representative hash code ha include: for j ₌ 1, 2,...K, calculating the number of +1 and -1 in each position, expressed as

and

in,

Represents an indicator function; according to the formula

Determine the jth position

The value of , thus returning the representative hash code _ha . This embodiment adopts the algorithm of bit voting to calculate the representative hash code, which provides an optimized target for targeted confrontation attacks, and can make the attack effect efficient and stable.

Based on the above-mentioned targeted attack method for deep hash retrieval, this embodiment provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be One or more processors execute to implement the steps in the targeted attack method for deep hash retrieval as described in the above embodiments.

Based on the above-mentioned targeted attack method for deep hash retrieval, the present invention also provides a terminal device, as shown in FIG. 3 , which includes at least one processor 20 ; a display screen 21 ; and a memory 22 , may also include a communications interface (Communications Interface) 23 and a bus 24. The processor 20 , the display screen 21 , the memory 22 and the communication interface 23 can communicate with each other through the bus 24 . The display screen 21 is set to display a user guide interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 may invoke logic instructions in the memory 22 to perform the methods in the above-described embodiments.

In addition, the above-mentioned logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

As a computer-readable storage medium, the memory 22 may be configured to store software programs and computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, ie, implements the methods in the above embodiments.

The memory 22 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. Additionally, memory 22 may include high-speed random access memory, and may also include non-volatile memory. For example, U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes, or temporary state storage medium.

In addition, the specific process of loading and executing the above-mentioned storage medium and the multiple instruction processor in the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

To sum up, the present invention provides a targeted attack method, storage medium and terminal device for deep hash retrieval. First, the targeted attack in retrieval is defined as a point-to-set optimization problem, that is, minimizing the impact of adversarial samples. The average distance between the hash code and the set of hash codes of the desired category; then a method of bit voting to obtain the optimal representative hash code of the set of hash codes of the desired category is designed; in order to ensure the invisibility of the adversarial samples, it is further proposed that in the infinite The adversarial noise is optimized under constraints so that the distance between the hash code of the adversarial sample and the representative hash code is as small as possible. The method of the invention not only ensures the indistinguishability between the confrontation sample and the original sample, but also obtains a good target attack effect; the invention adopts this attack method when designing the deep hash retrieval model, which is beneficial to improve the security and robustness of the model. The adversarial examples generated can make the retrieval model return the class samples expected by the attacker. The present invention provides support for improving the robustness and security of the retrieval system by proposing a targeted adversarial attack method for deep hash retrieval, verifying the robustness of the retrieval model under this attack. The invention destroys the model retrieval result by adding invisible anti-noise to the input image, and returns the sample of the desired category of the attacker.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A targeted attack method for deep hash retrieval, characterized in that it comprises the steps of:

Provide a sample set with label t, input all samples in the sample set into the deep hash retrieval model, and generate corresponding hash codes
Wherein, the label t specifies the category expected to be returned by the attacker, and the label t is different from the category of the query image x;

The representative hash code ha is obtained by adopting the bit voting algorithm;

Specify the size of the hyperparameter α from 0 to 1, and design the loss function as:

Among them, tanh is the hyperbolic tangent function, and x' is the adversarial sample;

Use the gradient descent method to calculate the gradient of x';

update x' with the computed gradient;

Project the generated adversarial sample x' so that x' satisfies the infinite constraints and the image space;

Determine whether the preset number of updates has been reached, and if so, get the adversarial sample x';

Input the adversarial sample x' into the deep hash retrieval model, and return samples of the desired class.
The targeted attack method for deep hash retrieval according to claim 1, wherein the deep hash retrieval model is F( ), and its hash code length is K, and the length of the hash code of sample x i is The generation formula is: h=F(x)=sign(f θ (x)), where f θ ( ) represents the deep neural network model, sign ( ) is the sign function,
represents N datasets divided into C categories, and y i ∈ {θ, 1} C represents the label vector.
The targeted attack method for deep hash retrieval according to claim 2, wherein the sample xi is a picture or a video.
The targeted attack method for deep hash retrieval according to claim 2, wherein the step of using a bit-voting algorithm to obtain the representative hash code ha comprises:

Hash code for all samples in the sample set
According to the bit voting method, the representative hash code ha is obtained.
The targeted attack method for deep hash retrieval according to claim 4, wherein the hash code of all samples in the sample set is
The steps of obtaining the representative hash code ha include:

For j=1,2,...K, count the number of +1 and -1 at each position, expressed as
and
in,
in,
Represents an indicator function;

According to the formula
Determine the jth position
The value of , thus returning the representative hash code ha .
A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors, so as to realize the invention as claimed in claim 1 -5 any one of the steps in the targeted attack method for deep hash retrieval.
A terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

The communication bus implements connection communication between the processor and the memory;

When the processor executes the computer-readable program, the steps in the targeted attack method for deep hash retrieval according to any one of claims 1-5 are implemented.