WO2026020964A1 - 样本净化方法、装置、设备及介质 - Google Patents

样本净化方法、装置、设备及介质

Info

Publication number
WO2026020964A1
WO2026020964A1 PCT/CN2025/096787 CN2025096787W WO2026020964A1 WO 2026020964 A1 WO2026020964 A1 WO 2026020964A1 CN 2025096787 W CN2025096787 W CN 2025096787W WO 2026020964 A1 WO2026020964 A1 WO 2026020964A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
adversarial
noise
text
visual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/096787
Other languages
English (en)
French (fr)
Inventor
张宇东
谢若冰
孙兴武
康战辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology (Shenzhen) Co Ltd
Original Assignee
Tencent Technology (Shenzhen) Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology (Shenzhen) Co Ltd filed Critical Tencent Technology (Shenzhen) Co Ltd
Publication of WO2026020964A1 publication Critical patent/WO2026020964A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • This application relates to the field of artificial intelligence technology, and in particular to a sample purification method, apparatus, device, and medium.
  • LVLMs Large Vision-Language Models
  • LVLMs are broadly defined as multimodal models that can learn from images and text.
  • LVLMs are susceptible to adversarial examples when processing tasks.
  • Adversarial examples are created by making minor modifications or perturbations to clean samples, causing the LVLM to produce incorrect predictions.
  • adversarial examples are cleaned up by performing certain preprocessing operations on the images input to the visual language model. For example, smoothing the image, i.e., applying a smoothing filter to reduce image noise, thereby reducing the impact of adversarial attacks; or scaling the image, i.e., changing the image features by altering the image scale (e.g., scaling), thereby reducing the effectiveness of adversarial attacks.
  • smoothing the image i.e., applying a smoothing filter to reduce image noise, thereby reducing the impact of adversarial attacks
  • scaling the image i.e., changing the image features by altering the image scale (e.g., scaling), thereby reducing the effectiveness of adversarial attacks.
  • a sample purification method comprising:
  • the adversarial sample is obtained by performing an adversarial attack on a clean sample, and the adversarial sample includes adversarial images and text;
  • a second noise is added to the adversarial image based on the first correlation and the second correlation to clean the adversarial image and obtain a cleaned image.
  • the acquisition module is used for:
  • the determining module is used for:
  • the adding module is used to add second noise to the adversarial image based on the first correlation and the second correlation to clean the adversarial image and obtain a cleaned image.
  • a computer device comprising: a processor and a memory, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor to implement the sample purification method as described above.
  • a computer storage medium wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the sample purification method as described above.
  • a computer program product comprising a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the sample purification method as described above.
  • a cleaned image By performing a cleansing operation on the adversarial image in the adversarial examples, a cleaned image can be obtained, thus yielding a cleaned example.
  • a noisy image is obtained by randomly adding noise to the adversarial image. Since the noisy cross-modal attention distribution of the noisy image (i.e., the first correlation between at least one noisy image feature and at least one text feature of the text) is very close to the cross-modal attention distribution of the clean image (i.e., the correlation between at least one clean image feature and at least one text feature of the text), the noisy cross-modal attention distribution of the noisy image can be used as an optimization target.
  • Random noise is added to the adversarial image, thereby making the adversarial cross-modal attention distribution of the adversarial image (i.e., the second correlation between at least one adversarial image feature and at least one text feature of the text) close to the noisy cross-modal attention distribution of the noisy image, thus completing the cleansing of the adversarial image and obtaining a cleaned image.
  • This application guided by the noisy cross-modal attention distribution, achieves adversarial image cleansing by adding noise to the adversarial image.
  • the visual language model can obtain correct prediction results. This cleansing method enhances the robustness of the visual language model.
  • the method of adding random noise to adversarial images by using the noisy cross-modal attention distribution as the estimation direction not only optimizes the adversarial images but also optimizes the cross-modal attention distribution itself. This optimization enables the attention mechanism to focus more accurately on the key information in the image, thereby improving the accuracy of the prediction results output by the visual language model.
  • Figure 1 is a schematic diagram of the architecture of a computer system provided in an exemplary embodiment of this application;
  • Figure 2 is a schematic diagram of a sample purification method provided in an exemplary embodiment of this application.
  • FIG. 3 is a flowchart of a sample purification method provided in an exemplary embodiment of this application.
  • Figure 4A is a flowchart of a sample purification method provided in an exemplary embodiment of this application.
  • Figure 4B shows a flowchart of a sample purification method provided in an exemplary embodiment of this application
  • FIG. 5 is a flowchart of a sample purification method provided in an exemplary embodiment of this application.
  • FIG. 6 is a flowchart of a sample purification method provided in an exemplary embodiment of this application.
  • Figure 7 is a structural block diagram of a sample purification device provided in an exemplary embodiment of this application.
  • Figure 8 is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.
  • first, second, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another.
  • a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter.
  • the word "if” as used herein may be interpreted as "when,” “when,” or "in response to determination.”
  • the data involved in this application is all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the settings and other information involved in this application were obtained with full authorization.
  • VLPs Visual language models
  • VLPs are multimodal models that combine visual (image) and linguistic (text) inputs. They can receive image and text inputs and generate task outputs. VLPs can understand and process the relationships between images and text, and are used to handle various tasks such as image captioning, visual question answering (VQA), and image-text matching.
  • Clean samples refer to original samples that have not been modified or perturbed in any way. Clean samples are standard samples under ideal conditions and serve as a benchmark in the evaluation of visual language models. Clean samples include clean images and clean text.
  • Adversarial examples are samples that make minor modifications or perturbations to clean samples, causing the visual language model to make incorrect predictions.
  • the perturbations in adversarial examples are not easily noticeable, but they lead to incorrect predictions or outputs from the visual language model.
  • Adversarial examples include adversarial images and text.
  • the cleanup operations for adversarial examples in this application can be viewed as cleanup operations for adversarial images.
  • noisy adversarial samples These are adversarial samples to which random noise has been added.
  • the range of random noise addition is limited or constrained by the infinity norm.
  • the goal of randomly noisy adversarial samples is to maintain the adversarial nature while making the cross-modal attention distribution of the noisy samples approximate that of the clean samples.
  • noisy samples include both noisy images and text.
  • Cleaned samples refer to samples that have undergone minor modifications or perturbations to adversarial samples. Cleaned samples can improve the probability of a visual language model generating correct predictions. The purpose of cleaned samples is to reduce adversarial perturbations that lead to incorrect predictions by the visual language model, thereby increasing the probability of the visual language model generating correct predictions. Cleaned samples include cleaned images and text. In the embodiments of this application, the cleaned sample obtained by the cleansing operation on the adversarial sample can be understood as the cleaned image obtained by the cleansing operation on the adversarial image.
  • Visual tokens are feature representations extracted from an image and encoded by a visual encoder in a visual language model.
  • the visual encoder divides the input image into several regions, each of which can be encoded as an image feature (or visual feature, region image feature). Several regions can be encoded as several image features.
  • the language alignment module in the visual language model aligns these image features to obtain multiple visual tokens.
  • the language alignment module processes these into 32 visual tokens, each of which can be a fused representation of the 256 image features.
  • different visual tokens may emphasize different aspects of the fused image features.
  • the first visual token may focus on the background region of the image, meaning it primarily represents the image features of the background region; the second visual token may focus on the object region of the image, meaning it primarily represents the image features of the object region.
  • Language tokens are feature representations extracted from text, obtained by encoding them through a language model segmenter in a visual language model.
  • the language model segmenter divides the input text into several words (or sub-words), and then converts each word into a corresponding feature vector.
  • Text can typically be divided into several sub-words, and the feature vector of each sub-word can be represented as a language token.
  • a language token usually corresponds to one sub-word in the text.
  • the feature vectors of multiple sub-words can be represented as a language token
  • a language token usually corresponds to multiple sub-words in the text.
  • a language token is the textual features of all sub-words in the text.
  • Cross-modal attention distribution refers to an attention mechanism employed by visual language models when processing multimodal data. Specifically, it describes how one modality (e.g., text) allocates attention or focus based on the content of another modality (e.g., an image) during multimodal data processing.
  • cross-modal attention distribution refers to how a visual language model focuses on relevant parts of an image in a sample when generating prediction results.
  • visual language models can focus on image regions related to the descriptive text or question based on cross-modal attention distribution.
  • the task of a visual language model is to answer a question in an image.
  • the image input to the visual language model is a picture of a red car driving on a highway, and the text input is the question "What is the color of the car?”.
  • the visual language model When the visual language model generates the first word "red” in the answer, it calculates the attention weights for all visual tokens (predefined image features) in the image. These attention weights represent the importance of each visual token to the currently generated word; that is, the visual language model evaluates the extent to which each visual token influences the generation of the first word "red".
  • the visual language model might focus on visual tokens representing the car portion of the image.
  • the cleansing operation is mainly performed on the adversarial image in the adversarial examples to obtain a cleaned sample with a cleaned image.
  • the cross-modal attention distribution corresponding to the clean sample can be understood as the cross-modal attention distribution corresponding to the clean image
  • the cross-modal attention distribution corresponding to the adversarial example can be understood as the cross-modal attention distribution corresponding to the adversarial image
  • the cross-modal attention distribution corresponding to the noisy sample can be understood as the cross-modal attention distribution corresponding to the noisy image
  • the cross-modal attention distribution corresponding to the cleaned sample can be understood as the cross-modal attention distribution corresponding to the cleaned image.
  • Figure 1 shows a schematic diagram of the architecture of a computer system provided in one embodiment of this application.
  • the computer system may include: a terminal 100 and a server 200.
  • Terminal 100 can be an electronic device such as a mobile phone, tablet computer, vehicle terminal (in-vehicle system), wearable device, personal computer (PC), or vehicle terminal.
  • a client application for the target application can be installed and run on terminal 100.
  • This target application can be an application that supports sample purification, and this application is not limited in this respect.
  • this application is not limited in the form of the target application, including but not limited to applications (Apps), mini-programs, clients, etc., installed on terminal 100, and it can also be in web page form.
  • Server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence handheld image recognition platforms.
  • Server 200 can be a backend server for the aforementioned target application, used to provide backend services to the clients of the target application.
  • Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing.
  • cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access.
  • Cloud computing technology will become a crucial support.
  • Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites.
  • every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.
  • the server described above can also be implemented as a node in a blockchain system.
  • Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms.
  • a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block.
  • a blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.
  • Terminal 100 and server 200 can communicate via a network, such as a wired or wireless network.
  • the sample purification method provided in this application can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Taking the computer system shown in Figure 1 as an example, the sample purification method can be executed by the terminal 100, the server 200, or by the terminal 100 and the server 200 interacting and cooperating to execute the sample purification method. This application does not limit this.
  • Figure 2 illustrates a schematic diagram of a sample purification method provided in an exemplary embodiment of this application.
  • the method is executed by a computer device, which may be the terminal 100 and/or server 200 shown in Figure 1.
  • the computer device is described as server 200.
  • the sample purification method is executed by server 200.
  • adversarial example cleanup is achieved by performing preprocessing operations on the images input to the visual language model. For example, smoothing the image (applying a smoothing filter to reduce image noise) can mitigate the impact of adversarial attacks; similarly, scaling the image (e.g., resizing) can alter image features and reduce the effectiveness of adversarial attacks.
  • these adversarial example cleanup techniques primarily rely on traditional image processing methods, resulting in less than ideal cleanup effects. In practical applications, adversarial examples can still impact the task output of the visual language model.
  • this application proposes a sample cleansing method.
  • adversarial examples primarily adversarial images
  • the cross-modal attention distribution is used as the target direction for the cleansing process, guiding the adversarial examples to complete the cleansing operation.
  • random noise can be added to the adversarial examples to guide the adversarial cross-modal attention distribution of the adversarial examples to approximate the clean cross-modal attention distribution of the clean examples, thereby enabling the visual language model to produce accurate predictions.
  • visual language models can answer natural language questions about images.
  • clean sample 10 includes a clean image and text
  • adversarial sample 20 includes an adversarial image and text
  • noisy sample 30 includes a noisy image and text
  • clean sample 40 includes a clean image and text.
  • the text in clean sample 10, adversarial sample 20, noisy sample 30, and clean sample 40 is a description of a natural language question: "What is this man doing?".
  • visual language model 50 calculates the clean cross-modal attention distribution of the clean image in clean sample 10.
  • visual language model 50 calculates the attention weights of all visual tokens in the clean image.
  • Visual language model 50 divides the clean image into several regions, and the image features of each region (or the image features corresponding to multiple regions) can be represented as a visual token.
  • the attention weights represent the importance of each visual token to the answer "hit the ball", that is, visual language model evaluates the extent to which each visual token influences the generation of "hit the ball”.
  • cross-modal attention distribution diagram 70 is a visualization of the cross-modal attention distribution, showing the attention paid by the visual language model to different visual tokens in the image when processing the task.
  • the cross-modal attention distributions of clean sample 10 and adversarial sample 20 are significantly different; this can also be understood as the cross-modal attention distributions of clean image and adversarial image being significantly different.
  • Adversarial sample 20 can be cleaned by guiding its adversarial cross-modal attention distribution to approximate the clean cross-modal attention distribution of clean sample 10.
  • adversarial cleanup scenarios it is often impossible to directly obtain clean sample 10, and consequently, its clean cross-modal attention distribution.
  • a noisy sample 30 is obtained.
  • the noisy cross-modal attention distribution of noisy sample 30 is closer to the clean cross-modal attention distribution corresponding to clean sample 10 (indicated by the attention weight distribution in the cross-modal attention distribution map corresponding to noisy sample 30 being close to the attention weight distribution in the cross-modal attention distribution map corresponding to clean sample 10). Therefore, using the noisy cross-modal attention distribution of noisy sample 30 as a reference estimation direction to guide the optimization of the adversarial cross-modal attention distribution of adversarial sample 20 can be considered as a cleanup operation for adversarial sample 20, making it closer to clean sample 10.
  • the cleanup of adversarial sample 20 is achieved.
  • the noisy cross-modal attention distribution corresponding to noisy sample 30 is used as the target to guide the adversarial cross-modal attention distribution corresponding to adversarial sample 20 to approach the noisy cross-modal attention distribution corresponding to noisy sample 30.
  • a second noise is added to the adversarial image to generate a cleaned image, i.e., a cleaned operation is performed on adversarial sample 20 to generate cleaned sample 40.
  • the attention difference value is the difference between the noisy cross-modal attention distribution corresponding to noisy sample 30 and the adversarial cross-modal attention distribution corresponding to adversarial sample 20.
  • the visual language model 50 can generate the correct answer "hit the ball" based on cleaned sample 40.
  • the cleaned sample 40 is obtained based on the following steps: First, the norm of the attention difference value is calculated, which measures the distance between the adversarial cross-modal attention distribution corresponding to the adversarial sample 20 and the noisy cross-modal attention distribution corresponding to the noisy sample 30. Second, the gradient of the attention difference value with respect to the pixels in the adversarial image is calculated, which indicates how to add second noise to minimize the attention difference value under the current adversarial sample 20. Then, the gradient direction is determined based on the sign function, which indicates the direction in which the second noise is added to the adversarial image in the adversarial sample 20, i.e., the pixels in the adversarial image are adjusted towards the gradient direction to reduce the attention difference value.
  • the gradient is used as the input to the sign function.
  • the output of the sign function is 1, meaning that the adversarial sample 20 increases the pixel values in the adversarial image according to the gradient direction.
  • the output of the sign function is -1, meaning that the adversarial sample 20 decreases the pixel values in the adversarial image according to the gradient direction.
  • the output of the sign function is 0, meaning that the adversarial sample keeps the pixel values in the adversarial image unchanged according to the gradient direction (basically, pixel values will change, and there is almost no case where they remain unchanged).
  • increasing or decreasing the pixel values in the adversarial image according to the gradient direction yields a cleaned image after adding the second noise.
  • the magnitude of the second noise added to the adversarial image is controlled.
  • the gradient includes not only gradient direction information but also intensity information.
  • the intensity of the gradient is positively correlated with the magnitude of the added second noise.
  • the adversarial images in the adversarial sample 20 are divided into pixel-level segments, and second noise of different perturbation levels (different sizes) is added to different pixel positions in the adversarial image based on the gradient direction.
  • pixels with larger gradients in the adversarial image should be subject to greater perturbation, i.e., relatively larger noise is added; pixels with smaller gradients in the adversarial image should be subject to smaller perturbation, i.e., relatively smaller noise is added.
  • the maximum and minimum gradient values are obtained, and a gradient standard value is obtained based on the maximum and minimum gradient values.
  • the gradient standard value is the gradient value after normalization, with the gradient normalized to [0, 1].
  • the average gradient standard value can be used to represent the gradient standard value.
  • the perturbation amplitude of the second noise is determined based on the gradient standard value and its average value.
  • the perturbation amplitude includes a perturbation factor used to determine the magnitude of the second noise, and the magnitude of the second noise added to each pixel in the adversarial image is determined based on the perturbation factor.
  • a purified image is obtained through the purification operation on the adversarial image described above, that is, a purified sample 40 is obtained through the purification operation on the adversarial sample 20.
  • the purified sample 40 is input into the visual language model 50, and the visual language model 50 can obtain the correct output result "hit the ball" based on the purified sample 40.
  • the noisy cross-modal attention distribution is also referred to as the first correlation between at least one noisy image feature of the noisy image and at least one text feature of the text;
  • the adversarial cross-modal attention distribution is also referred to as the second correlation between at least one adversarial image feature of the adversarial image and at least one text feature of the text.
  • the attention difference value is also referred to as the correlation difference value.
  • FIG 3 is a flowchart of a sample purification method provided in an exemplary embodiment of this application.
  • the method can be executed by a computer device, which may be the terminal 100 or server 200 in Figure 1.
  • the method includes the following steps.
  • Step 210 Obtain adversarial examples
  • Adversarial examples are samples generated after performing adversarial attacks on clean samples.
  • clean samples include clean images and text.
  • Adversarial examples include adversarial images and text.
  • a clean sample refers to an original sample that has not been modified or perturbed in any way.
  • An adversarial sample is a sample generated after performing an adversarial attack on a clean sample. The adversarial attack involves making minor modifications or perturbations to the clean sample. These modifications or perturbations are not easily noticeable, but they can cause the visual language model to make incorrect predictions and generate incorrect prediction results (outputs).
  • adversarial examples can be used to evaluate the performance of visual language models in the face of minor perturbations. Introducing adversarial examples allows for testing the robustness of visual language models in real-world applications, revealing their vulnerability to attacks or misdirection. Adversarial examples can be used as attack tools to attempt to gain a competitive advantage or conduct malicious activities by interfering with the visual language model. Therefore, the sanitization of adversarial examples is particularly important.
  • a clean image shows a cat
  • the visual language model correctly classifies it as "cat”.
  • an adversarial attack such as an adversarial attack based on the FGSM (Fast Gradient Sign Method)
  • an adversarial image can be generated.
  • the generated adversarial image still looks like a cat, but the visual language model may incorrectly classify it as "dog" or another category.
  • a cleansing operation is performed on the adversarial images in the adversarial examples to obtain cleaned images.
  • Step 220 Obtain the noisy image obtained after adding the first noise to the adversarial image
  • the noisy image is the image after applying a first perturbation to the adversarial image.
  • the first noise refers to random noise added to the adversarial image.
  • the first noise is generated randomly or according to certain rules, with the purpose of perturbing the pixel values of the adversarial image.
  • the first noise can be Gaussian noise, uniform noise, or other types of random noise.
  • a visual language model is a multimodal model that combines visual (image) and language (text) inputs, capable of receiving image and text inputs and generating task outputs.
  • the visual language model is able to understand and process the relationships between images and text, and can be used to handle various tasks.
  • the visual language model includes at least one of the following: a deep neural network model (DNN), a convolutional neural network model (CNN), a Transformer model, a contrastive language-image pre-training (CLIP) model, an extreme learning machine model (ELM), or other neural network models, but is not limited thereto; the embodiments of this application do not specifically limit this.
  • the first noise added to the adversarial image is uniform noise.
  • the noisy image can be represented as:
  • x'i represent the adversarial image
  • ⁇ represent the first noise, which is determined based on the first noise value obtained by pseudo-random sampling within the first distribution interval [ - ⁇ , ⁇ ], where [ - ⁇ , ⁇ ] is a uniformly distributed interval.
  • R( x'i , ⁇ ) represents adding random noise with an infinite norm constrained to ⁇ to the adversarial image x'i .
  • the first noise is added via a computer program.
  • a noise-adding program is installed on the computer device.
  • a uniformly distributed first noise can be generated and added to the adversarial image.
  • the noise-adding program uses the OpenCV library to read the adversarial image to which the first noise is to be added, and uses a random number generator to generate uniformly distributed random numbers, which will serve as the noise values for the first noise.
  • the value of the generated random noise (first noise) is adjusted according to the required noise intensity. The noise intensity can be controlled by adjusting the range of the random number generator. Then, the noise values of the first noise generated by the random number generator are added to the pixels of the adversarial image.
  • the first noise is added via a noise model, such as using Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), or other deep learning models to generate the first noise, which is then added to the adversarial image.
  • GANs Generative Adversarial Networks
  • VAEs Variational Autoencoders
  • the addition of the first noise to the adversarial image using a GAN to generate a noisy image is illustrated.
  • the GAN receives random noise (the first noise) as input.
  • the random noise can be sampled from a uniform distribution.
  • the GAN outputs a uniformly distributed noise image based on the random noise.
  • the GAN then performs noise addition processing, adding the generated noise image to the adversarial image, i.e., superimposing the pixel values of the noise image onto the adversarial image to obtain the noisy image.
  • This is merely an illustrative example, and this application does not limit the method of adding noise.
  • Step 230 Determine a first correlation between at least one noisy image feature of the noisy image and at least one text feature of the text;
  • This noisy cross-modal attention distribution describes the attention distribution of noisy images and text.
  • the cross-modal attention distributions of clean samples and adversarial samples differ significantly.
  • This can also be understood as the clean cross-modal attention distribution of a clean image differing significantly from the adversarial cross-modal attention distribution of an adversarial image.
  • Adversarial image cleansing can be achieved by guiding the adversarial cross-modal attention distribution of the adversarial image to approximate the clean cross-modal attention distribution of the clean image.
  • adversarial cleansing scenarios it is often impossible to directly obtain a clean image, and consequently, it is impossible to directly obtain the clean cross-modal attention distribution of a clean image.
  • a noisy image is obtained by adding first noise to the adversarial image.
  • the noisy cross-modal attention distribution of the noisy image is closer to the clean cross-modal attention distribution of the clean image. Therefore, using the noisy cross-modal attention distribution of the noisy image as a reference estimation direction to guide the optimization of the adversarial cross-modal attention distribution of the adversarial image can be considered as a cleansing operation for the adversarial image, making it closer to the clean image.
  • adversarial image cleansing is achieved by guiding the adversarial cross-modal attention distribution of the adversarial image to approach the noisy cross-modal attention distribution of the noisy image.
  • the noisy cross-modal attention distribution corresponding to the noisy image is calculated.
  • Cross-modal attention distribution refers to an attention distribution in which a visual language model processes multimodal data.
  • noisy cross-modal attention distribution refers to how a visual language model focuses on the magnitude (or attention weight) of the noisy image in the noisy sample when generating prediction results based on the noisy sample.
  • the visual language model calculates the attention weights of all visual tokens in the noisy image, where each visual token corresponds to a noisy image feature.
  • the visual language model divides the noisy image into several regions, each region corresponding to a region image feature. This can represent fusing the region image features of multiple regions into at least one noisy image feature, or fusing them into at least one visual token.
  • each region image feature corresponds to one visual token, or multiple image features corresponding to multiple regions can be represented as one visual token.
  • the attention weights represent the importance of each visual token to the prediction result generated by the visual language model, that is, the visual language model evaluates the extent to which each visual token affects the generation of the prediction result.
  • the task of a visual language model is to answer a question in a noisy image.
  • the noisy image input to the visual language model is an image of a car driving on a highway, and the text input to the visual language model is the question "What is the color of the car?".
  • the visual language model calculates attention weights for all visual tokens in the noisy image. These attention weights represent the importance of each visual token to the current prediction; that is, the visual language model evaluates the extent to which each visual token influences the generation of the prediction.
  • the visual language model may focus on visual tokens representing the car portion of the noisy image.
  • Step 240 Determine a second correlation between at least one adversarial image feature of the adversarial image and at least one text feature of the text;
  • the adversarial cross-modal attention distribution describes the attention distribution between the adversarial image and the text.
  • calculation method for adversarial cross-modal attention distribution can refer to the calculation method for noisy cross-modal attention distribution, and will not be repeated in the embodiments of this application.
  • the purified image is the image after applying a second perturbation to the adversarial image.
  • the second noise refers to random noise added to the adversarial image.
  • the second noise is generated randomly or according to certain rules, with the purpose of perturbing the pixel values of the adversarial image.
  • the second noise can be Gaussian noise, uniform noise, or other types of random noise.
  • the second correlation can be guided to approach the first correlation, thereby generating a cleaned image.
  • the random noise added to the adversarial image is noise value randomly sampled within a range of a range distribution.
  • the second noise is added to the adversarial image to generate a cleaned image, with the optimization objective of minimizing the difference between the first and second correlations.
  • a sanitized image may exhibit greater perturbation than an adversarial image.
  • Visual language models VLAs may produce incorrect predictions based on adversarial images but correct predictions based on sanitized images.
  • sanitized images may be visually noisier, the sanitized cross-modal attention distribution (i.e., the correlation between at least one sanitized image feature and at least one text feature) is likely closer to the clean cross-modal attention distribution (i.e., the correlation between at least one clean image feature and at least one text feature) of a clean image.
  • the clean sample includes a clean image and text; the method further includes: determining a third correlation between at least one clean image feature of the clean image and at least one text feature of the text; wherein a first correlation is closer to the third correlation than a second correlation.
  • the noisy cross-modal attention distribution i.e., the first correlation
  • the clean cross-modal attention distribution i.e., the third correlation
  • the adversarial cross-modal attention distribution i.e., the first correlation
  • calculation method for the third correlation can refer to the calculation methods for the first and second correlations, and this application will not elaborate on this further.
  • the method provided in this embodiment addresses the erroneous predictions made by the visual language model based on adversarial examples.
  • a cleaned image is obtained, which in turn yields a cleaned sample.
  • a noisy image is obtained by randomly adding noise to the adversarial image.
  • the noisy cross-modal attention distribution of the noisy image i.e., the first correlation between at least one noisy image feature of the noisy image and at least one text feature of the text
  • the cross-modal attention distribution of the clean image i.e., the correlation between at least one clean image feature of the clean image and at least one text feature of the text
  • random noise is added to the adversarial image with the noisy cross-modal attention distribution of the noisy image as the optimization objective.
  • this makes the adversarial cross-modal attention distribution of the adversarial image i.e., the second correlation between at least one adversarial image feature of the adversarial image and at least one text feature of the text
  • this application guides the adversarial image sanitization process by adding noise to adversarial examples, guided by a cross-modal attention distribution.
  • the visual language model can then obtain correct predictions based on the sanitized image (or sanitized example), enhancing the robustness of the visual language model.
  • this method of adding noise to adversarial images using a cross-modal attention distribution as the estimation method not only processes the adversarial image but also optimizes the cross-modal attention distribution itself. This optimization allows the attention distribution to more accurately focus on key information in the image, thereby improving the accuracy of the visual language model's output predictions.
  • the noisy cross-modal attention distribution of a noisy image is closer to the clean cross-modal attention distribution of a clean image than the adversarial cross-modal attention distribution of an adversarial image.
  • adversarial image cleansing is achieved by guiding the adversarial cross-modal attention distribution of the adversarial image to approximate the noisy cross-modal attention distribution of the noisy image.
  • the noisy cross-modal attention distribution corresponding to the noisy image is calculated using a visual language model.
  • the noisy cross-modal attention distribution corresponding to the noisy image can also be calculated using other models or other models similar to the visual language model. This application embodiment does not limit this.
  • FIG. 4A shows a flowchart of a sample purification method provided by an exemplary embodiment of this application, wherein step 230 can be replaced by steps 231, 232 and 233.
  • Step 231 Divide the noisy image into multiple regions based on the visual encoder, and encode each region as a region image feature; align the region image features of multiple regions through a visual language model to obtain at least one noisy image feature;
  • the visual language model includes a visual encoder, which extracts feature vectors from the noisy image and encodes the noisy image into region image features (or visual features) of multiple regions.
  • the visual encoder receives the noisy image as input and segments it into several regions, each of which can be of the same size. Then, feature extraction is performed on image patches within each region, and each image patch can be encoded as an image feature (i.e., a region image feature).
  • at least one noisy image feature is obtained by aligning the multiple region image features using a language alignment module in the visual language model.
  • Each noisy image feature corresponds to a visual token, or in other words, each visual token is used for at least one noisy image feature.
  • Each visual token or noisy image feature can be a fused representation of several region image features.
  • one region of the noisy image corresponds to one region image feature
  • a visual token can be a feature representation fused from several image features.
  • the language alignment module then processes these features into 32 visual tokens, each of which can be a fused representation of the 256 image features.
  • a region image feature can be viewed as a visual token or a noisy image feature in the noisy image, with each visual token indicating a region image feature in the noisy image.
  • multiple region image features can be viewed as a visual token or a noisy image feature in the noisy image, with each visual token indicating image features of multiple regions in the noisy image. For example, if the noisy image is divided into 16 ⁇ 16 regions, each region corresponding to a region image feature, the visual language model processes the noisy image to obtain 32 visual tokens, and each visual token can be a fused representation of the image features of all regions.
  • the following example illustrates the image features of a region of an image using a visual token.
  • a noisy image showing a man hitting a ball on a grassy field.
  • This noisy image is input into a visual language model.
  • the visual encoder in the model segments the noisy image into several fixed-size regions, for example, 32 regions.
  • the visual encoder extracts the features of that region to obtain image features.
  • the image features of each region are encoded into a visual token, and each visual token represents the visual information of the corresponding region in the noisy image.
  • region 1 represents grass in the noisy image, and visual token 1 is used to indicate the image features representing the grass portion in the noisy image;
  • region 2 represents grass and a man's feet in the noisy image, and visual token 2 is used to indicate the image features representing the grass and the man's feet portion in the noisy image;
  • region 3 represents a man's head in the noisy image, and visual token 3 is used to indicate the image features representing the man's head portion in the noisy image;
  • region 4 represents a man's arm and a golf club in the noisy image, and visual token 4 is used to indicate the image features representing the man's arm and the golf club portion in the noisy image;
  • region 5 represents a ball in the noisy image, and visual token 5 is used to indicate the image features representing the ball portion in the noisy image;
  • region 6 represents the base of the golf club and grass in the noisy image, and visual token 6 is used to indicate the image features representing the base of the golf club and the grass portion in the noisy image... and so on.
  • Step 232 Divide the text into multiple text units based on the language model word segmenter, and encode each text unit as a text unit feature; align the text unit features of multiple text units through the visual language model to obtain at least one text feature;
  • the visual language model includes a language model tokenizer, which extracts feature vectors from the text and encodes the text into text unit features of multiple text units.
  • the language model tokenizer receives text as input and breaks down the input text into smaller text units (i.e., text units, which may include words, subwords, or strings).
  • each sub-word can be encoded by a language model word segmenter to obtain the sub-word features (i.e., text unit features, which can also be understood as a feature vector). Then, each sub-word feature is used as a text feature, or multiple sub-word features are used as a text feature.
  • sub-word features i.e., text unit features, which can also be understood as a feature vector.
  • each sub-word can be encoded by a language model word segmenter to obtain the sub-word feature (i.e., text unit feature, which can also be understood as a feature vector).
  • the visual language model can also preserve the order information of each sub-word in the text and add a positional code to the sub-word feature of each sub-word to obtain a new sub-word feature.
  • the positional code is a value calculated based on the position of the sub-word in the text. Then, each new sub-word feature is used as a text feature, or multiple new sub-word features are used as a text feature.
  • the following example illustrates the textual features of a subword of a text using a language token.
  • the language model segmenter within the visual language model breaks the text down into several sub-words, for example, breaking "What is this man doing?" down into the following sub-words: ["this", “man”, “in”, “doing", "what", "?”].
  • the language model segmenter extracts the corresponding feature vector.
  • each sub-word's feature vector may be appended with a positional encoding.
  • the positional encoding is a value calculated based on the sub-word's position in the text.
  • the language alignment module in the visual language model aligns the feature vector and positional encoding of each sub-word, generating language tokens.
  • Each language token represents the textual features of a sub-word in a corresponding region of the text. For example, subword 1 represents “this” in the text, and language token 1 is used to indicate the text features of subword “this”; subword 2 represents “man” in the text, and language token 2 is used to indicate the text features of subword “man”; subword 3 represents “at” in the text, and language token 3 is used to indicate the text features of subword “at”; subword 4 represents “do” in the text, and language token 4 is used to indicate the text features of subword “do”; subword 5 represents “what” in the text, and language token 5 is used to indicate the text features of subword “what”... and so on.
  • Step 233 Calculate the first correlation when generating noisy prediction results using a visual language model.
  • the denoising prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one linguistic token for the denoised image; where each visual token corresponds to a denoised image feature and each linguistic token corresponds to a text feature.
  • all visual tokens and their corresponding attention weights in the noisy image are integrated together to form an overall representation of the noisy image.
  • the visual language model generates a noisy prediction result corresponding to the noisy image based on the integrated visual tokens and the description of the corresponding language tokens.
  • This noisy prediction result is the prediction result of the visual language model on the integrated visual tokens (noisy image) based on the language tokens (the question description corresponding to the text).
  • the visual language model when the visual language model generates a noisy prediction result based on a noisy image, it calculates the attention weights of all visual tokens in the noisy image.
  • the attention weights represent the importance of each visual token to the noisy prediction result; that is, the visual language model evaluates the extent to which each visual token influences the generation of the noisy prediction result.
  • the visual language model when generating the noisy prediction result, the visual language model focuses on visual tokens in the noisy image whose attention weights are greater than a first preset weight threshold, and ignores visual tokens in the noisy image whose attention weights are less than a second preset weight threshold.
  • the noisy cross-modal attention distribution can be represented by a noisy cross-modal attention distribution map, that is, the noisy cross-modal attention distribution map is a visual representation of the noisy cross-modal attention distribution.
  • the noisy cross-modal attention distribution map has several dimensions; for example, it can represent the number of layers and the number of visual tokens in the visual language model.
  • the noisy cross-modal attention distribution map can more clearly represent the attention distribution of visual tokens in a noisy image.
  • visual tokens in the noisy image with attention weights greater than a first preset weight threshold can be represented as a first form
  • visual tokens in the noisy image with attention weights less than a second preset weight threshold can be represented as a second form
  • visual tokens in the noisy image with attention weights less than the first preset weight threshold and greater than the second preset weight threshold can be represented as a third form.
  • the visual saliency of the first form is higher than that of the third form
  • the visual saliency of the third form is higher than that of the second form. This application does not limit this.
  • region 1 in the noisy image represents the grass
  • region 2 represents the grass and the man's feet
  • region 3 represents the man's head
  • region 4 represents the man's arms and the club
  • region 5 represents the ball
  • region 6 represents the sky, etc.
  • sub-word 1 in the text represents "this”
  • sub-word 2 represents “man”
  • sub-word 3 represents “at”
  • sub-word 4 represents “doing”
  • sub-word 5 represents "what", etc.
  • the visual language model When generating a noisy prediction result for the noisy image, the visual language model calculates the attention weight of each sub-word in the text to each region (visual token) in the noisy image. This reflects the degree of attention the visual language model pays to the visual token when generating the noisy prediction result.
  • the visual language model may focus on regions with high attention weights in the noisy image.
  • the attention weight of the visual token corresponding to region 1 in the noisy image is 0.05
  • the attention weight of the visual token corresponding to region 2 is 0.1
  • the attention weight of the visual token corresponding to region 3 is 0.1
  • the attention weight of the visual token corresponding to region 4 is 0.3
  • the attention weight of the visual token corresponding to region 5 is 0.2
  • the attention weight of the visual token corresponding to region 6 is 0.01, etc.
  • the visual language model when generating noisy prediction results, the visual language model will focus on the "man's arm and club” indicated by region 4 and the "ball” indicated by region 5 in the noisy image, and ignore the "sky” indicated by region 6 in the noisy image.
  • the visual language model combined with the question description corresponding to the text, obtains the noisy prediction result as "throwing the ball,” "hitting the ball,” or other results.
  • the embodiments of this application do not limit the structure of the visual language model.
  • the visual encoder and the language model segmenter described above can be replaced by a single model or module to achieve the functions of the visual encoder and the language model segmenter.
  • the method provided in this application encodes a noisy image into several region image features using a visual encoder.
  • Each visual token represents the image features (one or more sets of feature vectors) of one or more regions of the image.
  • This decomposition simplifies the complex information of the noisy image into a set of structured features, which helps the visual language model process and understand different parts of the noisy image.
  • the text is then encoded into several sets of text unit features, i.e., feature vectors, using a language model tokenizer.
  • Each language token represents one or more sub-words of the text. This decomposition transforms the text into structured features, which helps the visual language model understand the semantics of the text.
  • the method also calculates the noisy cross-modal attention distribution using a visual language model, allowing the model to focus on the correlation between the noisy image and the text when generating noisy prediction results. This helps the model determine which image regions in the noisy image are most relevant to which sub-words in the text, thereby more accurately understanding and interpreting the relationship between the image and the text, and generating more accurate and relevant prediction results.
  • adversarial image cleansing can be achieved by guiding the adversarial cross-modal attention distribution of the adversarial image to approximate the noisy cross-modal attention distribution of the noisy image.
  • the adversarial cross-modal attention distribution corresponding to the adversarial image is calculated using a visual language model.
  • the adversarial cross-modal attention distribution corresponding to the adversarial image can also be calculated using other models or other models similar to the visual language model. This application embodiment does not limit this.
  • FIG. 4B shows a flowchart of a sample purification method provided by an exemplary embodiment of this application, wherein step 240 can be replaced by steps 241, 242 and 243.
  • Each visual token or adversarial image feature can be a fused representation of several region image features.
  • one region of the adversarial image corresponds to one region image feature
  • a visual token can be a feature representation fused from several image features.
  • Each visual token can be a fused representation of the 256 image features.
  • a region image feature can be viewed as a visual token or adversarial image feature of the adversarial image, with each visual token indicating a region image feature of the adversarial image.
  • multiple region image features can be viewed as a visual token or adversarial image feature of the adversarial image, with each visual token indicating image features of multiple regions of the adversarial image.
  • the adversarial image is divided into 16 ⁇ 16 regions, each region corresponding to a region image feature.
  • the visual language model processes the adversarial image to obtain 32 visual tokens, each of which can be a fused representation of the image features of all regions.
  • Step 242 Divide the text into multiple text units based on the language model word segmenter, and encode each text unit as a text unit feature; align the text unit features of multiple text units through the visual language model to obtain at least one text feature;
  • step 242 For an explanation of step 242, please refer to step S232. This application embodiment will not repeat the details.
  • Step 243 Calculate the second correlation when generating adversarial test results using a visual language model.
  • the adversarial prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one linguistic token for the adversarial image; where each visual token corresponds to an adversarial image feature and each linguistic token corresponds to a text feature.
  • the adversarial prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one linguistic token for the adversarial image.
  • all visual tokens and their corresponding attention weights in the adversarial image are integrated to form an overall representation of the adversarial image.
  • the visual language model generates an adversarial prediction result corresponding to the adversarial image based on the integrated visual tokens and the description of the corresponding language tokens.
  • This adversarial prediction result is the prediction result of the visual language model on the integrated visual tokens (adversarial image) based on the language tokens (problem descriptions corresponding to the text).
  • the visual language model when the visual language model generates adversarial prediction results based on an adversarial image, it calculates the attention weights of all visual tokens in the adversarial image.
  • the attention weights represent the importance of each visual token to the adversarial prediction result; that is, the visual language model evaluates the extent to which each visual token influences the generation of the adversarial prediction result.
  • the visual language model when generating adversarial prediction results, the visual language model focuses on visual tokens in the adversarial image whose attention weights are greater than a first preset weight threshold, and ignores visual tokens in the adversarial image whose attention weights are less than a second preset weight threshold.
  • the adversarial cross-modal attention distribution can be represented by an adversarial cross-modal attention distribution map, that is, the adversarial cross-modal attention distribution map is a visual representation of the adversarial cross-modal attention distribution.
  • the adversarial cross-modal attention distribution map has several dimensions; for example, it can represent the number of layers in the visual language model and the number of visual tokens.
  • the adversarial cross-modal attention distribution map can more clearly represent the attention distribution of visual tokens in the adversarial image.
  • visual tokens in the adversarial image with attention weights greater than a first preset weight threshold can be represented in a first form
  • visual tokens in the adversarial image with attention weights less than a second preset weight threshold can be represented in a second form
  • visual tokens in the adversarial image with attention weights less than the first preset weight threshold and greater than the second preset weight threshold can be represented in a third form.
  • the visual saliency of the first form is higher than that of the third form
  • the visual saliency of the third form is higher than that of the second form. This application does not limit this.
  • the method provided in this application encodes adversarial images into several regions of image features using a visual encoder.
  • Each visual token represents the image features (one or more sets of feature vectors) of one or more regions of the image.
  • This decomposition simplifies the complex information of the adversarial image into a set of structured features, which helps the visual language model process and understand different parts of the adversarial image.
  • the text is then encoded into several sets of text unit features, i.e., feature vectors, using a language model tokenizer.
  • Each language token represents one or more subwords of the text. This decomposition transforms the text into structured features, which helps the visual language model understand the semantics of the text.
  • the model can focus on the correlation between the adversarial image and the text when generating adversarial prediction results. This helps the visual language model determine which image regions in the adversarial image are most relevant to which subwords in the text, thereby more accurately understanding and interpreting the relationship between the image and the text, and generating more accurate and relevant prediction results.
  • a first correlation is used as a target to guide a second correlation to approach the first correlation.
  • a second noise is added to the adversarial image to generate a cleaned image, with the goal of minimizing the difference between the first and second correlations.
  • FIG. 5 shows a flowchart of a sample purification method provided by an exemplary embodiment of this application, wherein step 250 can be replaced by step 251.
  • Step 251 With the goal of minimizing the correlation difference, add a second noise to the adversarial image to clean it and obtain a cleaned image.
  • the correlation difference refers to the difference between the first and second correlations.
  • This correlation difference can also be called the attention difference or attention value, which represents the difference between the noisy cross-modal attention distribution and the adversarial cross-modal attention distribution.
  • the noisy cross-modal attention distribution is the cross-modal attention distribution corresponding to the noisy image, used to describe the attention distribution of the noisy image and the text.
  • the adversarial cross-modal attention distribution is the cross-modal attention distribution corresponding to the adversarial image, used to describe the attention distribution of the adversarial image and the text.
  • the noisy cross-modal attention distribution corresponding to the noisy image is calculated based on visual tokens and linguistic tokens in the noisy image
  • the adversarial cross-modal attention distribution corresponding to the adversarial image is calculated based on visual tokens and linguistic tokens in the adversarial image.
  • an attention difference value can be obtained based on the adversarial cross-modal attention distribution and the noisy cross-modal attention distribution, where the attention difference value is the difference between the noisy cross-modal attention distribution and the adversarial cross-modal attention distribution.
  • a second noise is added to the adversarial image with the goal of minimizing the attention difference value.
  • the attention difference value can be expressed as:
  • ⁇ A A(x′ i ,x t ,f)-A(R(x′ i , ⁇ ⁇ ),x t ,f)
  • ⁇ A represents the attention difference value
  • x′ i represents the adversarial image
  • R(x′ i , ⁇ ⁇ ) represents the noisy image, that is, the noisy image obtained by adding the first noise ⁇ ⁇ to the adversarial image x′ i
  • x t represents the text
  • f represents the visual language model
  • A(x′ i ,x t ,f) represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the noisy cross-modal attention distribution corresponding to the noisy image.
  • the noisy cross-modal attention distribution corresponding to the noisy image is close to the clean cross-modal attention distribution corresponding to the clean image. Therefore, the noisy cross-modal attention distribution can be considered as a reference point for cleansing.
  • adversarial image cleansing is achieved by guiding the adversarial cross-modal attention distribution to approach the noisy cross-modal attention distribution, i.e., minimizing the difference (attention difference value) between the adversarial cross-modal attention distribution and the noisy cross-modal attention distribution.
  • adding a second noise to the adversarial image can influence the attention distribution (or attention weights) of visual tokens in the adversarial cross-modal attention distribution, thereby achieving adversarial image cleansing and obtaining a cleaned image.
  • the purified image can be represented as:
  • $ ⁇ frac ⁇ x'_i ⁇ $ represent the cleaned image
  • $x'_i$ represent the adversarial image
  • $R( x'_i , ⁇ _ ⁇ )$ represent the noisy image
  • $ x_t $ represent the text
  • $f$ represent the visual language model
  • $ ⁇ $ represent the second noise, which is determined based on the second noise value obtained by pseudo-random sampling in the second distribution interval [0, ⁇ _ ⁇ ], where the second distribution interval [0, ⁇ _ ⁇ ] is a uniformly distributed interval.
  • $A( x'_i , x_t , f)$ represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • $A(R( x'_i , ⁇ _ ⁇ ), x_t , f)$ represents the noisy cross-modal attention distribution corresponding to the noisy image
  • $A( x'_i , x_t , f) - A(R( x'_i , ⁇ _ ⁇ ), x_t , f)$ represents the attention difference value. This represents minimizing the attention difference value.
  • a second noise is added to the adversarial image to obtain a cleaned image, with the goal of minimizing attention differences.
  • the second noise added to the adversarial image is uniform noise.
  • an image with added second noise is input into a visual language model for prediction.
  • the image is determined to be a cleaned image based on the visual language model's prediction result for the image with added second noise: if the visual language model's prediction result for the image is correct (the same as the prediction result for the clean image), the image can be determined to be a cleaned image; if the visual language model's prediction result for the image is incorrect (different from the prediction result for the clean image), random noise is continued to be added to the adversarial image based on the noisy cross-modal attention distribution to adjust the attention distribution of visual tokens in the adversarial image until the visual language model's prediction result is a correct prediction result.
  • a preset question-answering score threshold is set in the visual language model, and the prediction result of the visual language model is determined based on the preset question-answering score threshold to determine whether the prediction result is correct.
  • the prediction result of the visual language model is a correct prediction result, and the image at this time is a cleaned image; when the question-answering score of the visual language model for the image after adding second noise to the adversarial image is less than the preset question-answering score threshold, the prediction result of the visual language model is an incorrect prediction result, and random noise is continued to be added to the adversarial image based on the noisy cross-modal attention distribution until the visual language model outputs a correct prediction result.
  • This application does not limit the specific value of the preset question-answering score threshold.
  • adding a second noise to the adversarial image based on the noisy cross-modal attention distribution and the adversarial cross-modal attention distribution to clean the adversarial image and obtain a cleaned image includes: adding a second noise to the adversarial image with the goal of minimizing a target difference value to obtain the cleaned image; wherein the target difference value is the product or sum of an attention difference value and a preset coefficient, and the attention difference value is the difference between the noisy cross-modal attention distribution and the adversarial cross-modal attention distribution.
  • the embodiments of this application do not limit the optimization target.
  • a second noise is added to the adversarial image.
  • This guidance method which uses the cross-modal attention distribution as the estimation direction, can obtain a cleaned image. This method not only processes the noise in the adversarial image, but also optimizes the cross-modal attention distribution mechanism itself, so that the attention mechanism can more accurately focus on the key information in the adversarial image and ignore irrelevant or interfering information, thereby improving the accuracy and efficiency of the cross-modal task.
  • a second noise is added to the adversarial image to generate a cleaned image, with the objective of minimizing the difference between the adversarial cross-modal attention distribution and the noisy cross-modal attention distribution.
  • the cleaned image is generated by controlling the direction of the noise added to the adversarial image.
  • a second noise is added to the adversarial image to generate a cleaned image, with the objective of minimizing the correlation difference.
  • the cleaned image is generated by controlling the direction of the noise added to the adversarial image.
  • FIG. 6 shows a flowchart of a sample purification method provided by an exemplary embodiment of this application, wherein step 251 can be replaced by steps 251a, 251b and 251c.
  • Step 251a Calculate the norm of the correlation difference
  • the norm is used to measure the distance between the first correlation and the second correlation.
  • both the first correlation and the second correlation can be represented as matrices, and the norm is used to measure the spatial distance between the first correlation and the second correlation.
  • the norm is the L2 norm (Euclidean norm), which is the square root of the sum of the squares of the vectors (attention matrix) and is used to measure the magnitude of the difference between the primary and secondary correlations.
  • the norm value corresponding to the norm provides a quantitative measure.
  • a smaller norm value indicates that the primary and secondary correlations are similar, while a larger norm value indicates that the primary and secondary correlations are significantly different.
  • calculating the norm of the correlation difference can also be understood as calculating the loss function of the correlation difference (the L2 distance between correlation differences).
  • the norm of the correlation difference can be expressed as:
  • x′ i represents the adversarial image
  • R(x′ i , ⁇ ⁇ ) represents the noisy image
  • x t represents the text
  • f represents the visual language model
  • A(x′ i ,x t ,f) represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the noisy cross-modal attention distribution corresponding to the noisy image
  • the first correlation, A(x′ i ,x t ,f)-A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the correlation difference
  • A(x′ i ,x t ,f)-ARxi′, ⁇ ,xt,f2 represents the norm of the correlation difference.
  • Step 251b Calculate the norm of the correlation difference for the gradient of each pixel in the adversarial image
  • the norm of the correlation difference can be considered as a loss function for the correlation difference, where the norm of the correlation difference is relative to the gradient of each pixel in the adversarial image.
  • This can be understood as the loss function being relative to the gradient of each pixel in the adversarial image.
  • the gradient is used to indicate the direction of the fastest descent of the loss function; that is, the gradient is used to indicate how to add a second noise to minimize the correlation difference.
  • the gradient can be expressed as:
  • A(x′ i ,x t ,f) represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the noisy cross-modal attention distribution corresponding to the noisy image. This represents the gradient of pixels in an adversarial image.
  • Step 251c Determine the noise addition method for each pixel based on the gradient of each pixel, and add second noise to each pixel according to the noise addition method of each pixel to obtain a cleaned image.
  • the gradient direction is determined based on a sign function.
  • the gradient direction is used to indicate the direction of adding second noise to the adversarial image, that is, adding second noise to the pixels in the adversarial image.
  • the pixels in the adversarial image are adjusted toward the gradient direction to reduce correlation differences.
  • the noise addition direction of each pixel is determined based on the gradient of each pixel in the adversarial image. This can also be understood as determining the noise addition method of each pixel based on the gradient of each pixel in the adversarial image.
  • the noise addition direction is the same as the gradient direction; that is, the noise addition method is to add a second noise along the gradient direction to the pixel of the adversarial image.
  • the noise addition method includes increasing the pixel value of the pixel in the adversarial image; or decreasing the pixel value of the pixel in the adversarial image; or keeping the pixel value unchanged (the case of unchanged pixel values is almost non-existent; essentially, the pixel value of each pixel in the adversarial image changes).
  • determining the noise addition direction (or noise addition method for each pixel) of the adversarial image based on the sign function can be expressed as:
  • A(x′ i ,x t ,f) represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the noisy cross-modal attention distribution corresponding to the noisy image
  • sign( ⁇ ) represents the sign function. This represents the gradient.
  • the gradient is used as input to the sign function.
  • the sign function outputs 1, indicating that the noise addition method for the i-th pixel is the first noise addition method, which adds second noise to the pixel value of the i-th pixel to increase its pixel value.
  • the sign function outputs -1, indicating that the noise addition method for the i-th pixel is the second noise addition method, which adds second noise to the pixel value of the i-th pixel to decrease its pixel value.
  • the sign function When the gradient is zero (i.e., the gradient of the i-th pixel in the adversarial image is zero), the sign function outputs 0, indicating that the noise addition method for the i-th pixel is to keep the pixel value unchanged. Optionally, keeping the pixel value of the i-th pixel unchanged (this situation is almost non-existent).
  • a cleaned image is obtained by applying the noise addition method to each pixel in the adversarial image as described above.
  • the gradient direction is determined by using a sign function.
  • the gradient direction is used to indicate the direction of adding second noise to the adversarial image. That is, the noise addition method of each pixel is determined based on the gradient.
  • the adversarial image can be precisely cleaned.
  • Using a sign function can simplify the processing of the gradient, directly converting the gradient into the direction of pixel value adjustment (increase or decrease), thus simplifying the calculation process.
  • the method provided in this application can quantitatively assess the difference between the first and second correlations by calculating the norm of the correlation difference.
  • the purification process becomes targeted.
  • the noise addition method is adaptively adjusted based on the gradient of each pixel.
  • the purified image obtained by adaptively adjusting the method for each pixel in the adversarial image can significantly reduce the impact of adversarial perturbations, enhance the robustness and security of the visual language model, and improve its defense against adversarial attacks.
  • adding second noise to the adversarial image with the goal of minimizing the correlation difference to clean the adversarial image and obtain the cleaned image includes: calculating the minimum difference value of the correlation difference; calculating the gradient of the norm of the minimum correlation difference with respect to each pixel in the adversarial image; determining the noise addition method for each pixel based on the gradient of each pixel; adding second noise to each pixel according to the noise addition method of each pixel to obtain the cleaned image.
  • the correlation difference is actually a matrix, and the minimum correlation difference is the smallest element in that matrix.
  • adding second noise to the adversarial image with the goal of minimizing the correlation difference to clean the adversarial image and obtain the cleaned image includes: calculating the maximum difference value of the correlation difference; calculating the gradient of the norm of the maximum correlation difference with respect to each pixel in the adversarial image; determining the noise addition method for each pixel based on the gradient of each pixel; adding second noise to each pixel according to the noise addition method of each pixel to obtain the cleaned image.
  • the correlation difference is actually a matrix, and the maximum correlation difference is the largest element in that matrix.
  • the gradient of each pixel in the adversarial image is calculated, and the gradient includes both direction and intensity information, which indicates the magnitude of pixel adjustment in the adversarial image.
  • the intensity information of the gradient is used to determine the second noise corresponding to each pixel in the adversarial image, which is used to indicate the gradient magnitude of each pixel in the adversarial image.
  • adversarial images in adversarial examples are divided into pixel-level segments, and second noise of varying perturbation levels (different noise magnitudes) is added to different pixel locations in the adversarial images based on gradient direction.
  • the gradient intensity information is used to indicate the gradient magnitude of each pixel in the adversarial image.
  • pixels with larger gradients in the adversarial image should be subject to greater perturbation, i.e., have relatively larger noise added; pixels with smaller gradients in the adversarial image should have smaller perturbation, i.e., have relatively smaller noise added.
  • the magnitude of the second noise added to each pixel in the adversarial image is determined by the gradient intensity information.
  • the intensity information reflects the magnitude of the gradient and can better guide the addition of the second noise, making the noise addition process more accurate.
  • a first gradient value and a second gradient value are obtained, wherein the first gradient value is the maximum gradient value of all pixels in the adversarial image, and the second gradient value is the minimum gradient value of all pixels in the adversarial image.
  • a gradient standard value is obtained based on the maximum and minimum gradient values, wherein the gradient standard value is the gradient value after normalization.
  • the gradient is normalized to [0, 1].
  • the gradient can be represented as:
  • A(x′ i ,x t ,f) represents the adversarial cross-modal attention distribution corresponding to the adversarial image
  • A(R(x′ i , ⁇ ⁇ ),x t ,f) represents the noisy cross-modal attention distribution corresponding to the noisy image.
  • g ⁇ sub>norm ⁇ /sub> represents the standard gradient value
  • g ⁇ sub> min ⁇ /sub> represents the first gradient value, i.e., the minimum gradient value
  • g ⁇ sub>max ⁇ /sub> represents the second gradient value, i.e., the maximum gradient value.
  • the gradient standard values can be represented based on their average value.
  • the perturbation amplitude of the second noise is determined based on the gradient standard values and their average value.
  • the perturbation amplitude includes a perturbation factor used to determine the magnitude of the second noise, and the magnitude of the second noise added to each pixel in the adversarial image is determined based on the perturbation factor.
  • the magnitude of the second noise is positively correlated with the intensity information of the gradient; that is, pixels with larger gradients (stronger intensity information) in the adversarial image should be subject to greater perturbation, i.e., a relatively larger amount of second noise should be added; pixels with smaller gradients (weaker intensity information) in the adversarial image should be subject to smaller perturbation, i.e., a relatively smaller amount of second noise should be added.
  • the magnitude of a second noise added to each pixel in the adversarial image is determined based on a perturbation factor to obtain a cleaned image.
  • the cleaned image can be represented as:
  • the average value of the gradient standard can be represented by... Determine the amplitude of the second noise disturbance.
  • the value of is a perturbation factor, used to determine the magnitude of the second noise ⁇ .
  • 0 represents the first normalized value
  • 1 represents the second normalized value.
  • the perturbation factor is the maximum of the first normalized value and the first value
  • the first value is the minimum of the second normalized value and the second value
  • the second value is the quotient of the standard gradient value and the average of the standard gradient values. That is, the first value is...
  • the second value is g norm and
  • the disturbance factor is a value within the disturbance amplitude.
  • the second noise is determined based on the product of a second noise value obtained by pseudo-random sampling in a second distribution interval and a perturbation factor, wherein the second distribution interval is a uniformly distributed interval range.
  • the size of the second noise corresponding to each pixel in the adversarial image can also be a preset size, that is, it is not necessary to use the intensity information of the gradient to determine the size of the second noise.
  • the embodiments of this application do not limit the calculation method of the above-mentioned gradient standard value.
  • the gradient standard value is the average value of the gradients of all pixels in the adversarial image.
  • a visual language model can be pre-trained. Inputting a sanitized sample into the visual language model yields a question-answer score, which indicates the probability that the sanitized sample belongs to the correct prediction (correct category).
  • the question-answer score is positively correlated with the sanitized cross-modal attention distribution of the sanitized sample.
  • the visual language model is trained as follows:
  • a clean sample dataset is obtained.
  • This dataset is a collection of samples used to train a visual language model.
  • the clean sample dataset includes multiple clean samples, each comprising a clean image and text.
  • the clean image is an image obtained using the method provided in the above embodiments.
  • a corresponding label on each clean sample indicates the category to which the clean image belongs.
  • the clean samples are input into the visual language model, which encodes the clean image as image features and the text as text features.
  • the visual language model fuses image features and text features based on the clean cross-modal attention distribution of the clean image.
  • the clean cross-modal attention distribution describes the attention distribution between the clean image and the text, allowing the visual language model to focus on text-related image regions within the clean image based on this distribution.
  • the visual language model predicts the fused features, which are features generated by fusing image and text features based on the attention distributions of image and text in the cleansing cross-modal attention distribution.
  • the visual language model predicts based on the fused features to obtain a question-answer score for the cleaned sample, which indicates the probability that the cleaned sample belongs to the correct prediction result (correct category).
  • the question-answer score is positively correlated with the cleansing cross-modal attention distribution of the cleaned image.
  • a score threshold is set in the visual language model.
  • the question-answer score of the visual language model When the question-answer score of the visual language model is greater than the score threshold, it indicates that the cleaned sample belongs to the correct prediction result; when the question-answer score of the visual language model is less than or equal to the score threshold, it indicates that the cleaned sample belongs to the incorrect prediction result, indicating that the input cleaned sample belongs to the incomplete cleansing result.
  • the visual language model's prediction result for the cleaned sample is an incorrect prediction result, it indicates that the cleansing cross-modal attention distribution in the cleaned sample needs to be further adjusted, that is, random noise needs to be added to the adversarial examples to adjust the adversarial cross-modal attention distribution of the adversarial examples.
  • the predicted result corresponding to the question-answer score is compared with the true result (the label corresponding to the cleaned sample) to obtain a calculated loss, which is used to indicate the error between the predicted result corresponding to the question-answer score and the true result.
  • a backpropagation algorithm is used to adjust the parameters of the visual language model so that the predicted result of the visual language model is closer to the label corresponding to the cleaned sample.
  • the visual language model is optimized through a loss function.
  • the loss function can be any type of loss function, for example, it can be the CE loss (Cross Entropy loss) function or the Mean Square Error (MSE) loss function.
  • the training of the visual language model provided in this embodiment is only illustrative and does not constitute a limitation on the training method of the visual language model.
  • the visual language model can be applied to visual question answering scenarios.
  • sample sanitization can be applied to the field of classification question answering.
  • the sanitized sample is input into a visual language model, which then performs recognition based on the sanitized sample to obtain the correct question answer corresponding to the sanitized sample.
  • Step 1 obtain classification adversarial examples
  • Classification adversarial examples are adversarial examples generated after performing adversarial attacks on clean samples.
  • Classification adversarial examples include a classification adversarial image and a problem description.
  • adversarial attacks involve making minor modifications or perturbations to clean samples. Modifications or perturbations in classification adversarial examples are often imperceptible, but can cause the visual language model to make incorrect predictions and generate incorrect output.
  • a clean image shows a bicycle parked in front of a building.
  • the question for this clean image is described as "What material is the door of this building made of?”
  • the visual language model correctly classifies it as "wood”.
  • an adversarial attack such as an adversarial attack based on the FGSM method
  • a classification adversarial image can be generated.
  • the question for this classification adversarial image is described as "What material is the door of this building made of?”
  • the visual language model incorrectly classifies it as "metal”.
  • the visual language model calculates the adversarial cross-modal attention distribution corresponding to the clean image based on visual tokens and linguistic tokens in the clean image.
  • the clean image is encoded into image features based on a visual encoder, and at least one visual token is obtained by aligning the image features through the visual language model, where each of the at least one visual token is used to indicate the image features of the clean image;
  • the question description is encoded into text features based on a language model tokenizer, and at least one linguistic token is obtained by aligning the text features through the visual language model, where each of the at least one linguistic token is used to indicate the text features of the question description;
  • the clean cross-modal attention distribution for generating a clean prediction result is calculated through the visual language model, where the clean prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one linguistic token.
  • the generated clean prediction result is "wood".
  • the visual language model calculates the attention weights of all visual tokens in the clean image.
  • the visual language model divides the clean image into several regions.
  • the image features of each region (or the image features corresponding to multiple regions) can be represented as a visual token.
  • the attention weights represent the importance of each visual token to the answer "wood”. That is, the visual language model evaluates the extent to which each visual token affects the generation of "wood”.
  • a visual language model when a visual language model predicts a classification adversarial image, it calculates the adversarial cross-modal attention distribution for that image.
  • the adversarial cross-modal attention distribution is the cross-modal attention distribution corresponding to the classification adversarial image, used to describe the attention distribution between the classification adversarial image and the question description.
  • the process for calculating the adversarial cross-modal attention distribution for a classification adversarial image is similar to the process for calculating the clean cross-modal attention distribution for a clean image described above.
  • Step 2 Obtain the noisy image obtained after adding the first noise to the adversarial image
  • the noisy image is the image after applying a first perturbation to the classification adversarial image.
  • the first noise refers to random noise added to the classification adversarial image.
  • the first noise is generated randomly or according to certain rules, with the aim of perturbing the pixel values of the classification adversarial image.
  • the first noise is added via a computer program.
  • the first noise is added via a noise model, such as using a generative adversarial network, variational autoencoder, or other deep learning model to generate the first noise, which is then added to the classification adversarial image.
  • Step 3 Determine the first correlation between at least one noisy image feature of the noisy image and at least one text feature of the text
  • the noisy cross-modal attention distribution is used to describe the correlation between at least one noisy image feature of the noisy image and at least one text feature of the problem description text.
  • a noisy image is obtained by adding first noise to the class adversarial image.
  • the noisy cross-modal attention distribution of the noisy image is closer to the clean cross-modal attention distribution of the clean image. Therefore, using the noisy cross-modal attention distribution of the noisy image as a reference estimation direction to guide the optimization of the adversarial cross-modal attention distribution of the class adversarial image can be regarded as a cleaning operation for the class adversarial image, making it closer to the clean image.
  • noisy cross-modal attention distribution for this noisy image is closer to the clean cross-modal attention distribution for the clean image.
  • the noisy cross-modal attention distribution corresponding to the noisy image is calculated.
  • the noisy cross-modal attention distribution refers to how the visual language model pays attention to the noisy image in the noisy sample when generating prediction results based on the noisy sample.
  • the adversarial cross-modal attention distribution for noisy images is similar to the process described above for the noisy cross-modal attention distribution of noisy images.
  • Step 4 Determine a second correlation between at least one adversarial image feature of the adversarial image and at least one text feature of the text;
  • Adversarial cross-modal attention distribution is used to describe the correlation between at least one adversarial image feature of the adversarial image and at least one text feature of the problem description text.
  • Step 5 Add second noise to the adversarial image based on the first correlation and the second correlation to clean the adversarial image and obtain a cleaned image.
  • the purified image is the image after applying a second perturbation to the adversarial image.
  • the second noise refers to random noise added to the classification adversarial image.
  • the second noise is generated randomly or according to certain rules, and its purpose is to perturb the pixel values of the classification adversarial image.
  • the visual language model might correctly classify it as "wood".
  • a cleaned image may be visually noisier, the cleaned cross-modal attention distribution of the cleaned image may be closer to the clean cross-modal attention distribution of the clean image. Therefore, the visual language model's prediction for the cleaned image is closer to its prediction for the clean image.
  • the second correlation can be guided to approach the first correlation, thereby generating a cleaned image.
  • a second noise is added to the classification adversarial image with the goal of minimizing the correlation difference, resulting in a cleaned image.
  • the direction of adding second noise can be determined.
  • the norm of the correlation difference is calculated, which measures the distance between the first and second correlations.
  • the gradient of the correlation difference with respect to pixels in the classification adversarial image is calculated.
  • the gradient direction is determined based on the sign function; this can also be understood as determining the noise addition method for each pixel based on its gradient, with the gradient direction indicating the direction of adding second noise.
  • the gradient is used as input to the sign function.
  • the gradient is positive, the pixel value in the adversarial image is increased according to the gradient direction; when the gradient is negative, the pixel value in the adversarial image is decreased according to the gradient direction.
  • Increasing or decreasing the pixel value in the adversarial image according to the gradient direction yields a cleaned image after adding second noise.
  • the magnitude of the added second noise can also be determined.
  • the gradient includes not only gradient direction information but also intensity information.
  • pixels with larger gradients in the adversarial image should be subjected to greater perturbation, i.e., relatively larger noise should be added; pixels with smaller gradients in the adversarial image should be subjected to smaller perturbation, i.e., relatively smaller noise should be added.
  • visual language models produce erroneous predictions based on classification adversarial examples.
  • cleaned images can be obtained, thus leading to cleaned samples.
  • a noisy image is obtained by randomly adding noise to the classification adversarial image. Since the noisy cross-modal attention distribution (i.e., the first correlation) of the noisy image is very close to the cross-modal attention distribution (i.e., the second correlation) of the clean image, the noisy cross-modal attention distribution of the noisy image can be used as the optimization target.
  • Random noise is added to the classification adversarial image, thereby making the adversarial cross-modal attention distribution of the classification adversarial image close to that of the noisy cross-modal attention distribution, thus completing the cleansing of the classification adversarial image and obtaining a cleaned image.
  • This application guided by the noisy cross-modal attention distribution, achieves the cleansing of classification adversarial images by adding noise.
  • the visual language model can obtain correct prediction results based on the cleaned image. This cleansing method enhances the robustness of the visual language model.
  • This scheme can be called FFF.
  • FFF This scheme can be called FFF.
  • the prediction result f(x ⁇ sub> i ⁇ /sub> , x ⁇ sub> t ⁇ /sub> ) of the visual language model is also correct, and the clean cross-modal attention distribution A(x ⁇ sub>i ⁇ /sub> , x ⁇ sub>t ⁇ /sub> , f) of the clean image is extracted.
  • the prediction result f(x′ ⁇ sub>i ⁇ /sub>, x ⁇ sub> t ⁇ /sub> ) of the visual language model is incorrect, and the adversarial cross-modal attention distribution A(x′ ⁇ sub> i ⁇ /sub> , x ⁇ sub>t ⁇ /sub> , f) of the adversarial image is extracted.
  • the adversarial cross-modal attention distribution of the adversarial image is significantly different from that of the clean image, which can show significant pattern differences between the adversarial cross-modal attention distribution map and the clean cross-modal attention distribution map.
  • Optimizing the adversarial cross-modal attention distribution of the adversarial image towards the noisy cross-modal attention distribution A may enable adversarial image desiccation.
  • the answer is... Correct, and the cleansing of the image's cross-modal attention distribution. It does indeed resemble a clean cross-modal attention distribution more like a clean image.
  • this method verifies whether the clean cross-modal attention distribution of a clean image can cleanse adversarial images.
  • This method targets the clean cross-modal attention distribution of a clean image and adds noise to the adversarial image; the results are shown in Table 1. Higher noise intensity leads to better cleansing, and the cross-modal attention distribution of the adversarial image after adding noise does indeed closely resemble the clean cross-modal attention distribution of the clean image.
  • the mean squared error is the L2 norm mentioned above.
  • the VQA score is the visual question-answering score of the visual language model based on the input image.
  • the first row is the score of the visual language model (visual question-answering score)
  • the second and third rows are measures of the difference between the cross-modal attention distribution corresponding to the adversarial image with added noise and the clean cross-modal attention distribution corresponding to the clean image.
  • this method verifies the impact of randomly adding noise to adversarial images on prediction results and cross-modal attention distribution.
  • Table 2 shows that the visual question-answering score of the visual language model improves to some extent but not significantly.
  • the cross-modal attention distribution corresponding to the noisy image is closer to the clean cross-modal attention distribution of the clean image.
  • This scheme uses the noisy cross-modal attention distribution of a randomly noisy image as an estimate of the clean cross-modal attention distribution of the clean image to add noise to the adversarial image.
  • the noise magnitude is randomly selected, and the estimated noisy cross-modal attention distribution is used to control the direction.
  • Table 3 The results are shown in Table 3. The first row shows the random direction, and the second row onwards represents the method of this scheme. It can be seen that this scheme can indeed achieve a certain degree of performance cleanup.
  • Each row represents the noise intensity added to the estimated clean cross-modal attention distribution of the clean image, and each column represents the noise intensity added to the cleaned adversarial image.
  • this solution only controlled the noise direction, while the noise magnitude was still randomly selected. This solution further controls the noise magnitude, and the results are shown in Table 4. The first row shows the random direction, and the second row onwards represents the method of this solution. It can be seen that controlling the noise magnitude is more effective than optimizing only the direction.
  • Figure 7 shows a structural block diagram of a sample purification device provided in an embodiment of this application.
  • This sample purification device has the function of implementing the sample purification method example described above. The function can be implemented by hardware or by hardware executing corresponding software.
  • the purification device can be the server described above, or it can be installed within a server. As shown in Figure 7, the purification device may include: an acquisition module 710, a determination module 720, and an addition module 730;
  • Module 710 is used for:
  • the adversarial sample is obtained by performing an adversarial attack on a clean sample, and the adversarial sample includes adversarial images and text;
  • the determination module 720 is used for:
  • the addition module 730 is used to add second noise to the adversarial image based on the first correlation and the second correlation in order to clean the adversarial image and obtain a cleaned image.
  • the adding module 730 is specifically used to: add a second noise to the adversarial image with the goal of minimizing the correlation difference, so as to clean the adversarial image and obtain a cleaned image; wherein the correlation difference is the difference between the first correlation and the second correlation.
  • the adding module 730 is specifically used to: calculate the norm of the correlation difference, the norm being used to measure the distance between the first correlation and the second correlation; calculate the gradient of the correlation difference norm for each pixel in the adversarial image; determine the noise addition method for each pixel based on the gradient of each pixel; add second noise to each pixel according to the noise addition method of each pixel to obtain a cleaned image.
  • the adding module 730 is specifically configured to: when the gradient of the i-th pixel in the adversarial image is positive, determine that the noise addition method of the i-th pixel is a first noise addition method, the first noise addition method being used to add second noise to the pixel value of the i-th pixel to increase the pixel value of the i-th pixel, where i is a positive integer; when the gradient of the i-th pixel is negative, determine that the noise addition method of the i-th pixel is a second noise addition method, the second noise addition method being used to add second noise to the pixel value of the i-th pixel to decrease the pixel value of the i-th pixel; and add second noise to each pixel based on the noise addition method for each pixel in the adversarial image to obtain a cleaned image.
  • the gradient includes intensity information
  • the determining module 720 is further configured to: determine a second noise corresponding to each pixel in the adversarial image based on the intensity information of the gradient, wherein the intensity information is used to indicate the gradient magnitude of each pixel in the adversarial image.
  • the determining module 720 is specifically configured to: obtain a first gradient value and a second gradient value, wherein the first gradient value is the maximum gradient value of all pixels in the adversarial image, and the second gradient value is the minimum gradient value of all pixels in the adversarial image; obtain a gradient standard value based on the first gradient value and the second gradient value, wherein the gradient standard value is the gradient value after normalization; determine the perturbation amplitude of the second noise based on the gradient standard value and the average value of the gradient standard value, wherein the perturbation amplitude includes a perturbation factor for determining the magnitude of the second noise; and determine the magnitude of the second noise added to each pixel in the adversarial image based on the perturbation factor, wherein the magnitude of the second noise is positively correlated with the intensity information of the gradient.
  • the perturbation factor is the maximum value of a first normalized value and a first value
  • the first value is the minimum value of a second normalized value and a second value
  • the second value is the quotient of the standard gradient value and the average of the standard gradient values
  • the first normalized value is 0, and the second normalized value is 1.
  • the purification device further includes a generation module 740, wherein the acquisition module 710 is further configured to acquire purification samples, the purification samples including purification images and text; the generation module 740 is configured to generate prediction results for the purification images based on the purification samples using a visual language model, and the prediction results for the purification images are correct prediction results.
  • the determining module 720 is specifically used to: determine a first relevance through a visual language model.
  • the visual language model includes a visual encoder and a language model segmenter; the determination module 720 is specifically configured to: divide the noisy image into multiple regions based on the visual encoder, and encode each region as a region image feature; align the region image features of the multiple regions through the visual language model to obtain at least one noisy image feature; divide the text into multiple text units based on the language model segmenter, and encode each text unit as a text unit feature; align the text unit features of the multiple text units through the visual language model to obtain at least one text feature; calculate the first correlation when generating the noisy prediction result through the visual language model, wherein the noisy prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one language token for the noisy image; wherein each visual token corresponds to a noisy image feature; and each language token corresponds to a text feature.
  • the determining module 720 is specifically used to: determine a second correlation through a visual language model.
  • the visual language model includes a visual encoder and a language model segmenter; the determination module 720 is specifically configured to: divide the adversarial image into multiple regions based on the visual encoder, and encode each region as a region image feature; align the region image features of the multiple regions through the visual language model to obtain at least one adversarial image feature; divide the text into multiple text units based on the language model segmenter, and encode each text unit as a text unit feature; align the text unit features of the multiple text units through the visual language model to obtain at least one text feature; calculate the second correlation when generating the adversarial test result through the visual language model, wherein the adversarial prediction result is the prediction result of the visual language model after fusing at least one visual token and at least one language token for the adversarial image; wherein each visual token corresponds to an adversarial image feature; and each language token corresponds to a text feature.
  • the first noise is determined based on a first noise value obtained by pseudo-random sampling in a first distribution interval.
  • the first distribution interval is a uniformly distributed interval range.
  • the second noise is determined based on a second noise value obtained by pseudo-random sampling in the second distribution interval.
  • the second distribution interval is a uniformly distributed interval range.
  • the clean sample includes a clean image and text; the determining module 720 is further configured to: determine a third correlation between at least one clean image feature of the clean image and at least one text feature of the text; wherein the first correlation is closer to the third correlation than the second correlation.
  • Each module of the above purification device can be implemented entirely or partially through software, hardware, or a combination thereof.
  • Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
  • sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
  • FIG. 8 shows a structural block diagram of a computer device 1200 according to an exemplary embodiment of this application.
  • This computer device can be used to implement the sample purification method provided in the above embodiments.
  • the computer device 1200 includes a central processing unit (CPU) 1201, a system memory 1204 including random access memory (RAM) 1202 and read-only memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the CPU 1201.
  • the computer device 1200 also includes a basic input/output system (I/O system) 1206 to facilitate information transfer between various devices within the computer device, and a mass storage device 1207 for storing an operating system 1213, application programs 1214, and other program modules 1215.
  • I/O system basic input/output system
  • the basic input/output system 1206 includes a display 1208 for displaying information and an input device 1209 for user input, such as a mouse or keyboard. Both the display 1208 and the input device 1209 are connected to the central processing unit 1201 via an input/output controller 1210 connected to the system bus 1205.
  • the basic input/output system 1206 may also include the input/output controller 1210 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input/output controller 1210 also provides output to a display screen, printer, or other types of output devices.
  • the mass storage device 1207 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205.
  • the mass storage device 1207 and its associated computer-readable storage media provide non-volatile storage for the terminal device 1200. That is, the mass storage device 1207 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.
  • CD-ROM compact disc read-only memory
  • the computer-readable storage medium may include computer storage media and communication media.
  • Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data.
  • Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other solid-state storage technologies
  • CD-ROM compact disc
  • DVD digital versatile disc
  • the memory stores one or more programs, which are configured to be executed by one or more central processing units 1201.
  • the one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 1201 executes the one or more programs to implement the sample purification methods provided by the above method embodiments.
  • the computer device 1200 can also be connected to a remote terminal device on a network, such as the Internet. That is, the computer device 1200 can be connected to the network 1212 via the network interface unit 1211 connected to the system bus 1205, or the network interface unit 1211 can be used to connect to other types of networks or remote terminal device systems (not shown).
  • the memory further includes one or more programs stored in the memory, and the one or more programs include the sample purification steps performed by the terminal device in the method provided in the embodiments of this application.
  • This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the sample purification method provided in the above-described method embodiments.
  • This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium.
  • the computer program is read from and executed by a processor of a computer device, causing the computer device to perform the sample purification method provided in the above-described method embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种样本净化方法、装置、设备及介质，属于人工智能技术领域。该方法包括：获取对抗样本，对抗样本包括对抗图像和文本；获取向对抗图像添加第一噪声后得到的加噪图像；确定第一相关性和第二相关性；基于第一相关性和第二相关性向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像。本申请以第一相关性，即跨模态注意力分布为引导，通过在对抗图像中添加噪声以实现净化，这种净化方式不仅是在处理对抗图像，同时也在优化跨模态注意力分布机制本身。视觉语言模型基于净化图像可以得到正确的预测结果，这种净化方式增强了视觉语言模型的鲁棒性。

Description

样本净化方法、装置、设备及介质
本申请要求于2024年07月23日提交中国专利局、申请号为202410994732.0、发明名称为“样本净化方法、装置、设备及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及人工智能技术领域，特别涉及一种样本净化方法、装置、设备及介质。
背景技术
大型视觉-语言模型(Large Vision-Language Models，LVLM)被广泛定义为可以从图像和文本中学习的多模态模型。大型视觉-语言模型(简称视觉语言模型)在处理任务时易受到对抗样本的影响，对抗样本是通过对干净样本进行微小修改或者扰动，使得视觉语言模型产生错误预测的样本。
相关技术中，通过对输入视觉语言模型的图像进行一些预处理操作，从而实现对抗样本的净化。例如，对图像进行平滑操作，即应用平滑滤波器减少图像噪声，从而降低对抗攻击的影响；又例如，对图像进行尺度变化操作，即通过改变图像的尺度(例如缩放)来改变图像特征，减少对抗攻击的有效性。
然而，相关技术中针对对抗样本的净化主要依赖于传统的图像处理技术，导致针对对抗样本的净化效果并不理想，在实际应用中对抗样本仍然会对视觉语言模型的任务输出(预测结果)产生影响。
发明内容
本申请提供了一种样本净化方法、装置、设备及介质，所述技术方案如下：
根据本申请的一方面，提供了一种样本净化方法，所述方法包括：
获取对抗样本，所述对抗样本通过对干净样本执行对抗攻击后生成，所述对抗样本中包括对抗图像和文本；
获取向所述对抗图像添加第一噪声后得到的加噪图像；
确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性；
确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性；
基于所述第一相关性和所述第二相关性向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像。
根据本申请的一方面，提供了一种样本净化装置，所述装置包括：获取模块、确定模块和添加模块；
所述获取模块，用于：
获取对抗样本，所述对抗样本通过对干净样本执行对抗攻击后生成，所述对抗样本中包括对抗图像和文本；
获取向所述对抗图像添加第一噪声后得到的加噪图像；
所述确定模块用于：
确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性；
确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性；
所述添加模块用于基于所述第一相关性和所述第二相关性向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像。
根据本申请的另一方面，提供了一种计算机设备，该计算机设备包括：处理器和存储器，存储器中存储有至少一条计算机程序，至少一条计算机程序由处理器加载并执行以实现如上方面所述的样本净化方法。
根据本申请的另一方面，提供了一种计算机存储介质，计算机可读存储介质中存储有至少一条计算机程序，至少一条计算机程序由处理器加载并执行以实现如上方面所述的样本净化方法。
根据本申请的另一方面，提供了一种计算机程序产品，上述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中；所述计算机程序由计算机设备的处理器从所述计算机可读存储介质读取并执行，使得所述计算机设备执行如上方面所述的样本净化方法。
本申请提供的技术方案带来的有益效果至少包括：
视觉语言模型基于对抗样本会产生错误的预测，通过针对对抗样本中的对抗图像执行净化操作后得到净化图像，进而可以得到净化样本。具体地，通过针对对抗图像进行随机加噪处理得到加噪图像，由于加噪图像的加噪跨模态注意力分布(即加噪图像的至少一个加噪图像特征和文本的至少一个文本特征之间的第一相关性)非常接近干净图像对应的跨模态注意力分布(即干净图像的至少一个干净图像特征和文本的至少一个文本特征之间的相关性)，因此可以将加噪图像的加噪跨模态注意力分布为优化目标，向对抗图像中加入随机噪声，进而使得对抗图像的对抗跨模态注意力分布(即对抗图像的至少一个对抗图像特征和文本的至少一个文本特征之间的第二相关性)接近加噪图像的加噪跨模态注意力分布，以完成针对对抗图像的净化，并得到净化图像。本申请以加噪跨模态注意力分布为引导，通过在对抗图像中添加噪声以实现对抗图像的净化，视觉语言模型基于净化图像可以得到正确的预测结果，这种净化方式增强了视觉语言模型的鲁棒性。
进一步地，以加噪跨模态注意力分布为估计方向引导对抗图像添加随机噪声的方式，不仅是在优化对抗图像，同时也在优化跨模态注意力分布本身，这种优化使得注意力机制能够更准确地聚焦于图像中的关键信息，从而提高视觉语言模型输出预测结果的准确性。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请一个示例性实施例提供的计算机系统的架构示意图；
图2是本申请一个示例性实施例提供的样本净化方法的示意图；
图3是本申请一个示例性实施例提供的样本净化方法的流程图；
图4A是本申请一个示例性实施例提供的样本净化方法的流程图；
图4B示出了本申请一个示例性实施例提供的样本净化方法的流程图；
图5是本申请一个示例性实施例提供的样本净化方法的流程图；
图6是本申请一个示例性实施例提供的样本净化方法的流程图；
图7是本申请一个示例性实施例提供的样本净化装置的结构框图；
图8是本申请一个示例性实施例提供的计算机设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开可能采用术语第一、第二等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一参数也可以被称为第二参数，类似地，第二参数也可以被称为第一参数。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
需要说明的是，本申请所涉及的数据(包括但不限于用于分析的数据、训练的数据、推荐的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。例如，本申请中涉及到的设置操作等信息都是在充分授权的情况下获取的。
在介绍本申请技术方案之前，先对本申请涉及的一些名词进行解释说明。以下相关解释作为可选方案与本申请实施例的技术方案可以进行任意结合，其均属于本申请实施例的保护范围。本申请实施例包括以下内容中的至少部分内容。
视觉语言模型：视觉语言模型是一种结合视觉(图像)和语言(文本)的多模态模型，可以接收图像和文本输入并生成任务输出。视觉语言模型能够理解并处理图像和文本之间的关联，用于处理多种任务，如图像描述生成(Image Captioning)、视觉问答(Visual Question Answering，VQA)、图像文本匹配(Image-Text Matching)等。
干净样本：干净样本是指未经任何修改或扰动的原始样本。干净样本为理想状态下的标准样本，在视觉语言模型的评估中起到基准作用。其中，干净样本中包括干净图像和文本。
对抗样本：对抗样本是通过对干净样本进行微小修改或者扰动，使得视觉语言模型产生错误预测的样本。对抗样本中的扰动不易察觉，但会导致视觉语言模型做出错误的预测或输出。其中，对抗样本中包括对抗图像和文本。本申请实施例中针对对抗样本的净化操作，可以看作是针对对抗图像的净化操作。
加噪样本：在原有的对抗样本上添加了随机噪声的对抗样本。随机噪声添加的范围由无穷范数限制或约束。随机加噪对抗样本的目的是在保持对抗性质的同时，加噪样本的跨模态注意力分布接近于干净样本的跨模态注意力分布。其中，加噪样本中包括加噪图像和文本。
净化样本：指将对抗样本中的微小修改或者扰动进行净化后的样本，净化样本可以提升视觉语言模型生成正确预测的概率。净化样本目的是减少导致视觉语言模型错误预测的对抗性扰动，提升视觉语言模型生成正确预测的概率。其中，净化样本包括净化图像和文本。本申请实施例中，针对对抗样本的净化操作得到净化样本，可以理解为，针对对抗图像的净化操作得到净化图像。
视觉令牌：视觉令牌是从图像中提取的特征表示，通过视觉语言模型中的视觉编码器编码后得到。视觉编码器将输入的图像划分为若干个区域，每个区域可以编码为一个图像特征(或称视觉特征、区域图像特征)，若干个区域可以编码为若干个图像特征，通过视觉语言模型中的语言对齐模块将若干个图像特征对齐后得到多个视觉令牌，每个视觉令牌可以是若干个图像特征的融合表示。例如，图像的一个区域对应一个图像特征，一个视觉令牌可以是若干个图像特征融合后的特征表示。示例性的，视觉语言模型中的视觉编码器将图像划分为16×16＝256个区域，并提取256个图像特征，由语言对齐模块将其处理为32个视觉令牌，每个视觉令牌都可以是256个图像特征的融合表示。可选地，不同视觉令牌融合图像特征的侧重点不同，例如，第一个视觉令牌重点关注图像中的背景区域，即第一个视觉令牌主要表征背景区域的图像特征；第二个视觉令牌重点关注图像中的物体区域，即第二个视觉令牌主要表征物体区域的图像特征。
语言令牌：语言令牌是从文本中提取出来的特征表示，通过视觉语言模型中的语言模型分词器编码后得到。可选地，语言模型分词器将输入的文本分成若干个词(或子词)，然后将每个词转换为相应的特征向量，文本通常可以分为若干个子词，每个子词的特征向量可以表示为一个语言令牌，一个语言令牌通常对应于文本中的一个子词。或者，多个子词的特征向量可以表示为一个语言令牌，一个语言令牌通常对应于文本中的多个子词，例如，一个语言令牌是文本中所有子词的文本特征。
跨模态注意力分布：指的是视觉语言模型在处理多模态数据时的一种注意力机制。跨模态注意力分布指在多模态数据处理中，一个模态(例如文本)如何依据另一个模态(例如图像)的内容来分配注意力或关注重点。本申请示例中，跨模态注意力分布指的是视觉语言模型在生成预测结果时，如何关注样本中图像的相关部分。
示例性的，在视觉问答的应用场景中，视觉语言模型在处理图像时，基于跨模态注意力分布能够关注与描述文本或问题相关的图像区域。例如，视觉语言模型的任务是回答图像中的问题，视觉语言模型的图像输入为一张行驶在高速公路上的红色汽车，视觉语言模型的文本输入为一个问题“What is the color of the car？”。当视觉语言模型生成回答的第一个词“red”时，会计算对图像中所有视觉令牌(预设图像特征)的注意力权重，注意力权重表示每个视觉令牌对当前生成的词的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响第一个词“red”的生成。可选地，视觉语言模型可能会重点关注图像中表示汽车部分的视觉令牌。
需要说明的是，本申请实施例中，主要是针对对抗样本中的对抗图像进行净化操作，从而得到具有净化图像的净化样本。在未经特别说明的情况下，干净样本对应的跨模态注意力分布，可以等同理解为干净图像对应的跨模态注意力分布；对抗样本对应的跨模态注意力分布，可以等同理解为对抗图像对应的跨模态注意力分布；加噪样本对应的跨模态注意力分布，可以等同理解为加噪图像对应的跨模态注意力分布；净化样本对应的跨模态注意力分布，可以等同理解为净化图像对应的跨模态注意力分布。
图1示出了本申请一个实施例提供的计算机系统的架构示意图。该计算机系统可以包括：终端100和服务器200。
终端100可以是诸如手机、平板电脑、车载终端(车机)、可穿戴设备、个人计算机(Personal Computer，PC)、车载终端等电子设备。终端100中可以安装运行目标应用程序的客户端，该目标应用程序可以是支持样本净化的应用程序，本申请对此不作限定。另外，本申请对该目标应用程序的形式不作限定，包括但不限于安装在终端100中的应用程序(Application，App)、小程序、客户端等，还可以是网页形式。
服务器200可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云计算服务的云服务器、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工掌部图像识别平台等基础云计算服务的云服务器。服务器200可以是上述目标应用程序的后台服务器，用于为目标应用程序的客户端提供后台服务。
其中，云技术(Cloud technology)是指在广域网或局域网内将硬件、软件、网络等系列资源统一起来，实现数据的计算、储存、处理和共享的一种托管技术。云技术基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术、应用技术等的总称，可以组成资源池，按需所用，灵活便利。云计算技术将变成重要支撑。技术网络系统的后台服务需要大量的计算、存储资源，如视频网站、图片类网站和更多的门户网站。伴随着互联网行业的高度发展和应用，将来每个物品都有可能存在自己的识别标志，都需要传输到后台系统进行逻辑处理，不同程度级别的数据将会分开处理，各类行业数据皆需要强大的系统后盾支撑，只能通过云计算来实现。
在一些实施例中，上述服务器还可以实现为区块链系统中的节点。区块链(Blockchain)是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层。
终端100和服务器200之间可以通过网络进行通信，如有线或无线网络。
本申请实施例提供的样本净化方法，各步骤的执行主体可以是计算机设备，计算机设备是指具备数据计算、处理和存储能力的电子设备。以图1所示的计算机系统为例，可以由终端100执行样本净化方法，也可以由服务器200执行样本净化方法，或者由终端100和服务器200交互配合执行样本净化方法，本申请对此不作限定。
示例性的，图2示出了本申请一个示例性实施例提供的样本净化方法的示意图。该方法由计算机设备执行，该计算机设备可以是图1所示的终端100和/或服务器200。本实施例中，以计算机设备是服务器200为例进行说明。由服务器200执行样本净化方法。
相关技术中，通过对输入视觉语言模型的图像进行一些预处理操作，从而实现对抗样本的净化。例如，对图像进行平滑操作，即应用平滑滤波器减少图像噪声，从而降低对抗性攻击的影响；又例如，对图像进行尺度变化操作，即通过改变图像的尺度(例如缩放)来改变图像特征，减少对抗性攻击的有效性。然而，相关技术中针对对抗样本的净化主要依赖于传统的图像处理技术，导致针对对抗样本的净化效果并不理想，在实际应用中对抗样本仍然会对视觉语言模型的任务输出产生影响。
为了解决上述问题，本申请提出了一种样本净化方法，为了减少对抗样本中对抗攻击，使得视觉语言模型产生正确的预测结果，可以对输入视觉语言模型的对抗样本(主要是针对对抗样本中的对抗图像)进行净化，经过净化过程后得到净化样本(净化图像)。本申请实施例以跨模态注意力分布为净化过程的目标方向，引导对抗样本完成净化操作。可选地，通过向对抗样本中添加随机噪声，以引导对抗样本的对抗跨模态注意力分布接近干净样本的干净跨模态注意力分布，进而使视觉语言模型产生正确的预测结果。
在视觉问答的应用场景中，视觉语言模型可以回答关于图像的自然语言问题。示例性的，结合参考图2进行说明，干净样本10中包括干净图像和文本；对抗样本20中包括对抗图像和文本；加噪样本30中包括加噪图像和文本；净化样本40中包括净化图像和文本。其中，干净样本10、对抗样本20、加噪样本30和净化样本40的文本为一个自然语言问题的描述“这个男人在做什么？”。
可选地，将干净样本10输入视觉语言模型50，可以得到一个输出结果60，即得到正确的回答“击球”，视觉语言模型50在生成回答“击球”时，会计算干净样本10中干净图像的干净跨模态注意力分布。可选地，视觉语言模型50计算干净图像中所有视觉令牌的注意力权重，视觉语言模型50将干净图像分为若干个区域，每个区域的图像特征(或者多个区域对应的图像特征)可以表示为一个视觉令牌，注意力权重表示每个视觉令牌对回答“击球”的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响“击球”的生成。可选地，跨模态注意力分布图70是跨模态注意力分布的可视化表示，展示了视觉语言模型在处理任务时，对图像中不同视觉令牌的关注度。
在一些实施例中，干净样本10和对抗样本20的跨模态注意力分布具有显著差异，也可以理解为，干净图像和对抗图像的跨模态注意力分布具有显著差异。可以通过引导对抗样本20的对抗跨模态注意力分布接近干净样本10的干净跨模态注意力分布，从而实现对抗样本20的净化。但是在对抗净化的场景中，往往无法直接获取干净样本10，继而无法直接获取干净样本10的干净跨模态注意力分布。
在一种可能的实现的方式中，通过在对抗样本20的对抗图像上加入第一噪声，得到加噪样本30，相较于对抗样本20的对抗跨模态注意力分布，加噪样本30的加噪跨模态注意力分布更接近干净样本10对应的干净跨模态注意力分布(表现为加噪样本30对应的跨模态注意力分布图中注意力权重分布接近干净样本10对应的跨模态注意力分布图中注意力权重分布)，因此，以加噪样本30的加噪跨模态注意力分布为参考估计方向，引导对抗样本20的对抗跨模态注意力分布进行优化，可以认为是针对对抗样本20的净化操作，使其更接近干净样本10。可选地，通过引导对抗样本20对应的对抗跨模态注意力分布接近加噪样本30对应的加噪跨模态注意力分布，从而实现对抗样本20的净化。
在一些实施例中，将加噪样本30对应的加噪跨模态注意力分布作为目标，引导对抗样本20对应的对抗跨模态注意力分布接近加噪样本30对应的加噪跨模态注意力分布。可选地，以最小化注意力差异值为优化的目标，在对抗图像中添加第二噪声以生成净化图像，即针对对抗样本20进行净化操作并生成净化样本40。其中，注意力差异值为加噪样本30对应的加噪跨模态注意力分布和对抗样本20对应的对抗跨模态注意力分布之间的差异。视觉语言模型50基于净化样本40可以生成正确的回答“击球”。
在一些实施例中，基于以下步骤得到净化样本40：首先计算注意力差异值的范数，注意力差异值的范数用于衡量对抗样本20对应的对抗跨模态注意力分布和加噪样本30对应的加噪跨模态注意力分布之间的距离。其次，计算注意力差异值针对对抗图像中像素的梯度，该梯度用于指示在当前对抗样本20下，如何添加第二噪声以最小化注意力差异值。然后，基于符号函数确定梯度的梯度方向，梯度方向用于指示对抗样本20中对抗图像添加第二噪声的方向，即在对抗图像中的像素上添加第二噪声，对抗图像中的像素朝梯度方向进行调整，以减小注意力差异值。可选地，梯度作为符号函数的输入，当梯度为正值时，符号函数的输出为1，即控制对抗样本20按照梯度方向增加对抗图像中的像素值；当梯度为负值时，符号函数的输出为-1，即控制对抗样本20按照梯度方向减少对抗图像中的像素值；当梯度为零时，符号函数的输出为0，即控制对抗样本按照梯度方向保持对抗图像中的像素值不变(基本上像素值都会变化，几乎不存在不变的情况)。可选地，按照梯度方向增加或减少对抗图像中的像素值，以得到添加第二噪声后的净化图像。
在一些实施例中，控制添加到对抗图像中的第二噪声的大小。可选地，梯度中不仅包含梯度方向的信息，同时还包括强度信息。其中，梯度的强度信息与添加第二噪声的大小呈现正相关关系。可选地，以像素粒度划分对抗样本20中的对抗图像，基于梯度方向在对抗图像的不同像素位置添加不同扰动程度(不同大小)的第二噪声。可选地，对抗图像中梯度较大的像素应该受到更大的扰动，即添加相对较大的噪声；对抗图像中梯度较小的像素应该有较小的扰动，即添加相对较小的噪声。可选地，获取梯度最大值和梯度最小值，基于梯度最大值和梯度最小值得到梯度标准值，梯度标准值是将梯度进行归一化处理后的梯度值，将梯度归一化到[0，1]。在一些实施例中，由于梯度标准值差异较大，可以基于梯度标准值的平均值表示梯度标准值。根据梯度标准值以及梯度标准值的平均值确定第二噪声的扰动幅度，扰动幅度中包括用于确定第二噪声大小的扰动因子，基于扰动因子确定对抗图像中每个像素添加的第二噪声大小。
在一些实施例中，通过上述针对对抗图像的净化操作得到净化图像，即针对对抗样本20的净化操作得到净化样本40。可选地，将净化样本40输入视觉语言模型50，视觉语言模型50基于净化样本40可以得到正确的输出结果“击球”。
应理解的是，在本申请实施例中，加噪跨模态注意力分布也被称为加噪图像的至少一个加噪图像特征和文本的至少一个文本特征之间的第一相关性；对抗跨模态注意力分布也被称为对抗图像的至少一个对抗图像特征和文本的至少一个文本特征之间的第二相关性。注意力差异值也被称为相关性差异值。
图3是本申请一个示例性实施例提供的样本净化方法的流程图。该方法可以由计算机设备执行，计算机设备可以是图1中的终端100或服务器200。该方法包括以下步骤。
步骤210：获取对抗样本；
其中，对抗样本是对干净样本执行对抗攻击后生成的样本。可选地，干净样本中包括干净图像和文本。对抗样本中包括对抗图像和文本。
在一些实施例中，干净样本是指未经任何修改或扰动的原始样本。对抗样本是对干净样本执行对抗攻击后生成的样本。其中，对抗攻击包括对干净样本进行微小修改或扰动。对抗样本中的修改或者扰动不易察觉，但会导致视觉语言模型做出错误的预测并生成错误的预测结果(输出结果)。
在一些实施例中，对抗样本可以用来评估视觉语言模型在面对微小扰动时的表现。通过引入对抗样本可以测试视觉语言模型在实际应用中的鲁棒性，是否容易受到攻击或误导。对抗样本可能被用作攻击工具，试图通过干扰视觉语言模型来获取竞争优势或进行恶意活动。因此，对抗样本的净化操作尤为重要。
示例性的，有一张干净图像显示一只猫，在视觉问答场景下，视觉语言模型正确地将其分类为“猫”。通过对该干净图像应用对抗攻击，例如基于FGSM(Fast Gradient Sign Method)方法对干净图像进行对抗攻击，可以生成一个对抗图像，生成的对抗图像看起来仍然像一只猫，但视觉语言模型可能会错误地将其分类为“狗”或其他类别。
在一些实施例中，为了减少对抗样本中对抗攻击，使得视觉语言模型产生正确的预测结果，针对对抗样本中的对抗图像进行净化操作，以得到净化图像。
步骤220：获取向对抗图像添加第一噪声后得到的加噪图像；
其中，加噪图像是针对对抗图像执行第一扰动后的图像。第一噪声指的是在对抗图像上添加的随机噪声。第一噪声是随机的或按照某种规则生成的，目的是扰动对抗图像的像素值。可选地，第一噪声可以是高斯噪声、均匀噪声或其他类型的随机噪声。
在一些实施例中，视觉语言模型是一种结合了视觉(图像)和语言(文本)的多模态模型，可以接收图像和文本输入并生成任务输出。视觉语言模型能够理解并处理图像和文本之间的关联，用于处理多种任务。可选地，视觉语言模型包括深度学习神经网络模型(DeepNeuralNetwork，DNN)、卷积神经网络模型(ConvolutionalNeuralNetwork，CNN)、Transformer模型、对比语言-图像预训练模型(Contrastive Language-Image Pre-Training，CLIP)、极限学习机模型(ExtremeLearningMachine，ELM)或其他的神经网络模型中的至少一种，但不限于此，本申请实施例对此不作具体限定。
在一些实施例中，向对抗图像中添加的第一噪声为均匀噪声。加噪图像可以表示为：
其中，表示加噪图像，x′i表示对抗图像，α表示第一噪声，即第一噪声基于在第一分布区间[-α∞,α∞]进行伪随机采样得到的第一噪声值确定，第一分布区间[-α∞,α∞]为均匀分布的区间范围。R(x′i,α∞)表示在对抗图像x′i上添加无穷范数限制为α∞的随机噪声。
在一种可能实现的方式中，通过计算机程序实现第一噪声的添加。可选地，计算机设备中安装有添加噪声的程序，利用编程语言(如Python中的NumPy、OpenCV库等)可以生成均匀分布的第一噪声，并将第一噪声添加到对抗图像中。可选地，计算机设备中安装有添加噪声的程序使用OpenCV库读取要添加第一噪声的对抗图像，使用随机数生成器生成均匀分布的随机数，这些随机数将作为第一噪声的噪声值。可选地，根据需要的噪声强度，调整生成的随机噪声(第一噪声)的值。其中，噪声强度可以通过调整随机数生成器的范围来控制。然后，将随机数生成器生成的第一噪声的噪声值添加到对抗图像的像素中。
在一种可能实现的方式中，通过噪声模型实现第一噪声的添加，例如使用生成对抗网络(GANs)、变分自编码器(VAEs)或其他深度学习模型来生成第一噪声，然后将第一噪声添加到对抗图像上。示例性的，以生成对抗网络为对抗图像添加第一噪声以生成加噪图像为例进行说明。生成对抗网络接收一个随机噪声(第一噪声)作为输入。随机噪声可以是从均匀分布中采样得到的，生成对抗网络根据随机噪声输出一个均匀分布的噪声图像，然后生成对抗网络进行加噪处理，将生成的噪声图像添加到对抗图像中，即将噪声图像的像素值叠加到对抗图像以得到加噪图像。以上仅为示例性的举例说明，本申请对添加噪声的方式不作限定。
步骤230：确定加噪图像的至少一个加噪图像特征和文本的至少一个文本特征之间的第一相关性；
换句话讲，可以计算加噪跨模态注意力分布。其中，加噪跨模态注意力分布用于描述加噪图像和文本的注意力分布。
在一些实施例中，干净样本和对抗性样本的跨模态注意力分布具有显著差异，也可以理解为，干净图像的干净跨模态注意力分布和对抗图像的对抗跨模态注意力分布具有显著差异。可以通过引导对抗图像的对抗跨模态注意力分布接近干净图像的干净跨模态注意力分布，从而实现对抗图像的净化。但是在对抗净化的场景中，往往无法直接获取干净图像，继而无法直接获取干净图像的干净跨模态注意力分布。
在一种可能的实现的方式中，通过在对抗图像上加入第一噪声，得到加噪图像，相较于对抗图像的对抗跨模态注意力分布，加噪图像的加噪跨模态注意力分布更接近干净图像的干净跨模态注意力分布，因此，以加噪图像的加噪跨模态注意力分布为参考估计方向，引导对抗图像的对抗跨模态注意力分布进行优化，可以认为是针对对抗图像的净化操作，使其更接近干净图像。可选地，通过引导对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布，从而实现对抗图像的净化。
在一些实施例中，计算加噪图像对应的加噪跨模态注意力分布。跨模态注意力分布指的是视觉语言模型在处理多模态数据时的一种注意力分布。加噪跨模态注意力分布指的是视觉语言模型基于加噪样本生成预测结果时，如何关注加噪样本中加噪图像的注意力大小(或注意力权重)。
可选地，视觉语言模型计算加噪图像中所有视觉令牌的注意力权重，其中，每个视觉令牌对应一个加噪图像特征，视觉语言模型将加噪图像分为若干个区域，每个区域对应一个区域图像特征，可以表示将多个区域的区域图像特征融合为至少一个加噪图像特征，或者说融合为至少一个视觉令牌，其中，每个区域图像特征对应一个视觉令牌，或者多个区域对应的多个图像特征可以表示为一个视觉令牌，注意力权重表示每个视觉令牌对视觉语言模型生成预测结果的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响预测结果的生成。
示例性的，视觉语言模型的任务是回答加噪图像中的问题，视觉语言模型的加噪图像输入为一张行驶在高速公路上的汽车，视觉语言模型的文本输入为一个问题“What is the color of the car？”。当视觉语言模型生成预测结果时，会计算对加噪图像中所有视觉令牌的注意力权重，注意力权重表示每个视觉令牌对当前预测结果的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响预测结果的生成。可选地，视觉语言模型可能会重点关注加噪图像中表示汽车部分的视觉令牌。
步骤240：确定对抗图像的至少一个对抗图像特征和文本的至少一个文本特征之间的第二相关性；
换句话讲，计算对抗图像对应的对抗跨模态注意力分布。其中，对抗跨模态注意力分布用于描述对抗图像和文本的注意力分布。
应理解的是，关于对抗跨模态注意力分布的计算方式可以参考加噪跨模态注意力分布的计算方式，本申请实施例对此不再赘述。
步骤250：基于第一相关性和第二相关性向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像。
其中，净化图像为向对抗图像执行第二扰动后的图像。第二噪声指的是在对抗图像上添加的随机噪声。第二噪声是随机的或按照某种规则生成的，目的是扰动对抗图像的像素值。可选地，第二噪声可以是高斯噪声、均匀噪声或其他类型的随机噪声。
在一些实施例中，将加噪图像的加噪跨模态注意力分布(即第一相关性)作为目标，引导对抗图像的对抗跨模态注意力分布(即第二相关性)接近加噪图像对应的加噪跨模态注意力分布(即第一相关性)。
可选地，通过在对抗图像中继续添加随机噪声的方式，可以引导第二相关性接近第一相关性，进而生成净化图像。可选地，在对抗图像中添加的随机噪声是在范围分布的区间范围中随机采样得到的噪声值。以最小化第一相关性和第二相关性之间的差异为优化目标，在对抗图像中添加第二噪声以生成净化图像。
在一些实施例中，视觉语言模型基于净化图像可以得到正确的预测结果。
以干净图像为参照，净化图像相较于干净图像的扰动程度可能大于对抗图像相较于干净图像的扰动程度，视觉语言模型基于对抗图像得到错误的预测结果，基于净化图像却可以得到正确的结果。尽管净化图像在视觉上可能更嘈杂，但是净化图像的净化跨模态注意力分布(即净化图像的至少一个净化图像特征和文本的至少一个文本特征之间的相关性)可能更接近于干净图像的干净跨模态注意力分布(即干净图像的至少一个干净图像特征和文本的至少一个文本特征之间的相关性)，因为净化图像中添加的第二噪声调整了视觉语言模型的注意力分布，增强视觉语言模型对净化图像中与预测任务相关特征的关注，因此视觉语言模型对净化图像的预测结果更接近对干净图像的预测结果。
在一些实施例中，干净样本包括干净图像和文本；所述方法还包括：确定干净图像的至少一个干净图像特征和文本的至少一个文本特征之间的第三相关性；其中，第一相关性相对于第二相关性更接近于第三相关性。
换句话讲，加噪跨模态注意力分布(即第一相关性)相对于对抗跨模态注意力分布(即第一相关性)更接近于干净跨模态注意力分布(即第三相关性)，使得以加噪图像的加噪跨模态注意力分布为优化目标，向对抗图像中加入随机噪声，进而使得对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布，以完成对对抗图像的净化并得到净化图像。
应理解的是，关于第三相关性的计算方式可以参考第一相关性、第二相关性的计算方式，本申请对此不再赘述。
综上所述，本实施例提供的方法，视觉语言模型基于对抗样本会产生错误的预测，通过对对抗样本中的对抗图像执行净化操作后得到净化图像，进而可以得到净化样本。具体地，通过对对抗图像进行随机加噪处理得到加噪图像，由于加噪图像的加噪跨模态注意力分布(即加噪图像的至少一个加噪图像特征和文本的至少一个文本特征之间的第一相关性)非常接近干净图像对应的跨模态注意力分布(即干净图像的至少一个干净图像特征和文本的至少一个文本特征之间的相关性)，因此以加噪图像的加噪跨模态注意力分布为优化目标，向对抗图像中加入随机噪声，进而使得对抗图像的对抗跨模态注意力分布(即对抗图像的至少一个对抗图像特征和文本的至少一个文本特征之间的第二相关性)接近加噪图像的加噪跨模态注意力分布，以完成对对抗图像的净化并得到净化图像。进一步地，本申请以跨模态注意力分布为引导，通过在对抗样本中添加噪声以实现对抗图像的净化，视觉语言模型基于净化图像(或者净化样本)可以得到正确的预测，这种净化方式增强了视觉语言模型的鲁棒性。进一步地，以跨模态注意力分布为估计方式引导对抗图像添加噪声的方式，不仅是在处理对抗图像，同时也在优化跨模态注意力分布本身，这种优化使得注意力分布能够更准确地聚焦于图像中的关键信息，从而提高视觉语言模型输出预测结果的准确性。
计算加噪图像对应的跨模态注意力分布
在一些实施例中，相较于对抗图像的对抗跨模态注意力分布，加噪图像的加噪跨模态注意力分布更接近干净图像的干净跨模态注意力分布。可选地，通过引导对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布，从而实现对抗图像的净化。
在一些实施例中，通过视觉语言模型计算所述加噪图像对应的加噪跨模态注意力分布，当然，也可以通过其他模型或者是类似于视觉语言模型的其他模型计算所述加噪图像对应的加噪跨模态注意力分布，本申请实施例对此不做限制。
下面以通过视觉语言模型计算加噪跨模态注意力分布为例，对加噪跨模态注意力分布的计算过程进行阐述：
图4A示出了本申请一个示例性实施例提供的样本净化方法的流程图，上述步骤230可以替换为步骤231、步骤232和步骤233。
步骤231：基于视觉编码器将加噪图像划分为多个区域，并将每个区域编码为区域图像特征；通过视觉语言模型对齐多个区域的区域图像特征，得到至少一个加噪图像特征；
在一些实施例中，视觉语言模型中包括视觉编码器，视觉编码器用于从加噪图像中提取特征向量，将加噪图像编码为多个区域的区域图像特征(或称视觉特征)。可选地，视觉编码器接收加噪图像作为输入，并将加噪图像分割为若干个区域，每个区域的大小可以是相同的，之后对每个区域的图像块进行特征提取，每个区域的图像块可以编码为一个图像特征(即区域图像特征)。可选地，通过视觉语言模型中的语言对齐模块将多个区域图像特征对齐后得到至少一个加噪图像特征，其中，每个加噪图像特征对应一个视觉令牌，或者说，每个视觉令牌用于至少一个加噪图像特征，每个视觉令牌或者加噪图像特征可以是若干个区域图像特征的融合表示。例如，加噪图像的一个区域对应一个区域图像特征，一个视觉令牌可以是若干个图像特征融合后的特征表示。示例性的，视觉语言模型中的视觉编码器将加噪图像划分为16*16＝256个区域，并提取256个图像特征，由语言对齐模块将其处理为32个视觉令牌，每个视觉令牌都可以是256个图像特征的融合表示。
在一种可能实现的方式中，一个区域图像特征可以看作加噪图像的一个视觉令牌或者加噪图像特征，每个视觉令牌用于指示加噪图像的一个区域图像特征。或者，多个区域图像特征可以看作加噪图像的一个视觉令牌或者加噪图像特征，每个视觉令牌用于指示加噪图像的多个区域的图像特征，例如，加噪图像划分为16×16个区域，每个区域对应一个区域图像特征，视觉语言模型将加噪图像处理后得到32个视觉令牌，每个视觉令牌可以是所有区域的图像特征的融合表示。
下面的举例以一个视觉令牌指示图像的一个区域的图像特征进行说明。
示例性的，假设一张加噪图像，加噪图像展示一个在草地上击球的男人，将该加噪图像输入视觉语言模型，视觉语言模型中的视觉编码器将该加噪图像分割为若干个固定大小的区域，例如，将加噪图像划分为32个区域。可选地，对于每个区域，视觉编码器提取该区域的特征得到图像特征。每个区域的图像特征被编码为一个视觉令牌，每个视觉令牌代表了加噪图像中相应区域的视觉信息。例如，区域1表示加噪图像中的草地，视觉令牌1用于指示加噪图像中表示草地部分的图像特征；区域2表示加噪图像中的草地和男人的脚部，视觉令牌2用于指示加噪图像中表示草地和男人的脚部部分的图像特征；区域3表示加噪图像中男人的头部，视觉令牌3用于指示加噪图像中表示男人的头部部分的图像特征；区域4表示加噪图像中男人的手臂和球杆，视觉令牌4用于指示加噪图像中表示男人的手臂和球杆部分的图像特征；区域5表示加噪图像中的球，视觉令牌5用于指示加噪图像中表示球部分的图像特征；区域6表示加噪图像中球杆的底部和草地，视觉令牌6用于指示加噪图像中表示球杆的底部和草地的图像特征……诸如此类。
步骤232：基于语言模型分词器将文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；通过视觉语言模型对齐多个文本单元的文本单元特征，得到至少一个文本特征；
在一些实施例中，视觉语言模型中包括语言模型分词器，语言模型分词器用于从文本中提取特征向量，将文本编码为多个文本单元的文本单元特征。可选地，语言模型分词器接收文本作为输入，将输入的文本分解成更小的文本单元(即)，可选地，文本单元包括单词、子词或字符串。
下面以将文本分解为若个子词为例对文本特征的得到过程进行说明：
在一些实施例中，通过语言模型分词器可以每个子词进行编码，得到该子词的子词特征(即文本单元特征，也可以被理解为是一个特征向量)，接着将每个子词特征作为一个文本特征，或者，将多个子词特征作为一个文本特征。
在一些实施例中，通过语言模型分词器可以每个子词进行编码，得到该子词的子词特征(即文本单元特征，也可以被理解为是一个特征向量)，并且视觉语言模型还可以保留每个子词在文本中的顺序信息，在每个子词的子词特征上添加一个位置编码，得到新的子词特征，位置编码是根据子词在文本中的位置计算的值，接着将每个新子词特征作为一个文本特征，或者，将多个新子词特征作为一个文本特征。
下面的举例以一个语言令牌指示文本的一个子词的文本特征进行说明。
示例性的，假设针对上述加噪图像的文本为一个问题描述，文本指示的问题描述为“这个男人在做什么？”。将该文本输入视觉语言模型，视觉语言模型中的语言模型分词器将该文本分解成若干个子词，例如将“这个男人在做什么？”分解为以下子词：[“这个”，“男人”，“在”，“做”，“什么”，“？”]。对于每个子词，语言模型分词器提取相应的特征向量，为了保留文本中子词的顺序信息，每个子词的特征向量可能会加上一个位置编码，位置编码是根据子词在文本中的位置计算的值，视觉语言模型中的语言对齐模块将每个子词的特征向量和位置编码对齐，生成语言令牌，每个语言令牌代表了文本中相应区域的子词的文本特征。例如，子词1表示文本中的“这个”，语言令牌1用于指示子词“这个”的文本特征；子词2表示文本中的“男人”，语言令牌2用于指示子词“男人”的文本特征；子词3表示文本中的“在”，语言令牌3用于指示子词“在”的文本特征；子词4表示文本中的“做”，语言令牌4用于指示子词“做”的文本特征；子词5表示文本中的“什么”，语言令牌5用于指示子词“什么”的文本特征……诸如此类。
步骤233：通过视觉语言模型计算生成加噪预测结果时的第一相关性。
其中，加噪预测结果为视觉语言模型针对加噪图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个视觉令牌对应一个加噪图像特征；每个语言令牌对应一个文本特征。
在一些实施例中，加噪图像中所有的视觉令牌及其对应的注意力权重被整合在一起，形成了对加噪图像的整体表示，视觉语言模型根据整合后的视觉令牌和对应的语言令牌的描述，生成加噪图像对应的加噪预测结果，该加噪预测结果是视觉语言模型根据语言令牌(文本对应的问题描述)对整合后的视觉令牌(加噪图像)的预测结果。
在一些实施例中，视觉语言模型基于加噪图像生成加噪预测结果时，会计算加噪图像中所有视觉令牌的注意力权重，注意力权重表示每个视觉令牌对加噪预测结果的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响加噪预测结果的生成。可选地，视觉语言模型在生成加噪预测结果时会重点关注加噪图像中注意力权重大于第一预设权重阈值的视觉令牌，并忽略加噪图像中注意力权重小于第二预设权重阈值的视觉令牌。
在一些实施例中，可以通过加噪跨模态注意力分布图表示加噪跨模态注意力分布，即，加噪跨模态注意力分布图为加噪跨模态注意力分布的可视化表示。加噪跨模态注意力分布图具有若干维度，例如，加噪跨模态注意力分布图可以表示视觉语言模型的层数和视觉令牌数目。加噪跨模态注意力分布图可以更清楚的表示加噪图像中视觉令牌的注意力分布。可选地，在加噪跨模态注意力分布图中，可以将加噪图像中注意力权重大于第一预设权重阈值的视觉令牌表示为第一形式，将加噪图像中注意力权重小于第二预设权重阈值的视觉令牌表示为第二形式，将加噪图像中注意力权重小于第一预设权重阈值且大于第二预设权重阈值的视觉令牌表示为第三形式。可选地，第一形式的视觉显著程度高于第三形式，第三形式的视觉显著程度高于第二形式。本申请对此不作限定。
示例性的，假设向视觉语言模型输入一张加噪图像，加噪图像展示一个在草地上击球的男人，加噪图像对应的文本为一个问题描述，文本指示的问题描述为“这个男人在做什么？”。例如，加噪图像中的区域1表示草地；加噪图像中的区域2表示草地和男人的脚部；加噪图像中的区域3表示男人的头部；加噪图像中的区域4表示男人的手臂和球杆；加噪图像中的区域5表示球；加噪图像中的区域6表示天空等等；文本的子词1表示“这个”；文本的子词2表示“男人”；文本的子词3表示“在”；文本的子词4表示“做”；文本的子词5表示“什么”等等。视觉语言模型在生成针对加噪图像的加噪预测结果时，会计算文本中每个子词对加噪图像中每个区域(视觉令牌)的注意力权重，这反映了视觉语言模型在生成加噪预测结果时，对视觉令牌的关注程度。可选地，视觉语言模型在生成加噪预测结果时，可能会关注加噪图像中注意力权重比较高的区域，例如，加噪图像中区域1对应的视觉令牌的注意力权重为0.05，加噪图像中区域2对应的视觉令牌的注意力权重为0.1，加噪图像中区域3对应的视觉令牌的注意力权重为0.1，加噪图像中区域4对应的视觉令牌的注意力权重为0.3，加噪图像中区域5对应的视觉令牌的注意力权重为0.2，加噪图像中区域6对应的视觉令牌的注意力权重为0.01等。例如，视觉语言模型在生成加噪预测结果时会重点关注加噪图像中的区域4指示的“男人的手臂和球杆”和区域5指示的“球”，并忽略加噪图像中的区域6指示的“天空”，视觉语言模型结合文本对应的问题描述得到加噪预测结果为“抛球”、“击球”或其它结果。
应理解的是，本申请实施例对视觉语言模型的结构不做限制，比如，上述视觉编码器和语言模型分词器可以被一个整体模型或者模块所替代，以实现视觉编码器和语言模型分词器的功能。
综上所述，本申请实施例提供的方法，通过视觉编码器将加噪图像编码为若干区域图像特征，每个视觉令牌代表图像的一个或多个区域的图像特征(一组或多组特征向量)，这种分解可以将加噪图像的复杂信息简化为一组结构化的特征，有助于视觉语言模型处理和理解加噪图像中的不同部分。通过语言模型分词器将文本编码为若干组文本单元特征，即特征向量，每个语言令牌代表文本的一个或多个子词，这种分解将文本转化为结构化的特征，有助于视觉语言模型理解文本的语义。通过视觉语言模型计算加噪跨模态注意力分布，允许视觉语言模型在生成加噪预测结果时关注加噪图像和文本之间的相关性，帮助视觉语言模型确定加噪图像中哪些图像区域与文本中的哪个子词最相关，从而更准确地理解和解释图像和文本的关系，从而生成较为准确和相关的预测结果。
计算对抗图像对应的跨模态注意力分布
可选地，通过引导对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布，从而实现对抗图像的净化。
在一些实施例中，通过视觉语言模型计算所述对抗图像对应的对抗跨模态注意力分布，当然，也可以通过其他模型或者是类似于视觉语言模型的其他模型计算所述对抗图像对应的对抗跨模态注意力分布，本申请实施例对此不做限制。
下面以通过视觉语言模型计算对抗跨模态注意力分布为例，对对抗跨模态注意力分布的计算过程进行阐述：
图4B示出了本申请一个示例性实施例提供的样本净化方法的流程图，上述步骤240可以替换为步骤241、步骤242和步骤243。
步骤241：基于视觉编码器将对抗图像划分为多个区域，并将每个区域编码为区域图像特征；通过视觉语言模型对齐多个区域的区域图像特征，得到至少一个对抗图像特征；
在一些实施例中，视觉语言模型中包括视觉编码器，视觉编码器用于从对抗图像中提取特征向量，将对抗图像编码为多个区域的区域图像特征(或称视觉特征)。可选地，视觉编码器接收对抗图像作为输入，并将对抗图像分割为若干个区域，每个区域的大小可以是相同的，之后对每个区域的图像块进行特征提取，每个区域的图像块可以编码为一个图像特征(即区域图像特征)。可选地，通过视觉语言模型中的语言对齐模块将多个区域图像特征对齐后得到至少一个对抗图像特征，其中，每个对抗图像特征对应一个视觉令牌，或者说，每个视觉令牌用于至少一个对抗图像特征，每个视觉令牌或者对抗图像特征可以是若干个区域图像特征的融合表示。例如，对抗图像的一个区域对应一个区域图像特征，一个视觉令牌可以是若干个图像特征融合后的特征表示。示例性的，视觉语言模型中的视觉编码器将对抗图像划分为16*16＝256个区域，并提取256个图像特征，由语言对齐模块将其处理为32个视觉令牌，每个视觉令牌都可以是256个图像特征的融合表示。
在一种可能实现的方式中，一个区域图像特征可以看作对抗图像的一个视觉令牌或者对抗图像特征，每个视觉令牌用于指示对抗图像的一个区域图像特征。或者，多个区域图像特征可以看作对抗图像的一个视觉令牌或者对抗图像特征，每个视觉令牌用于指示对抗图像的多个区域的图像特征，例如，对抗图像划分为16×16个区域，每个区域对应一个区域图像特征，视觉语言模型将对抗图像处理后得到32个视觉令牌，每个视觉令牌可以是所有区域的图像特征的融合表示。
步骤242：基于语言模型分词器将文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；通过视觉语言模型对齐多个文本单元的文本单元特征，得到至少一个文本特征；
关于步骤242的解释说明可以参考步骤S232，本申请实施例对此不再赘述。
步骤243：通过视觉语言模型计算生成对抗测结果时的第二相关性。
其中，对抗预测结果为视觉语言模型针对对抗图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个视觉令牌对应一个对抗图像特征；每个语言令牌对应一个文本特征。
其中，对抗预测结果为视觉语言模型针对对抗图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果。
在一些实施例中，对抗图像中所有的视觉令牌及其对应的注意力权重被整合在一起，形成了对对抗图像的整体表示，视觉语言模型根据整合后的视觉令牌和对应的语言令牌的描述，生成对抗图像对应的对抗预测结果，该对抗预测结果是视觉语言模型根据语言令牌(文本对应的问题描述)对整合后的视觉令牌(对抗图像)的预测结果。
在一些实施例中，视觉语言模型基于对抗图像生成对抗预测结果时，会计算对抗图像中所有视觉令牌的注意力权重，注意力权重表示每个视觉令牌对对抗预测结果的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响对抗预测结果的生成。可选地，视觉语言模型在生成对抗预测结果时会重点关注对抗图像中注意力权重大于第一预设权重阈值的视觉令牌，并忽略对抗图像中注意力权重小于第二预设权重阈值的视觉令牌。
在一些实施例中，可以通过对抗跨模态注意力分布图表示对抗跨模态注意力分布，即，对抗跨模态注意力分布图为对抗跨模态注意力分布的可视化表示。对抗跨模态注意力分布图具有若干维度，例如，对抗跨模态注意力分布图可以表示视觉语言模型的层数和视觉令牌数目。对抗跨模态注意力分布图可以更清楚的表示对抗图像中视觉令牌的注意力分布。可选地，在对抗跨模态注意力分布图中，可以将对抗图像中注意力权重大于第一预设权重阈值的视觉令牌表示为第一形式，将对抗图像中注意力权重小于第二预设权重阈值的视觉令牌表示为第二形式，将对抗图像中注意力权重小于第一预设权重阈值且大于第二预设权重阈值的视觉令牌表示为第三形式。可选地，第一形式的视觉显著程度高于第三形式，第三形式的视觉显著程度高于第二形式。本申请对此不作限定。
综上所述，本申请实施例提供的方法，通过视觉编码器将对抗图像编码为若干区域图像特征，每个视觉令牌代表图像的一个或多个区域的图像特征(一组或多组特征向量)，这种分解可以将对抗图像的复杂信息简化为一组结构化的特征，有助于视觉语言模型处理和理解对抗图像中的不同部分。通过语言模型分词器将文本编码为若干组文本单元特征，即特征向量，每个语言令牌代表文本的一个或多个子词，这种分解将文本转化为结构化的特征，有助于视觉语言模型理解文本的语义。通过视觉语言模型计算加噪跨模态注意力分布，允许视觉语言模型在生成对抗预测结果时关注对抗图像和文本之间的相关性，帮助视觉语言模型确定对抗图像中哪些图像区域与文本中的哪个子词最相关，从而更准确地理解和解释图像和文本的关系，从而生成较为准确和相关的预测结果。
以最小化相关性差异为目标向对抗图像添加第二噪声
在一些实施例中，将第一相关性作为目标，引导第二相关性接近第一相关性。可选地，以最小化第一相关性和第二相关性之间的差异值为目标，向对抗图像中添加第二噪声，以生成净化图像。
图5示出了本申请一个示例性实施例提供的样本净化方法的流程图，上述步骤250可以替换为步骤251。
步骤251：以最小化相关性差异为目标，向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像。
其中，相关性差异为第一相关性和第二相关性之间的差异。其中，该相关性差异也可以被称为注意力差异或者注意力值，其表示加噪跨模态注意力分布和对抗跨模态注意力分布之间的差异。
在一些实施例中，加噪跨模态注意力分布是加噪图像对应的跨模态注意力分布，用于描述加噪图像和文本的注意力分布。对抗跨模态注意力分布是对抗图像对应的跨模态注意力分布，用于描述对抗图像和文本的注意力分布。可选地，基于加噪图像中的视觉令牌和语言令牌计算加噪图像对应的加噪跨模态注意力分布，基于对抗图像中的视觉令牌和语言令牌计算对抗图像对应的对抗跨模态注意力分布。
在一些实施例中，基于对抗跨模态注意力分布和加噪跨模态注意力分布可以得到注意力差异值，注意力差异值为加噪跨模态注意力分布和对抗跨模态注意力分布之间的差异。可选地，以最小化注意力差异值为目标，向对抗图像中添加第二噪声。
可选地，注意力差异值可以表示为：
ΔA＝A(x′i,xt,f)-A(R(x′i,α∞),xt,f)
其中，ΔA表示注意力差异值，x′i表示对抗图像，R(x′i,α∞)表示加噪图像，即在对抗图像x′i上添加第一噪声α∞得到的加噪图像，xt表示文本，f表示视觉语言模型，A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，A(R(x′i,α∞),xt,f)表示加噪图像对应的加噪跨模态注意力分布。
在一些实施例中，加噪图像对应的加噪跨模态注意力分布接近干净图像对应的干净跨模态注意力分布，因此，可以将加噪跨模态注意力分布可以认为是净化的参照点。可选地，通过引导对抗跨模态注意力分布接近加噪跨模态注意力分布，即最小化对抗跨模态注意力分布和加噪跨模态注意力分布之间的差异(注意力差异值)实现对抗图像的净化。在一种可能实现的方式中，通过在对抗图像中添加第二噪声的方式可以影响对抗跨模态注意力分布中视觉令牌的注意力分布(或理解为注意力权重)，从而实现对抗图像的净化并得到净化图像。
可选地，净化图像可以表示为：
其中，表示净化图像，x′i表示对抗图像，R(x′′,α∞)表示加噪图像，xt表示文本，f表示视觉语言模型，β表示第二噪声，即第二噪声基于在第二分布区间[0,β∞]进行伪随机采样得到的第二噪声值确定，第二分布区间[0,β∞]为均匀分布的区间范围。A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，A(R(x′i,α∞),xt,f)表示加噪图像对应的加噪跨模态注意力分布，A(x′i,xt,f)-A(R(x′i,α∞),xt,f)表示注意力差异值，表示最小化注意力差异值。
在一些实施例中，以最小化注意力差异值为目标，向对抗图像中添加第二噪声得到净化图像。可选地，向对抗图像中添加的第二噪声为均匀噪声。
在一些实施例中，将向对抗图像添加第二噪声后的图像输入视觉语言模型进行预测。可选地，基于视觉语言模型针对添加第二噪声后的图像的预测结果判断该图像是否属于净化图像：若视觉语言模型针对该图像的预测结果为正确预测结果时(和干净图像的预测结果相同)，可以判断该图像为净化图像；若视觉语言模型针对该图像的预测结果为错误预测结果时(和干净图像的预测结果不同)，则基于加噪跨模态注意力分布继续向对抗图像添加随机噪声，以调整对抗图像中视觉令牌的注意力分布，直至视觉语言模型的预测结果为正确预测结果。
在一种可能实现的方式中，在视觉语言模型中设置一个预设问答分数阈值，基于预设问答分数阈值判断视觉语言模型的预测结果是否为正确预测结果。可选地，当视觉语言模型针对向对抗图像添加第二噪声后的图像的问答分数大于预设问答分数阈值时，视觉语言模型的预测结果为正确预测结果，此时的图像为净化图像；当视觉语言模型针对向对抗图像添加第二噪声后的图像的问答分数小于预设问答分数阈值时，视觉语言模型的预测结果为错误预测结果，则基于加噪跨模态注意力分布继续向对抗图像添加随机噪声，直至视觉语言模型输出正确预测结果。本申请对预设问答分数阈值的具体数值不作限定。
在一些可实现方式中，所述基于所述加噪跨模态注意力分布和所述对抗跨模态注意力分布向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像，包括：以最小化目标差异值为目标，向所述对抗图像添加第二噪声，得到所述净化图像；其中，所述目标差异值是注意力差异值与一个预设系数的乘积或者之和，注意力差异值为所述加噪跨模态注意力分布和所述对抗跨模态注意力分布之间的差异。总之，在优化目标设计时，只要可以使得对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布即可，本申请实施例对优化目标不做限制。
本申请实施例中，通过计算对抗图像对应的对抗跨模态注意力分布和加噪图像对应的加噪跨模态注意力分布，以最小化对抗跨模态注意力分布和加噪跨模态注意力分布之间的差值(注意力差异值)为目标，向对抗图像中添加第二噪声，这种以跨模态注意力分布为估计方向的引导方式可以得到净化图像，这种方式不仅是在处理对抗图像的噪声，同时也在优化跨模态注意力分布机制本身，使得注意力机制能够更准确地聚焦于对抗图像中的关键信息，忽略无关或干扰信息，从而提高跨模态任务的准确性和效率。
控制向对抗图像添加第二噪声的噪声方向
在一些实施例中，以最小化对抗跨模态注意力分布和加噪跨模态注意力分布之间的差异值为目标，向对抗图像中添加第二噪声以生成净化图像。可选地，通过控制向对抗图像添加第二噪声的噪声方向生成净化图像。换句话讲，以最小化相关性差异为目标，向对抗图像中添加第二噪声以生成净化图像。可选地，通过控制向对抗图像添加第二噪声的噪声方向生成净化图像。
图6示出了本申请一个示例性实施例提供的样本净化方法的流程图，上述步骤251可以替换为步骤251a、步骤251b和步骤251c。
步骤251a：计算相关性差异的范数；
其中，所述范数用于衡量所述第一相关性和所述第二相关性之间的距离。
在一些实施例中，对第一相关性和第二相关性均可以表示为矩阵，范数用于衡量对第一相关性和第二相关性在空间上的距离。
在一些实施例中，范数为L2范数(欧几里得范数)，是向量(注意力矩阵)平方和的平方根，用于度量第一相关性和第二相关性之间差异的大小。范数对应的范数值提供了一个量化的度量。可选地，较小的范数值表示第一相关性和第二相关性相似，而较大的范数值表示第一相关性和第二相关性有显著差异。
在一些实施例中，计算相关性差异的范数也可以理解为，计算相关性差异的损失函数(相关性差异之间的二范数距离)。可选地，相关性差异的范数可以表示为：
Y＝||A(x′i,xt,f)-A(R(x′i,α∞),xt,f)||2
其中，x′i表示对抗图像，R(x′i,α∞)表示加噪图像，xt表示文本，f表示视觉语言模型，A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，即第二相关性，A(R(x′i,α∞),xt,f)表示加噪图像对应的加噪跨模态注意力分布，即第一相关性，A(x′i,xt,f)-A(R(x′i,α∞),xt,f)表示相关性差异，||A(x′i,xt,f)-ARxi′,α∞,xt,f2表示相关性差异的范数。
步骤251b：计算相关性差异的范数针对对抗图像中每个像素的梯度；
在一些实施例中，相关性差异的范数可以认为是相关性差异的损失函数，相关性差异的范数针对对抗图像中每个像素的梯度，可以理解为，损失函数针对对抗图像中每个像素的梯度。可选地，梯度用于指示损失函数下降最快的方向，即，梯度用于指示如何添加第二噪声以最小化相关性差异。
可选地，梯度可以表示为：
其中，g表示梯度，A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，A(R(x′i,α∞),xt,f)表示为加噪图像对应的加噪跨模态注意力分布，表示针对对抗图像中像素的梯度。
步骤251c：基于每个像素的梯度确定每个像素的加噪方式，按照每个像素的加噪方式为每个像素添加第二噪声，得到净化图像。
在一些实施例中，基于符号函数确定梯度的梯度方向，梯度方向用于指示对抗图像添加第二噪声的加噪方向，即在对抗图像中的像素上添加第二噪声，对抗图像中的像素朝梯度方向进行调整，以减小相关性差异。
在一些实施例中，基于对抗图像中每个像素的梯度确定每个像素的加噪方向，也可以理解为，基于对抗图像中每个像素的梯度确定每个像素的加噪方式。其中，加噪方向与梯度方向相同，即，加噪方式是在对抗图像的像素上沿着梯度方向添加第二噪声。可选地，加噪方式包括在对抗图像的像素上增加像素值；或者，在对抗图像的像素上减少像素值；或者，在对抗图像的像素上保持像素值不变(像素值不变的情况几乎不存在，基本上对抗图像的每个像素的像素值都会发生变化)。
可选地，基于符号函数确定对抗图像中添加第二噪声的加噪方向(或称每个像素的加噪方式)，可以表示为：
其中，A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，A(R(x′i,α∞),xt,f)表示为加噪图像对应的加噪跨模态注意力分布，sign(·)表示符号函数，表示梯度。
在一些实施例中，梯度作为符号函数的输入，在梯度为正值时，即在对抗图像中的第i个像素的梯度为正值的情况下，符号函数的输出为1，表示第i个像素的加噪方式是第一加噪方式，所述第一加噪方式用于向所述第i个像素的像素值添加第二噪声，以使所述第i个像素的像素值增大。在梯度为负值时，即在第i个像素的梯度为负值的情况下，符号函数的输出为-1，表示所述第i个像素的加噪方式是第二加噪方式，所述第二加噪方式用于向所述第i个像素的像素值添加第二噪声，以使所述第i个像素的像素值降低。在梯度为零时，即在对抗图像中的第i个像素的梯度为零的情况下，符号函数的输出为0，表示第i个像素的加噪方式是保持像素值不变，可选地，保持第i个像素的像素值不变(这种情况几乎不存在)。通过上述基于针对对抗图像中的每个像素的加噪方式，得到净化图像。
本申请实施例，通过使用符号函数确定梯度的梯度方向，梯度方向用于指示对抗图像添加第二噪声的加噪方向，即基于梯度确定每个像素的加噪方式，利用梯度指导像素值的调整，可以精确地针对对抗图像进行净化，使用符号函数可以简化对梯度的处理，将梯度直接转化为像素值调整的方向(增加或减少)，简化了计算过程。
综上所述，本申请实施例提供的方法，通过计算相关性差异的范数，可以定量评估第一相关性和第二相关性之间的差异。利用每个像素的梯度确定其加噪方式，使得净化过程具有针对性，基于每个像素的梯度决定其噪声添加方式，可以自适应地进行调整。通过向对抗图像中每个像素的自适应调整方式得到的净化图像，可以显著减少对抗扰动的影响，增强了视觉语言模型的鲁棒性和安全性，提高了其对抗攻击的防御能力。
在一些实施例中，所述以最小化相关性差异为目标，向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到所述净化图像，包括：计算所述相关性差异的最小差异值；计算所述最小相关性差异的范数针对所述对抗图像中每个像素的梯度；基于所述每个像素的梯度确定所述每个像素的加噪方式，按照所述每个像素的加噪方式为所述每个像素添加第二噪声，得到所述净化图像。
应理解的是，所述相关性差异实际是一个矩阵，而所述最小相关性差异是该矩阵中的最小元素。
应理解的是，关于梯度计算方式以及针对每个像素的加噪方式可以参考上文，本申请实施例对此不再赘述。
在一些实施例中，所述以最小化相关性差异为目标，向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到所述净化图像，包括：计算所述相关性差异的最大差异值；计算所述最大相关性差异的范数针对所述对抗图像中每个像素的梯度；基于所述每个像素的梯度确定所述每个像素的加噪方式，按照所述每个像素的加噪方式为所述每个像素添加第二噪声，得到所述净化图像。
应理解的是，所述相关性差异实际是一个矩阵，而所述最大相关性差异是该矩阵中的最大元素。
应理解的是，关于梯度计算方式以及针对每个像素的加噪方式可以参考上文，本申请实施例对此不再赘述。
以像素为粒度，控制向对抗图像添加第二噪声的噪声大小
在一些实施例中，通过计算对抗图像中每个像素的梯度，梯度中包括了方向信息，同时梯度中还包括强度信息，强度信息用于指示对抗图像中的像素调整的幅度大小。
基于梯度的强度信息确定对抗图像中每个像素对应的第二噪声，强度信息用于指示对抗图像中每个像素的梯度大小。
在一些实施例中，以像素粒度划分对抗样本中的对抗图像，基于梯度方向在对抗图像的不同像素位置添加不同扰动程度(不同噪声大小)的第二噪声。其中，梯度的强度信息用于指示对抗图像中每个像素的梯度大小。可选地，对抗图像中梯度较大的像素应该受到更大的扰动，即添加相对较大的噪声；对抗图像中梯度较小的像素应该有较小的扰动，即添加相对较小的噪声。
本申请实施例中，通过梯度的强度信息来确定对抗图像中每个像素添加的第二噪声的大小，强度信息反映了梯度的大小，能够更好地指导第二噪声的添加，使得噪声添加过程更加精确。
在一些实施例中，获取第一梯度值和第二梯度值，第一梯度值为对抗图像中全部像素的梯度最大值，第二梯度值为对抗图像中全部像素的梯度最小值。可选地，基于梯度最大值和梯度最小值得到梯度标准值，梯度标准值是将梯度进行归一化处理后的梯度值。可选地，将梯度归一化到[0，1]。
可选地，梯度表示为：
其中，g表示梯度，A(x′i,xt,f)表示对抗图像对应的对抗跨模态注意力分布，A(R(x′i,α∞),xt,f)表示为加噪图像对应的加噪跨模态注意力分布。
可选地，梯度标准值表示为：
gnorm＝(g-gmin)/(gmax-gmin)
其中，gnorm表示梯度标准值，gmin表示第一梯度值，即梯度最小值，gmax表示第二梯度值，即梯度的最大值。
在一些实施例中，由于梯度标准值差异较大，可以基于梯度标准值的平均值表示梯度标准值。可选地，根据梯度标准值以及梯度标准值的平均值确定第二噪声的扰动幅度，扰动幅度中包括用于确定第二噪声大小的扰动因子，基于扰动因子确定对抗图像中每个像素添加的第二噪声大小。其中，第二噪声大小与梯度的强度信息呈现正相关关系，即，对抗图像中梯度较大的像素(强度信息较强)应该受到更大的扰动，即添加相对较大的第二噪声；对抗图像中梯度较小的像素(强度信息较弱)应该有较小的扰动，即添加相对较小的第二噪声。
可选地，基于扰动因子确定对抗图像中每个像素添加的第二噪声大小，以得到净化图像。净化图像可以表示为：
其中，表示净化图像，x′i表示对抗图像，sign(g)用于指示对抗图像中添加第二噪声的加噪方向(或称每个像素的加噪方式)，gnorm表示梯度标准值，表示梯度标准值的平均值，可以基于确定第二噪声的扰动幅度，的取值为扰动因子，扰动因子用于确定第二噪声β∞的大小。可选地，0为第一归一化取值，1为第二归一化取值。扰动因子为第一归一化取值和第一取值中的最大值，第一取值为第二归一化值和第二取值中的最小值，第二取值为梯度标准值和梯度标准值的平均值的商。即第一取值为第二取值为gnorm和的商可选地，扰动因子为扰动幅度内的取值。
在一些实施例中，第二噪声基于在第二分布区间伪随机采样得到的第二噪声值和扰动因子的乘积确定，第二分布区间为均匀分布的区间范围。
本申请实施例中，通过获取对抗图像中全部像素的梯度最大值和梯度最小值，并对梯度值进行归一化处理，得到梯度标准值，使得不同尺度的梯度值能够统一处理。根据梯度标准值及其平均值确定第二噪声的扰动幅度，使得噪声添加过程更加灵活和自适应。这种自适应调整方式能够根据实际梯度情况，合理确定添加第二噪声大小，即梯度越大，添加的第二噪声越大，这种在对抗图像中添加第二噪声的方式提高净化效果。
在一些实施例中，对抗图像中每个像素对应的第二噪声的大小也可以为预设大小，即无需采用上述梯度的强度信息确定第二噪声的大小。
应理解的是，本申请实施例对上述梯度标准值的计算方式不做限制，比如，该梯度标准值是对抗图像中全部像素的梯度的平均值。
视觉语言模型的训练过程
在一些实施例中，可以预训练一个视觉语言模型，将净化样本输入视觉语言模型，可以得到针对视觉语言模型的问答分数，问答分数用于指示净化样本属于正确预测结果(正确类别)的正确概率。可选地，问答分数与净化样本的净化跨模态注意力分布呈现正相关关系。
在一种可能实现的方式中，对视觉语言模型的训练方式如下：
在一些实施例中，获取净化样本数据集，净化样本数据集是用于训练视觉语言模型的样本的集合，净化样本数据集中包括多个净化样本，每个净化样本包括净化图像和文本，其中，净化图像是通过上述实施例提供的方法得到的图像。每个净化样本上对应的标签，用于指示净化样本中净化图像所属的类别。然后，将净化样本输入视觉语言模型，视觉语言模型将净化图像编码为图像特征，将文本编码为文本特征。可选地，视觉语言模型基于净化图像的净化跨模态注意力分布，将图像特征和文本特征融合。其中，净化跨模态注意力分布用于描述净化图像和文本的注意力分布，视觉语言模型可以基于净化跨模态注意力分布能够重点关注净化图像中与文本相关的图像区域。
在一些实施例中，视觉语言模型对融合后的特征进行预测，融合后的特征是视觉语言模型基于净化跨模态注意力分布中图像和文本的注意力分布，将图像特征和文本特征融合生成的特征。可选地，视觉语言模型基于融合后的特征进行预测，可以得到针对净化样本的问答分数，问答分数用于指示净化样本属于正确预测结果(正确类别)的正确概率。可选地，问答分数与净化图像的净化跨模态注意力分布呈现正相关关系。可选地，在视觉语言模型中设置一个分数阈值，当视觉语言模型的问答分数大于分数阈值，表示净化样本属于正确预测结果；当视觉语言模型的问答分数小于或等于分数阈值，表示净化样本属于错误预测结果，说明输入的净化样本属于非完全净化结果。当视觉语言模型针对净化样本的预测结果为错误预测结果时，表示该净化样本中的净化跨模态注意力分布需要继续调整，即，需要继续向对抗样本中添加随机噪声以调整对抗样本的对抗跨模态注意力分布。
在一些实施例中，将问答分数对应的预测结果与真实结果(净化样本对应的标签)进行比较，得到计算损失，计算损失用于指示问答分数对应的预测结果与真实结果之间的误差。可选地，使用反向传播算法来调整视觉语言模型的参数，使视觉语言模型的预测结果和净化样本对应的标签更加接近。可选地，通过损失函数对视觉语言模型进行优化。损失函数可以是任意一种损失函数，例如，损失函数可以是CE loss(Cross Entropy loss，交叉熵损失)函数或均方误差(Mean Square Error，MSE)损失函数。
需要说明的是，本实施例提供的视觉语言模型的训练仅为示意性说明，并不构成对视觉语言模型的训练方式的限定。
在一些实施例中，视觉语言模型可以应用于视觉问答场景。
在一种应用场景中，样本净化的方法可以运用于分类问答领域。将净化样本输入视觉语言模型，视觉语言模型基于净化后的净化样本进行识别，得到净化样本对应的正确问答结果。
步骤1，获取分类对抗样本；
其中，分类对抗样本是对干净样本执行对抗攻击后生成的对抗样本。分类对抗样本中包括分类对抗图像和问题描述。
在一些实施例中，对抗攻击包括对干净样本进行微小修改或扰动。分类对抗样本中的修改或者扰动不易察觉，但会导致视觉语言模型做出错误的预测并生成错误的预测结果(输出结果)。
示例性的，一张干净图像显示为在建筑物的门口停放一辆自行车，在问答场景下，针对该干净图像的问题描述为“这个建筑物的大门是什么材质”，视觉语言模型正确地将其分类为“木头”。通过对该干净图像应用对抗攻击，例如基于FGSM方法对干净图像进行对抗攻击，可以生成一个分类对抗图像，在问答场景下，针对该分类对抗图像的问题描述为“这个建筑物的大门是什么材质”，视觉语言模型错误地将其分类为“金属”。
在一些实施例中，视觉语言模型基于干净图像中的视觉令牌和语言令牌计算干净图像对应的对抗跨模态注意力分布。可选地，基于视觉编码器将干净图像编码为图像特征，通过视觉语言模型对齐图像特征得到至少一个视觉令牌，至少一个视觉令牌中的每个视觉令牌用于指示干净图像的图像特征；基于语言模型分词器将问题描述编码为文本特征，通过视觉语言模型对齐文本特征得到至少一个语言令牌，至少一个语言令牌中的每个语言令牌用于指示问题描述的文本特征；通过视觉语言模型计算生成干净预测结果时的干净跨模态注意力分布，干净预测结果为视觉语言模型针对至少一个视觉令牌和至少一个语言令牌融合后的预测结果。
示例性的，视觉语言模型对干净图像进行预测时，生成的干净预测结果为“木头”，视觉语言模型会计算干净图像中所有视觉令牌的注意力权重，视觉语言模型将干净图像分为若干个区域，每个区域的图像特征(或者多个区域对应的图像特征)可以表示为一个视觉令牌，注意力权重表示每个视觉令牌对回答“木头”的重要性，即视觉语言模型会评估每个视觉令牌在多大程度上影响“木头”的生成。
在一些实施例中，视觉语言模型对分类对抗图像进行预测时，会计算分类对抗图像的对抗跨模态注意力分布。对抗跨模态注意力分布是分类对抗图像对应的跨模态注意力分布，用于描述分类对抗图像和问题描述的注意力分布。针对分类对抗图像的对抗跨模态注意力分布类似上述干净图像的干净跨模态注意力分布的过程。
步骤2，获取向对抗图像添加第一噪声后得到的加噪图像；
其中，加噪图像是针对分类对抗图像执行第一扰动后的图像。第一噪声指的是在分类对抗图像上添加的随机噪声。第一噪声是随机的或按照某种规则生成的，目的是扰动分类对抗图像的像素值。在一种可能实现的方式中，通过计算机程序实现第一噪声的添加。在一种可能实现的方式中，通过噪声模型实现第一噪声的添加，例如使用生成对抗网络、变分自编码器或其他深度学习模型来生成第一噪声，然后将第一噪声添加到分类对抗图像上。
步骤3，确定加噪图像的至少一个加噪图像特征和文本的至少一个文本特征之间的第一相关性；
换句话将，可以计算加噪图像对应的加噪跨模态注意力分布。
其中，加噪跨模态注意力分布用于描述加噪图像的至少一个加噪图像特征和问题描述文本的至少一个文本特征之间的相关性。
在一种可能的实现的方式中，通过在分类对抗图像上加入第一噪声，得到加噪图像，相较于分类对抗图像的对抗跨模态注意力分布，加噪图像的加噪跨模态注意力分布更接近干净图像的干净跨模态注意力分布，因此，以加噪图像的加噪跨模态注意力分布为参考估计方向，引导分类对抗图像的对抗跨模态注意力分布进行优化，可以认为是针对分类对抗图像的净化操作，使其更接近干净图像。
示例性的，针对加噪图像的问题描述为“这个建筑物的大门是什么材质”，视觉语言模型可能错误地将其分类为“砖块”。但是该加噪图像对应的加噪跨模态注意力分布更接近干净图像的干净跨模态注意力分布。
在一些实施例中，计算加噪图像对应的加噪跨模态注意力分布。加噪跨模态注意力分布指的是视觉语言模型基于加噪样本生成预测结果时，如何关注加噪样本中加噪图像的注意力大小(或注意力权重)。针对加噪图像的对抗跨模态注意力分布类似上述加噪图像的加噪跨模态注意力分布的过程。
步骤4，确定对抗图像的至少一个对抗图像特征和文本的所述至少一个文本特征之间的第二相关性；
换句话讲，计算对抗图像对应的对抗跨模态注意力分布；
对抗跨模态注意力分布用于描述对抗图像的至少一个对抗图像特征和问题描述文本的至少一个文本特征之间的相关性。
步骤5，基于第一相关性和所述第二相关性向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像。
其中，净化图像为向对抗图像执行第二扰动后的图像。第二噪声指的是在分类对抗图像上添加的随机噪声。第二噪声是随机的或按照某种规则生成的，目的是扰动分类对抗图像的像素值。
示例性的，针对加噪图像的问题描述为“这个建筑物的大门是什么材质”，视觉语言模型可能正确地将其分类为“木头”。尽管净化图像在视觉上可能更嘈杂，但是净化图像的净化跨模态注意力分布可能更接近于干净图像的干净跨模态注意力分布，因此视觉语言模型对净化图像的预测结果更接近对干净图像的预测结果。
在一些实施例中，通过在分类对抗图像中继续添加随机噪声的方式，可以引导第二相关性接近第一相关性，进而生成净化图像。可选地，以最小化相关性差异为目标，向分类对抗图像添加第二噪声，得到净化图像。
在一些实施例中，可以确定添加第二噪声的方向。首先计算相关性差异的范数，相关性差异的范数用于衡量第一相关性与第二相关性之间的距离。其次，计算相关性差异针对分类对抗图像中像素的梯度。然后，基于符号函数确定梯度的梯度方向，也可以理解为，基于每个像素的梯度确定每个像素的加噪方式，梯度方向用于指示添加第二噪声的方向。可选地，梯度作为符号函数的输入，在梯度为正值时，按照梯度方向增加对抗图像中的像素值；在梯度为负值时，按照梯度方向减少对抗图像中的像素值，按照梯度方向增加或减少对抗图像中的像素值，以得到添加第二噪声后的净化图像。
在一些实施例中，还可以确定添加第二噪声的大小。可选地，梯度中不仅包含梯度方向的信息，同时还包括强度信息。可选地，对抗图像中梯度较大的像素应该受到更大的扰动，即添加相对较大的噪声；对抗图像中梯度较小的像素应该有较小的扰动，即添加相对较小的噪声。
综上所述，视觉语言模型基于分类对抗样本会产生错误的预测，通过针对对抗样本中的分类对抗图像执行净化操作后得到净化图像，进而可以得到净化样本。具体地，通过针对分类对抗图像进行随机加噪处理得到加噪图像，由于加噪图像的加噪跨模态注意力分布(即第一相关性)非常接近干净图像对应的跨模态注意力分布(即第二相关性)，因此可以将加噪图像的加噪跨模态注意力分布为优化目标，向分类对抗图像中加入随机噪声，进而使得分类对抗图像的对抗跨模态注意力分布接近加噪图像的加噪跨模态注意力分布，以完成针对分类对抗图像的净化并得到净化图像。本申请以加噪跨模态注意力分布为引导，通过在分类对抗图像中添加噪声以实现分类对抗图像的净化，视觉语言模型基于净化图像可以得到正确的预测结果，这种净化方式增强了视觉语言模型的鲁棒性。
本方案的定量结果
本方案可以称为FFF，具体来说，针对干净样本中的干净图像xi，视觉语言模型的预测结果f(xi,xt)也是正确的，提取干净图像的干净跨模态注意力分布A(xi,xt,f)。对于对抗样本中的对抗图像x′i，视觉语言模型的预测结果f(x′i,xt)是错误的，提取对抗图像的对抗跨模态注意力分布A(x′i,xt,f)。通过比对可以发现，对抗图像的对抗跨模态注意力分布与干净图像的干净跨模态注意力分布有显著差异，可以表现对抗跨模态注意力分布图和干净跨模态注意力分布图上具有显著的模式差别。通过进行可行性验证，即，如果以干净图像的干净跨模态注意力分布作为目标优化对抗图像，可以实现对抗图像的净化。
然而，在对抗净化的场景中，无法得知干净图像，也就无法提取得到干净图像的干净跨模态注意力分布，因此，需要合理的估计干净图像的干净跨模态注意力分布。测试发现，通过在对抗样本的对抗图像上添加无穷范数限制为α∞的随机噪声，即R(x′′i,α∞)，虽然视觉语言模型的预测结果f(R(x′′i,α∞),xt)仍然是错误的，但是加噪跨模态注意力分布A(R(x′′i,α∞),xt,f)有向干净图像的干净跨模态注意力分布靠近的趋势。如果向加噪跨模态注意力分布A(R(x′i,α∞),xt,f)优化对抗图像的对抗跨模态注意力分布，可能可以实现对抗图像的净化。实验证明确实如此，以随机加噪对抗图像的对抗跨模态注意力分布为目标，在对抗图像上添加噪声，得到净化图像，其答案正确，且净化图像的净化跨模态注意力分布也确实更像干净图像的干净跨模态注意力分布。
下面我们将给出一些定量结果。
首先，本方案验证使用干净图像的干净跨模态注意力分布是否可以净化对抗图像。本方案以干净图像的干净跨模态注意力分布为目标，在对抗图像上添加噪声，结果如表1所示。噪声强度越高，净化效果越好，且添加噪声后的对抗图像对应的跨模态注意力分布确实越靠近干净图像的干净跨模态注意力分布。
表1
其中，均方误差即上文中提到的L2范数。VQA分数是视觉语言模型基于输入图像的视觉问答分数。表1中第一行为视觉语言模型的评分(视觉问答分数)，第二行和第三行为对抗图像加噪后对应的跨模态注意力分布与干净图像对应的干净跨模态注意力分布的差异衡量。
其次，本方案验证在对抗图像上随机加噪得到的加噪图像对预测结果和跨模态注意力分布的影响。由表2可见，随着噪声强度的增加，视觉语言模型的视觉问答分数有一定的提升但不大，但加噪图像对应的加噪跨模态注意力分布却更靠近干净图像的干净跨模态注意力分布。
表2
本方案利用随机加噪的加噪图像的加噪跨模态注意力分布作为干净图像的干净跨模态注意力分布的估计，在对抗图像上添加噪声。噪声的大小随机选择，利用估计的加噪跨模态注意力分布控制器方向，结果如表3。第一行为随机方向，第二行开始为本方案的方法。可以看到，本方案确实可以实现一定程度的性能净化。每一行为估计干净图像的干净跨模态注意力分布加入的噪声强度，每一列为净化对抗图像加入的噪声强度。
表3
之前本方案只是控制了噪声方向，噪声大小还是随机取的，本方案进一步控制噪声大小，结果如表4所示。第一行为随机方向，第二行开始为本方案的方法。可以看到，相比只控制方向的优化，控制噪声大小的效果更好。
表4
图7示出了本申请一个实施例提供的样本净化装置的结构框图。该样本净化装置具有实现上述样本净化方法示例的功能，功能可以由硬件实现，也可以由硬件执行相应的软件实现。该净化装置可以是上文介绍的服务器，也可以设置在服务器中。如图7所示，该净化装置可以包括：获取模块710、确定模块720和添加模块730；
获取模块710，用于：
获取对抗样本，所述对抗样本通过对干净样本执行对抗攻击后生成，所述对抗样本中包括对抗图像和文本；
获取向所述对抗图像添加第一噪声后得到的加噪图像；
确定模块720用于：
确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性；
确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性；
添加模块730用于基于第一相关性和第二相关性向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像。
在一些实施例中，添加模块730具体用于：以最小化相关性差异为目标，向对抗图像添加第二噪声，以对对抗图像进行净化，得到净化图像；其中，相关性差异为第一相关性和第二相关性之间的差异。
在一些实施例中，添加模块730具体用于：计算相关性差异的范数，范数用于衡量第一相关性和第二相关性之间的距离；计算相关性差异的范数针对对抗图像中每个像素的梯度；基于每个像素的梯度确定每个像素的加噪方式，按照每个像素的加噪方式为每个像素添加第二噪声，得到净化图像。
在一些实施例中，添加模块730具体用于：在对抗图像中的第i个像素的梯度为正值的情况下，确定第i个像素的加噪方式是第一加噪方式，第一加噪方式用于向第i个像素的像素值添加第二噪声，以使第i个像素的像素值增大，i为正整数；在第i个像素的梯度为负值的情况下，确定第i个像素的加噪方式是第二加噪方式，第二加噪方式用于向第i个像素的像素值添加第二噪声，以使第i个像素的像素值降低；基于针对对抗图像中的每个像素的加噪方式为每个像素添加第二噪声，得到净化图像。
在一些实施例中，梯度包括强度信息，确定模块720还用于：基于梯度的强度信息确定对抗图像中每个像素对应的第二噪声，强度信息用于指示对抗图像中每个像素的梯度大小。
在一些实施例中，确定模块720具体用于：获取第一梯度值和第二梯度值，第一梯度值为对抗图像中全部像素的梯度最大值，第二梯度值为对抗图像中全部像素的梯度最小值；基于第一梯度值和第二梯度值得到梯度标准值，梯度标准值是将梯度执行归一化处理后的梯度值；根据梯度标准值以及梯度标准值的平均值确定第二噪声的扰动幅度，扰动幅度中包括用于确定第二噪声大小的扰动因子；基于扰动因子确定对抗图像中每个像素添加的第二噪声大小，第二噪声大小与梯度的强度信息呈现正相关关系。
在一些实施例中，扰动因子为第一归一化值和第一取值中的最大值，第一取值为第二归一化值和第二取值中的最小值，第二取值为梯度标准值和梯度标准值的平均值的商，第一归一化值为0，第二归一化值为1。
在一些实施例中，净化装置还包括：生成模块740，其中，获取模块710还用于获取净化样本，净化样本包括净化图像和文本；生成模块740用于通过视觉语言模型基于净化样本生成针对净化图像的预测结果，净化图像的预测结果为正确预测结果。
在一些实施例中，确定模块720具体用于：通过视觉语言模型确定第一相关性。
在一些实施例中，视觉语言模型包括视觉编码器和语言模型分词器；确定模块720具体用于：基于视觉编码器将加噪图像划分为多个区域，并将每个区域编码为区域图像特征；通过视觉语言模型对齐多个区域的区域图像特征，得到至少一个加噪图像特征；基于语言模型分词器将文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；通过视觉语言模型对齐多个文本单元的文本单元特征，得到至少一个文本特征；通过视觉语言模型计算生成加噪预测结果时的第一相关性，加噪预测结果为视觉语言模型针对加噪图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个视觉令牌对应一个加噪图像特征；每个语言令牌对应一个文本特征。
在一些实施例中，确定模块720具体用于：通过视觉语言模型确定第二相关性。
在一些实施例中，视觉语言模型包括视觉编码器和语言模型分词器；确定模块720具体用于：基于视觉编码器将对抗图像划分为多个区域，并将每个区域编码为区域图像特征；通过视觉语言模型对齐多个区域的区域图像特征，得到至少一个对抗图像特征；基于语言模型分词器将文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；通过视觉语言模型对齐多个文本单元的文本单元特征，得到至少一个文本特征；通过视觉语言模型计算生成对抗测结果时的第二相关性，对抗预测结果为视觉语言模型针对对抗图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个视觉令牌对应一个对抗图像特征；每个语言令牌对应一个文本特征。
在一些实施例中，第一噪声基于在第一分布区间伪随机采样得到的第一噪声值确定。
在一些实施例中，第一分布区间为均匀分布的区间范围。
在一些实施例中，第二噪声基于在第二分布区间伪随机采样得到的第二噪声值确定。
在一些实施例中，第二分布区间为均匀分布的区间范围。
在一些实施例中，干净样本包括干净图像和文本；确定模块720还用于：确定干净图像的至少一个干净图像特征和文本的至少一个文本特征之间的第三相关性；其中，第一相关性相对于第二相关性更接近于第三相关性。
需要说明的是，上述提供的一个或多个样本净化装置的具体限定可以参见上文中对于样本净化方法的限定，在此不再赘述。上述净化装置的各模块可全部或部分通过软件、硬件及其组合来实现，各模块可以用硬件形式内嵌或独立于计算机设备的处理器中，也可以用软件形式存储在计算机设备的存储器中，以便于处理器调用执行各模块对应的操作。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
图8示出了本申请一个示例性实施例示出的计算机设备1200的结构框图。该计算机设备可用于实施上述实施例中提供的样本净化方法。所述计算机设备1200包括中央处理单元(Central Processing Unit，CPU)1201、包括随机存取存储器(Random Access Memory，RAM)1202和只读存储器(Read-Only Memory，ROM)1203的系统存储器1204，以及连接系统存储器1204和中央处理单元1201的系统总线1205。所述计算机设备1200还包括帮助计算机设备内的各个器件之间传输信息的基本输入/输出系统(Input/Output系统，I/O系统)1206，和用于存储操作系统1213、应用程序1214和其他程序模块1215的大容量存储设备1207。
所述基本输入/输出系统1206包括有用于显示信息的显示器1208和用于用户输入信息的诸如鼠标、键盘之类的输入设备1209。其中所述显示器1208和输入设备1209都通过连接到系统总线1205的输入输出控制器1210连接到中央处理单元1201。所述基本输入/输出系统1206还可以包括输入输出控制器1210以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入输出控制器1210还提供输出到显示屏、打印机或其他类型的输出设备。
所述大容量存储设备1207通过连接到系统总线1205的大容量存储控制器(未示出)连接到中央处理单元1201。所述大容量存储设备1207及其相关联的计算机可读存储介质为终端设备1200提供非易失性存储。也就是说，所述大容量存储设备1207可以包括诸如硬盘或者只读光盘(Compact Disc Read-Only Memory，CD-ROM)驱动器之类的计算机可读存储介质(未示出)。
不失一般性，所述计算机可读存储介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读存储指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、可擦除可编程只读寄存器(Erasable Programmable Read Only Memory，EPROM)、电子抹除式可复写只读存储器(Electrically-Erasable Programmable Read-Only Memory，EEPROM)、闪存或其他固态存储其技术，CD-ROM、数字多功能光盘(Digital Versatile Disc，DVD)或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然，本领域技术人员可知所述计算机存储介质不局限于上述几种。上述的系统存储器1204和大容量存储设备1207可以统称为存储器。
存储器存储有一个或多个程序，一个或多个程序被配置成由一个或多个中央处理单元1201执行，一个或多个程序包含用于实现上述方法实施例的指令，中央处理单元1201执行该一个或多个程序实现上述各个方法实施例提供的样本净化方法。
根据本申请的各种实施例，所述计算机设备1200还可以通过诸如因特网等网络连接到网络上的远程终端设备运行。也即计算机设备1200可以通过连接在所述系统总线1205上的网络接口单元1211连接到网络1212，或者说，也可以使用网络接口单元1211来连接到其他类型的网络或远程终端设备系统(未示出)。
所述存储器还包括一个或者一个以上的程序，所述一个或者一个以上程序存储于存储器中，所述一个或者一个以上程序包含用于进行本申请实施例提供的方法中由终端设备所执行的样本净化的步骤。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质中存储有至少一条计算机程序，该至少一条计算机程序由处理器加载并执行以实现上述各方法实施例提供的样本净化方法。
本申请实施例还提供一种计算机程序产品，所述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中；所述计算机程序由计算机设备的处理器从所述计算机可读存储介质读取并执行，使得所述计算机设备执行以实现上述各方法实施例提供的样本净化方法。
可以理解的是，在本申请的具体实施方式中，涉及到的数据，历史数据，以及画像等与用户身份或特性相关的用户数据处理等相关的数据，当本申请以上实施例运用到具体产品或技术中时，需要获得用户许可或者同意，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
需要说明的是，除非本文中另外明确定义，否则用于权利要求中的所有术语根据它们在技术领域中的普通含义来解释。除非另外明确叙述，否则对“一个元件、装置、部件、设备、步骤等”的所有参考将被开放地解释为指代元件、装置、部件、设备、步骤等的至少一个实例。除非明确叙述，否则本文所公开的任意方法的步骤不是必须以所公开的确切顺序来执行。
应当理解的是，在本文中提及的“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。

Claims (21)

  1. 一种样本净化方法，其特征在于，所述方法包括：
    获取对抗样本，所述对抗样本通过对干净样本执行对抗攻击后生成，所述对抗样本中包括对抗图像和文本；
    获取向所述对抗图像添加第一噪声后得到的加噪图像；
    确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性；
    确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性；
    基于所述第一相关性和所述第二相关性向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述第一相关性和所述第二相关性向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像，包括：
    以最小化相关性差异为目标，向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到所述净化图像；
    其中，所述相关性差异为所述第一相关性和所述第二相关性之间的差异。
  3. 根据权利要求2所述的方法，其特征在于，所述以最小化相关性差异为目标，向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到所述净化图像，包括：
    计算所述相关性差异的范数，所述范数用于衡量所述第一相关性和所述第二相关性之间的距离；
    计算所述相关性差异的范数针对所述对抗图像中每个像素的梯度；
    基于所述每个像素的梯度确定所述每个像素的加噪方式，按照所述每个像素的加噪方式为所述每个像素添加第二噪声，得到所述净化图像。
  4. 根据权利要求3所述的方法，其特征在于，所述基于所述每个像素的梯度确定所述每个像素的加噪方式，按照所述每个像素的加噪方式为所述每个像素添加第二噪声，得到所述净化图像，包括：
    在所述对抗图像中的第i个像素的梯度为正值的情况下，确定所述第i个像素的加噪方式是第一加噪方式，所述第一加噪方式用于向所述第i个像素的像素值添加第二噪声，以使所述第i个像素的像素值增大，所述i为正整数；
    在所述第i个像素的梯度为负值的情况下，确定所述第i个像素的加噪方式是第二加噪方式，所述第二加噪方式用于向所述第i个像素的像素值添加第二噪声，以使所述第i个像素的像素值降低；
    基于针对所述对抗图像中的每个像素的加噪方式为所述每个像素添加第二噪声，得到所述净化图像。
  5. 根据权利要求3或4所述的方法，其特征在于，所述梯度包括强度信息，所述方法还包括：
    基于所述梯度的强度信息确定所述对抗图像中每个像素对应的第二噪声，所述强度信息用于指示所述对抗图像中每个像素的梯度大小。
  6. 根据权利要求5所述的方法，其特征在于，所述基于所述梯度的强度信息确定所述对抗图像中每个像素对应的第二噪声，包括：
    获取第一梯度值和第二梯度值，所述第一梯度值为所述对抗图像中全部像素的梯度最大值，所述第二梯度值为所述对抗图像中全部像素的梯度最小值；
    基于所述第一梯度值和所述第二梯度值得到梯度标准值，所述梯度标准值是将所述梯度执行归一化处理后的梯度值；
    根据所述梯度标准值以及所述梯度标准值的平均值确定所述第二噪声的扰动幅度，所述扰动幅度中包括用于确定第二噪声大小的扰动因子；
    基于所述扰动因子确定所述对抗图像中每个像素添加的第二噪声大小，所述第二噪声大小与所述梯度的强度信息呈现正相关关系。
  7. 根据权利要求6所述的方法，其特征在于，所述扰动因子为第一归一化值和第一取值中的最大值，所述第一取值为第二归一化值和第二取值中的最小值，所述第二取值为所述梯度标准值和所述梯度标准值的平均值的商，所述第一归一化值为0，所述第二归一化值为1。
  8. 根据权利要求1至7任一所述的方法，其特征在于，所述方法还包括：
    获取净化样本，所述净化样本包括所述净化图像和所述文本；
    通过视觉语言模型基于所述净化样本生成针对所述净化图像的预测结果，所述净化图像的预测结果为正确预测结果。
  9. 根据权利要求1至8任一所述的方法，其特征在于，所述确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性，包括：
    通过视觉语言模型确定所述第一相关性。
  10. 根据权利要求9所述的方法，其特征在于，所述视觉语言模型包括视觉编码器和语言模型分词器；
    所述通过视觉语言模型确定所述第一相关性，包括：
    基于所述视觉编码器将所述加噪图像划分为多个区域，并将每个区域编码为区域图像特征；
    通过所述视觉语言模型对齐所述多个区域的区域图像特征，得到所述至少一个加噪图像特征；
    基于所述语言模型分词器将所述文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；
    通过所述视觉语言模型对齐所述多个文本单元的文本单元特征，得到所述至少一个文本特征；
    通过所述视觉语言模型计算生成加噪预测结果时的所述第一相关性，所述加噪预测结果为所述视觉语言模型针对所述加噪图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个所述视觉令牌对应一个所述加噪图像特征；每个所述语言令牌对应一个所述文本特征。
  11. 根据权利要求1至8任一所述的方法，其特征在于，所述确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性，包括：
    通过视觉语言模型确定所述第二相关性。
  12. 根据权利要求11所述的方法，其特征在于，所述视觉语言模型包括视觉编码器和语言模型分词器；
    所述通过视觉语言模型确定所述第二相关性，包括：
    基于所述视觉编码器将所述对抗图像划分为多个区域，并将每个区域编码为区域图像特征；
    通过所述视觉语言模型对齐所述多个区域的区域图像特征，得到所述至少一个对抗图像特征；
    基于所述语言模型分词器将所述文本划分为多个文本单元，并将每个文本单元编码为文本单元特征；
    通过所述视觉语言模型对齐所述多个文本单元的文本单元特征，得到所述至少一个文本特征；
    通过所述视觉语言模型计算生成对抗测结果时的所述第二相关性，所述对抗预测结果为所述视觉语言模型针对所述对抗图像的至少一个视觉令牌和至少一个语言令牌融合后的预测结果；其中，每个所述视觉令牌对应一个所述对抗图像特征；每个所述语言令牌对应一个所述文本特征。
  13. 根据权利要求1至12任一所述的方法，其特征在于，所述第一噪声基于在第一分布区间伪随机采样得到的第一噪声值确定。
  14. 根据权利要求13所述的方法，其特征在于，所述第一分布区间为均匀分布的区间范围。
  15. 根据权利要求1至14任一所述的方法，其特征在于，所述第二噪声基于在第二分布区间伪随机采样得到的第二噪声值确定。
  16. 根据权利要求15所述的方法，其特征在于，所述第二分布区间为均匀分布的区间范围。
  17. 根据权利要求1-16任一项所述的方法，其特征在于，所述干净样本包括干净图像和文本；所述方法还包括：
    确定所述干净图像的至少一个干净图像特征和所述文本的至少一个文本特征之间的第三相关性；
    其中，所述第一相关性相对于所述第二相关性更接近于所述第三相关性。
  18. 一种样本净化装置，其特征在于，所述装置包括：获取模块、确定模块和添加模块；
    所述获取模块，用于：
    获取对抗样本，所述对抗样本通过对干净样本执行对抗攻击后生成，所述对抗样本中包括对抗图像和文本；
    获取向所述对抗图像添加第一噪声后得到的加噪图像；
    所述确定模块用于：
    确定所述加噪图像的至少一个加噪图像特征和所述文本的至少一个文本特征之间的第一相关性；
    确定所述对抗图像的至少一个对抗图像特征和所述文本的所述至少一个文本特征之间的第二相关性；
    所述添加模块用于基于所述第一相关性和所述第二相关性向所述对抗图像添加第二噪声，以对所述对抗图像进行净化，得到净化图像。
  19. 一种计算机设备，其特征在于，所述计算机设备包括：处理器和存储器，所述存储器中存储有至少一条计算机程序，至少一条所述计算机程序由所述处理器加载并执行以实现如权利要求1至17中任一项所述的样本净化方法。
  20. 一种计算机存储介质，其特征在于，所述计算机可读存储介质中存储有至少一条计算机程序，至少一条计算机程序由处理器加载并执行以实现如权利要求1至17中任一项所述的样本净化方法。
  21. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中；所述计算机程序由计算机设备的处理器从所述计算机可读存储介质读取并执行，使得所述计算机设备执行如权利要求1至17中任一项所述的样本净化方法。
PCT/CN2025/096787 2024-07-23 2025-05-23 样本净化方法、装置、设备及介质 Pending WO2026020964A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410994732.0A CN118865024B (zh) 2024-07-23 2024-07-23 样本净化方法、装置、设备及介质
CN202410994732.0 2024-07-23

Publications (1)

Publication Number Publication Date
WO2026020964A1 true WO2026020964A1 (zh) 2026-01-29

Family

ID=93174622

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/096787 Pending WO2026020964A1 (zh) 2024-07-23 2025-05-23 样本净化方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN118865024B (zh)
WO (1) WO2026020964A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118865024B (zh) * 2024-07-23 2025-04-25 腾讯科技（深圳）有限公司 样本净化方法、装置、设备及介质
CN119782770B (zh) * 2025-03-11 2025-06-17 天津大学 扰动样本对生成方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20230095244A (ko) * 2021-12-22 2023-06-29 고려대학교 산학협력단 초해상화가 탑재된 물체 탐지 모델의 오작동을 유도하는 적대적 공격 방법
CN117036184A (zh) * 2023-07-05 2023-11-10 腾讯科技（深圳）有限公司 图像合成方法、装置、电子设备及存储介质
CN117437317A (zh) * 2023-10-25 2024-01-23 腾讯科技（深圳）有限公司 图像生成方法、装置、电子设备、存储介质和程序产品
CN117830627A (zh) * 2023-11-24 2024-04-05 中国人民解放军军事科学院系统工程研究院 基于动态正则化对抗损失项的真实物理对抗补丁生成方法
CN118865024A (zh) * 2024-07-23 2024-10-29 腾讯科技（深圳）有限公司 样本净化方法、装置、设备及介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200372301A1 (en) * 2019-05-21 2020-11-26 Retrace Labs Adversarial Defense Platform For Automated Dental Image Classification
US11217350B2 (en) * 2019-06-27 2022-01-04 Retrace Labs Systems and method for artificial-intelligence-based dental image to text generation
CN113554089B (zh) * 2021-07-22 2023-04-18 西安电子科技大学 一种图像分类对抗样本防御方法、系统及数据处理终端
CN117765320B (zh) * 2023-12-23 2026-07-24 西安交通大学 一种针对图像分类任务的对抗样本防御方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20230095244A (ko) * 2021-12-22 2023-06-29 고려대학교 산학협력단 초해상화가 탑재된 물체 탐지 모델의 오작동을 유도하는 적대적 공격 방법
CN117036184A (zh) * 2023-07-05 2023-11-10 腾讯科技（深圳）有限公司 图像合成方法、装置、电子设备及存储介质
CN117437317A (zh) * 2023-10-25 2024-01-23 腾讯科技（深圳）有限公司 图像生成方法、装置、电子设备、存储介质和程序产品
CN117830627A (zh) * 2023-11-24 2024-04-05 中国人民解放军军事科学院系统工程研究院 基于动态正则化对抗损失项的真实物理对抗补丁生成方法
CN118865024A (zh) * 2024-07-23 2024-10-29 腾讯科技（深圳）有限公司 样本净化方法、装置、设备及介质

Also Published As

Publication number Publication date
CN118865024B (zh) 2025-04-25
CN118865024A (zh) 2024-10-29

Similar Documents

Publication Publication Date Title
CN114663335B (zh) 图像瑕疵检测方法、装置、电子设备及介质
US12205037B2 (en) Clustering autoencoder
US11531874B2 (en) Regularizing machine learning models
US20230222369A1 (en) Facilitating device fingerprinting through assignment of fuzzy device identifiers
CN114842267A (zh) 基于标签噪声域自适应的图像分类方法及系统
US12141238B2 (en) Deep neural network hardener
CN115485741A (zh) 用于图像分割的神经网络模型
EP4664416A1 (en) Training method and apparatus for liveness detection model, and medium, electronic device and product
Yu et al. Self-calibrated tuning of vision-language models for out-of-distribution detection
CN105960647A (zh) 紧凑人脸表示
CN113343235B (zh) 基于Transformer的应用层恶意有效负载检测方法、系统、设备及介质
WO2024194341A1 (en) Multi-stage watermarking of a digital object generated by a machine learning model
CN118865024B (zh) 样本净化方法、装置、设备及介质
CN113222480A (zh) 对抗样本生成模型的训练方法及装置
CN114140670A (zh) 基于外源特征进行模型所有权验证的方法和装置
CN111178146A (zh) 基于人脸特征识别主播的方法及装置
TWI803243B (zh) 圖像擴增方法、電腦設備及儲存介質
CN111079930A (zh) 数据集质量参数的确定方法、装置及电子设备
CN114693978B (zh) 模型训练方法、样本生成方法、装置、设备及存储介质
CN117114139A (zh) 一种面向噪声标签的联邦学习方法
Song et al. Deep adaptive temporal pooling for activity recognition
CN120217406A (zh) 一种黑盒场景下的模型对抗攻击鲁棒性评估方法和系统
Liu et al. Enhancing generalization in few-shot learning for detecting unknown adversarial examples
CN111373416A (zh) 通过离散神经网络输入来增强神经网络的安全性
CN115640847A (zh) 联邦学习中成员推理攻击方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25843353

Country of ref document: EP

Kind code of ref document: A1