CN117152565A

CN117152565A - Model training method, related device and storage medium

Info

Publication number: CN117152565A
Application number: CN202311340648.9A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Beijing Real AI Technology Co Ltd
Current assignee: Beijing Real AI Technology Co Ltd
Priority date: 2023-10-16
Filing date: 2023-10-16
Publication date: 2023-12-01

Abstract

The application relates to the technical field of artificial intelligence, and discloses a model training method, a related device and a storage medium, wherein the model training method comprises the following steps: selecting an attack category and attack logic; constructing a model to be trained according to the attack logic; inputting a preset sample picture and the attack category into a model to be trained, and generating candidate countermeasure disturbance; obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance; inputting at least one substitution model by using the countermeasure sample to obtain a target loss value; and when the target loss value does not meet the preset condition, updating the candidate countermeasure disturbance, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance at present as a model after training. According to the application, based on the countermeasure sample generation model meeting the actual application requirements, a large number of countermeasure samples meeting the requirements can be generated to evaluate the target detection model, so that the evaluation efficiency of the target detection model is higher and the model evaluation is more accurate.

Description

Model training method, related device and storage medium

Technical Field

The application relates to the technical field of artificial intelligent model training, in particular to a model training method, a related device and a storage medium.

Background

The existing model attack methods can be divided into two main categories according to the attack results: the first type is directed attack, and an attack sample can enable all recognition results to be target types when the model is predicted. The second type is non-directional attack, i.e. the attack sample can make the model be in error type when predicting all the recognition results.

The directional attack is mainly used for evaluating the target detection model based on deep learning, and the directional attack needs a great number of countermeasure samples to perform robustness evaluation on the target detection model.

At present, under the attack of a countersample, a model prediction error is easily caused by adding the countersample with a micro disturbance structure in the target detection model, and the countersample meeting the requirements cannot be obtained, so that the evaluation efficiency of the target detection model is low and the model evaluation is inaccurate.

Disclosure of Invention

The embodiment of the application provides a model training method, a related device and a storage medium, which can obtain an countermeasure sample generation model meeting the requirements of practical application, and can generate a large number of countermeasure samples meeting the requirements based on the countermeasure sample generation model meeting the requirements of practical application to evaluate a target detection model, so that the evaluation efficiency of the target detection model is higher and the model evaluation is more accurate.

In a first aspect, an embodiment of the present application provides a model training method, including:

selecting an attack category and attack logic;

constructing a model to be trained according to the attack logic, wherein the attack logic comprises the attack category;

inputting a preset sample picture and the attack category into a model to be trained, and generating candidate countermeasure disturbance;

obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance;

inputting at least one substitution model by using the countermeasure sample to obtain a target loss value;

and when the target loss value does not meet the preset condition, updating the candidate countermeasure disturbance, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance currently as a model after training.

In one embodiment of the present application, the selected attack class and attack logic comprises:

selecting an attack category;

acquiring initial attack logic aiming at the model to be trained;

and solving the optimal attack logic through a genetic algorithm and the initial attack logic to obtain the attack logic aiming at the model to be trained.

In one embodiment of the present application, the initial attack logic further includes an auxiliary semantic guidance category strongly related to the attack category semantic information, and a category sample constraint set belonging to the auxiliary semantic guidance category;

The obtaining initial attack logic for the model to be trained comprises the following steps:

determining an auxiliary semantic guidance category which is strongly related to the attack category semantic information through preset semantic priori knowledge;

calculating a category sample constraint set belonging to an auxiliary semantic guidance category through a preset semantic information extraction network;

and determining initial attack logic aiming at the model to be trained according to the attack category, the auxiliary semantic guidance category and the category sample constraint set.

In one embodiment of the present application, the constructing a model to be trained according to the attack logic includes:

embedding the attack category into a category mapping network for category mapping so as to embed the attack category into a model to be trained;

the determining, through preset semantic priori knowledge, an auxiliary semantic guidance category strongly related to the attack category semantic information includes:

inputting the sample picture into the category mapping network to perform category mapping, and outputting an implicit vector of a preset specific target;

inputting the sample picture into an initial model to be trained to obtain a sample vector;

and expanding the implicit vector and the sample vector along the height and width directions in the category mapping network so as to splice feature graphs of the implicit vector and the sample vector in a channel dimension to obtain an auxiliary semantic guidance category which is strongly related to the attack category semantic information.

In one embodiment of the present application, the inputting at least one surrogate model using the challenge sample results in a target loss value, comprising:

inputting the challenge sample into at least one surrogate model, resulting in a loss of the at least one surrogate model;

inputting the countermeasure sample and the category sample constraint set into the semantic feature extraction network, and outputting a first semantic feature and a second semantic feature;

calculating the distance between the first semantic feature and the second semantic feature, and taking the distance as a loss between sample constraints;

a target loss value is determined based on losses between the losses of the at least one surrogate model and the sample constraints.

In one embodiment of the present application, said determining a target loss value from a loss between the loss of the at least one surrogate model and the sample constraint comprises:

acquiring a first loss parameter preset by each substitution model in the at least one substitution model;

acquiring a second loss parameter preset by the semantic feature extraction network;

a target loss value is calculated based on the first loss parameter, the second loss parameter, the loss of the at least one surrogate model, and the loss between the sample constraints.

In a second aspect, an embodiment of the present application provides a method for generating an challenge sample, where the method uses a model trained by the model training method according to the first aspect to generate the challenge sample.

In a third aspect, an embodiment of the present application provides a model evaluation method for evaluating the robustness of a target detection model by generating a plurality of challenge samples using the opponent sample generation method described in the second aspect.

In a fourth aspect, an embodiment of the present application provides a computing device, including a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the model training method according to the first aspect, the challenge sample generation method according to the second aspect, and the model evaluation method according to the third aspect when executing the computer program.

In a fifth aspect, embodiments of the present application provide a computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the model training method as described in the first aspect, the challenge sample generation method as described in the second aspect, and the model evaluation method as described in the third aspect.

In a sixth aspect, embodiments of the present application provide a computer program product comprising instructions, the computer program product comprising program instructions which, when run on a computer or a processor, cause the computer or the processor to perform the model training method according to any of the first aspects.

In a seventh aspect, an embodiment of the present application provides a chip system, including:

a communication interface for inputting and/or outputting information;

a processor configured to execute a computer-executable program to cause a device on which the chip system is installed to execute the model training method according to the first aspect, the challenge sample generation method according to the second aspect, and the model evaluation method according to the third aspect.

In one possible design, the above chip system further includes a memory for holding program instructions and data necessary for the terminal. The chip system may be formed of a chip or may include a chip and other discrete devices.

In an eighth aspect, an embodiment of the present application provides a model training apparatus having a function of implementing a model training method corresponding to the above first aspect. The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above, which may be software and/or hardware.

In one embodiment of the present application, the model training apparatus includes:

the selecting module is used for selecting attack categories and attack logics;

the construction module is used for constructing a model to be trained according to the attack logic, wherein the attack logic comprises the attack category;

the first generation module is used for inputting a preset sample picture and the attack category into a model to be trained to generate candidate countermeasure disturbance;

the second generation module is used for obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance;

the calculation loss module is used for inputting at least one substitution model by using the countermeasure sample to obtain a target loss value;

and the determining module is used for updating the candidate countermeasure disturbance when the target loss value does not meet the preset condition, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance currently as a model after training.

In one embodiment of the application, the selected module is specifically configured to:

selecting an attack category;

acquiring initial attack logic aiming at the model to be trained; and solving the optimal attack logic through a genetic algorithm and the initial attack logic to obtain the attack logic aiming at the model to be trained.

the selected module is specifically configured to:

In one embodiment of the present application, the building block is specifically configured to:

the selected module is specifically configured to:

In one embodiment of the present application, the calculation loss module is specifically configured to:

the calculation loss module is specifically used for:

In one embodiment of the present application, the calculation loss module is further configured to:

Compared with the prior art, in the embodiment of the application, the target loss value is calculated based on at least one substitution model, whether the current countermeasure sample meets the requirement is checked based on the target loss value, if the current countermeasure sample does not meet the requirement, the candidate countermeasure disturbance is updated, and after the countermeasure sample meets the requirement, the model for generating the countermeasure sample at present is the trained model, therefore, the embodiment of the application introduces at least one substitution model of a third party to check whether the countermeasure sample meets the requirement, thereby indirectly detecting whether the model for generating the countermeasure sample at present is trained to the extent that the countermeasure sample meeting the requirement can be generated, and further, a large number of countermeasure samples meeting the requirement can be generated to evaluate the target detection model based on the countermeasure sample generation model meeting the practical application requirement, so that the evaluation efficiency of the target detection model is higher and the model evaluation is more accurate.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of a model training system provided by an embodiment of the present application;

FIG. 2 is a flow chart of one embodiment of a model training method provided in an embodiment of the present application;

FIG. 3 is an algorithmic schematic of the model training method provided in embodiments of the present application;

FIG. 4 is a schematic structural diagram of a model training apparatus according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a model training computing device according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a mobile phone according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

In the description that follows, embodiments of the application will be described with reference to steps and symbols performed by one or more computers, unless otherwise indicated. Thus, these steps and operations will be referred to in several instances as being performed by a computer, which as referred to herein performs operations that include processing units by the computer that represent electronic signals that represent data in a structured form. This operation transforms the data or maintains it in place in the computer's memory system, which may reconfigure or otherwise alter the computer's operation in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the foregoing text and are not meant to be limiting, and one skilled in the art will recognize that various steps and operations described below may also be implemented in hardware.

The term "module" or "unit" as used herein may be considered a software object executing on the computing system. The various components, modules, engines, and services described herein may be viewed as implementing objects on the computing system. The apparatus and methods described herein are preferably implemented in software, but may of course also be implemented in hardware, all within the scope of the application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.

The embodiment of the application provides a model training method, a related device and a storage medium.

Referring to fig. 1, fig. 1 is a schematic view of a scenario of a model training system provided by an embodiment of the present application, where the model training system may include a computing device 100, where the computing device 100 is connected through a network, and a model training apparatus is integrated in the computing device 100. In an embodiment of the present application, the computing device 100 may be a terminal device or a server.

In the embodiment of the present application, in the case where the computing device 100 is a server, the server may be an independent server, or may be a server network or a server cluster formed by servers, for example, a server described in the embodiment of the present application includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud server formed by a plurality of servers. Wherein the Cloud server is composed of a large number of computers or web servers based on Cloud Computing (Cloud Computing). In embodiments of the present application, communication between the server and the client may be accomplished by any means of communication, including, but not limited to, mobile communication based on the third generation partnership project (3rd Generation Partnership Project,3GPP), long term evolution (Long Term Evolution, LTE), worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wiMAX), or computer network communication based on the TCP/IP protocol family (TCP/IP Protocol Suite, TCP/IP), user datagram protocol (User Datagram Protocol, UDP), etc.

It will be appreciated that when the computing device 100 used in embodiments of the present application is a terminal device, the terminal device may be a device that includes both receive hardware and transmit hardware, i.e., a device having receive and transmit hardware capable of performing bi-directional communications over a bi-directional communication link. Such a terminal device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computing device 100 may be a desktop terminal or a mobile terminal, and the computing device 100 may be one of a mobile phone, a tablet computer, a notebook computer, and the like.

The terminal device according to the embodiment of the present application may also be a device that provides voice and/or data connectivity to a user, a handheld device with a wireless connection function, or other processing device connected to a wireless modem. Such as mobile telephones (or "cellular" telephones) and computers with mobile terminals, which can be portable, pocket, hand-held, computer-built-in or car-mounted mobile devices, for example, which exchange voice and/or data with radio access networks. For example, personal communication services (English full name: personal Communication Service, english short name: PCS) telephones, cordless telephones, session Initiation Protocol (SIP) phones, wireless local loop (Wireless Local Loop, english short name: WLL) stations, personal digital assistants (English full name: personal Digital Assistant, english short name: PDA) and the like.

Those skilled in the art will appreciate that the application environment shown in fig. 1 is merely an application scenario of the present application, and is not limited to the application scenario of the present application, and other application environments may further include more or fewer computing devices than those shown in fig. 1, or a network connection relationship of computing devices, for example, only 1 computing device is shown in fig. 1, and it is understood that the model training system may further include one or more other computing devices, or/and one or more other computing devices that are network connected to the computing device 100, and is not limited herein.

In addition, as shown in FIG. 1, the model training system may also include a memory 300 for storing data, such as sample pictures, anti-disturbance data, anti-sample data.

It should be noted that, the schematic view of the scenario of the model training system shown in fig. 1 is only an example, and the model training system and the scenario described in the embodiment of the present application are for more clearly describing the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application, and those skilled in the art can know that, with the evolution of the model training system and the appearance of a new service scenario, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

The scheme provided by the embodiment of the application relates to artificial intelligence (Artificial Intelligence, AI), computer Vision (CV), machine Learning (ML) and other technologies, and is specifically described by the following embodiments:

the AI is a theory, a method, a technology and an application system which simulate, extend and extend human intelligence by using a digital computer or a machine controlled by the digital computer, sense environment, acquire knowledge and acquire an optimal result by using the knowledge. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

AI technology is a comprehensive discipline, and relates to a wide range of technologies, both hardware and software. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

At present, AI technology is widely used in various fields such as man-machine interaction and safety protection. Because the input form of the machine learning algorithm is a numerical vector, an attacker can make misjudgment of the machine learning model by designing a specific numerical vector, which is the process of resisting the attack.

Attack methods can be divided into two main categories according to the attack results: the first type is directed attack, and an attack sample can enable all recognition results to be target types when the model is predicted. The second type is non-directional attack, i.e. the attack sample can make the model be in error type when predicting all the recognition results.

Directional attacks are more challenging than the one, which requires the detector to identify the target as a specific class of attack, which is more targeted. Object detection is a multi-task learning problem, and multiple tasks include foreground-background discrimination, multiple object discrimination, and object location recognition, object detection has demonstrated its ability to solve practical problems. Current challenge sample research for target detection scenarios, although numerous, is still mostly focused on the non-directional field.

In a related art, existing directed acyclic graphs (Directed Acyclic Graph, DAG) are the earliest proposed attack detection model algorithms that generate a number of missuggestion regions by manipulating candidate region extraction networks (Region Proposal Network, RPN) to attack the model. For a two-stage object detection model, the algorithm adopts a two-stage attack strategy. In a first phase, candidate regions that may contain targets are generated using the RPN network. In the second stage, the candidate region set obtained in the first stage is used for training and generating the countermeasure disturbance. The method assigns a randomly selected tag to each candidate region and then performs an iterative gradient attack.

The algorithm has high time cost and large consumption resources, the algorithm using gradient optimization attack can seriously depend on a substitution model, and the consideration and design of an integration mode are omitted in the algorithm, so that the migration capability is poor, and the attack capability is insufficient.

The embodiment of the application can generate a large number of countermeasure samples meeting the requirements based on the countermeasure sample generation model meeting the actual application requirements to evaluate the target detection model, so that the evaluation efficiency of the target detection model is higher and the model evaluation is more accurate, and the problem of insufficient attack capability in the prior art is solved.

The following describes in detail specific embodiments.

In the present embodiment, description will be made from the viewpoint of a model training method, which may be integrated in the computing device 100 in particular.

The application provides a model training method, which comprises the following steps: selecting an attack category and attack logic; constructing a model to be trained according to the attack logic, wherein the attack logic comprises the attack category; inputting a preset sample picture and the attack category into a model to be trained, and generating candidate countermeasure disturbance; obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance; inputting at least one substitution model by using the countermeasure sample to obtain a target loss value; and when the target loss value does not meet the preset condition, updating the candidate countermeasure disturbance, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance currently as a model after training.

Referring to fig. 2, a flowchart of an embodiment of a model training method according to an embodiment of the present application is shown, where the model training method includes the following steps 201 to 206:

201. An attack class and attack logic are selected.

The attack category may be a category in which the target object of the model is identified as a directional attack, for example, a red light category is an attack of a green light category, and the green light category is a selected attack category.

Specifically, the attack category can be selected according to the sample picture category and the actual requirement. For example: when the method is applied to a video monitoring system, in order to interfere with monitoring, when the sample picture is a person type, the attack type can be a cat type.

The attack logic is selected according to the attack category, the attack logic is an attack plan of a model to be trained, and the attack logic can comprise the attack category, an auxiliary semantic guidance category which is strongly related to the attack category semantic information and a category sample constraint set which belongs to the auxiliary semantic guidance category.

202. And constructing a model to be trained according to the attack logic.

Wherein the model to be trained is a model for generating an countermeasure sample, and the model to be trained can be a generation network _。

203. And inputting a preset sample picture and the attack category into a model to be trained, and generating candidate countermeasure disturbance.

Wherein the sample picture may be represented by a vector of pixel integration. In particular, the sample pictures may be from video frames, such as: the sample picture may be a portrait picture in video surveillance, and the specific source of the sample picture is not limited herein.

After the countermeasures are added to the sample picture, the neural network recognizes the sample picture as an attack category, and the attack category and the sample picture are different in category, namely, the countermeasures are added to the sample picture, so that the neural network generates misjudgment.

In particular, the disturbance countermeasure may be added to each pixel unit of the sample picture, or may be added to several pixel units of the sample picture, where the specific manner of adding the disturbance countermeasure is not limited. In addition, the candidate challenge disturbance is not the final challenge disturbance.

204. And obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance.

Specifically, the challenge sample is obtained using a text-Image Pre-training model (Contrastive Language-Image Pre-training, CLIP) and a mapping network. Wherein, the class mapping network processing by the CLIP is helpful to eliminate illegal values in the disturbance countermeasure, and the success rate of the sample attack countermeasure is improved.

205. And inputting at least one substitution model by using the countermeasure sample to obtain a target loss value.

Wherein the surrogate model may be one or more, in one particular embodiment, the surrogate model may comprise model M1, model M2, and/or model M3, wherein m1=yolv3, m2=fasterrcnn, m3=detr, one or more of them, or a combination thereof, without limitation herein.

206. And when the target loss value does not meet the preset condition, updating the candidate countermeasure disturbance, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance currently as a model after training.

Specifically, when a target loss value is calculated, comparing the target loss value with a preset loss value, when the target loss value does not meet a preset condition, continuing to input a sample picture and an attack category, iteratively updating candidate countermeasure disturbance, recalculating the target loss value according to the updated candidate countermeasure disturbance, and comparing the updated target loss value with the preset loss value until the target loss value meets the preset condition, and taking a to-be-trained model of the candidate countermeasure disturbance generated currently as a trained model.

According to the method and the device, the target loss value is calculated based on at least one substitution model, whether the current countermeasure sample meets the requirement or not is checked based on the target loss value, if the current countermeasure sample does not meet the requirement, candidate countermeasure disturbance is updated, and after the countermeasure sample meets the requirement, the model which is generated at present and is used for generating the countermeasure sample is the trained model, so that at least one substitution model of a third party is introduced to check whether the countermeasure sample is required or not, whether the model which is generated at present and is used for generating the countermeasure sample meeting the requirement is indirectly detected, a countermeasure sample generating model meeting the requirement of practical application can be obtained, and further, a large number of countermeasure samples meeting the requirement can be generated based on the countermeasure sample generating model meeting the requirement to evaluate the target detection model, so that the evaluation efficiency of the target detection model is higher and the model evaluation is more accurate.

According to the embodiment of the application, based on the challenge sample generation model meeting the actual application requirements, the candidate challenge disturbance is updated continuously, so that the challenge sample meeting the preset standard can be updated and generated, the training model capable of generating the challenge sample meeting the requirements is obtained, and the attack success rate of the challenge sample can be improved by the training model, so that the robustness of the model is evaluated.

In one embodiment of the present application, the step 201 selects attack categories and attack logic, including:

selecting an attack category, acquiring initial attack logic aiming at the model to be trained, and solving the optimal attack logic through a genetic algorithm and the initial attack logic to obtain the attack logic aiming at the model to be trained. Wherein the initial attack logic is determined according to attack categories, and the attack logic comprises attack categories.

Genetic algorithms, also known as Genetic Algorithm (GA). The main idea is to simulate the inheritance and variation of organisms. It is very versatile and can be used to accelerate certain algorithms that maximize or minimize.

According to the embodiment of the application, the optimal attack logic can be obtained by solving the optimal attack logic through the genetic algorithm and the initial attack logic, so that the aggressiveness is enhanced.

In one embodiment of the present application, the initial attack logic further includes a secondary semantic guidance category strongly correlated with the attack category semantic information, and a category sample constraint set belonging to the secondary semantic guidance category.

Specifically, the initial attack logic comprises an attack category, an auxiliary semantic guidance category strongly related to the attack category semantic information, and a category sample constraint set belonging to the auxiliary semantic guidance category.

At this time, the acquiring initial attack logic for the model to be trained includes:

and determining an auxiliary semantic guidance category which is strongly related to the attack category semantic information through preset semantic priori knowledge. Calculating a category sample constraint set belonging to an auxiliary semantic guidance category through a preset semantic information extraction network; and determining initial attack logic aiming at the model to be trained according to the attack category, the auxiliary semantic guidance category and the category sample constraint set.

The semantic priori knowledge is integrated from a generalized data set and human priori information, and is used for guiding information which can attack a semantic layer on disturbance rejection.

Specifically, the auxiliary semantic guidance category strongly related to the attack category semantic information is obtained by analyzing the scene where the attack category appears and scene category information. Namely, by analyzing the scene of the attack category, and combining preset semantic priori knowledge, the auxiliary semantic guidance category which is strongly related to the attack category semantic information is obtained. For example: when the attack category is a tree category, the scene in which the tree category appears is analyzed through semantic priori knowledge, the tree is usually rooted in the soil, and then the auxiliary semantic guidance category which is strongly related to the semantic information of the attack category can be a soil category.

The auxiliary semantic guidance category and the category sample constraint set are knowledge complementation with different dimensions, and the auxiliary semantic guidance category and the category sample constraint set jointly provide category layer guidance and semantic layer guidance.

In the embodiment of the present application, an attack plan is initialized, for example, the attack plan P includes predefined attack categoriesAnd auxiliary semantic guidance category->Class sample constraint set +.>

Before an attack, the type of the attack is defined to pass the attack typeThe attack category is used as the input for constructing a semantic priori information guiding algorithm, and the algorithm outputs three categories which are strongly related to semantic information> Therefore, attack category->Is performed by a pre-set advanced semantic information extraction network ++>Calculate belonging to->High confidence sample set of categories-> And->And->Is a knowledge complement of different dimensions and is mutually constrained. />And->Together, category layer guidance and semantic layer guidance are provided.

The embodiment of the application improves the aggressiveness of the semantic aspect by utilizing the semantic information through semantic priori knowledge and auxiliary semantic guidance categories which are strongly related to the semantic information of the attack category.

In one embodiment of the present application, the step 202 builds the model to be trained according to the attack logic, including: and embedding the attack category into a category mapping network for category mapping so as to embed the attack category into a model to be trained. Wherein the class mapping network is embedded in the training method.

Further, the determining, through preset semantic priori knowledge, an auxiliary semantic guidance category strongly related to the attack category semantic information includes: and inputting the sample picture into the category mapping network to perform category mapping, and outputting an implicit vector of a preset specific target.

Specifically, before the sample picture flows from the encoding end to the decoding end, the class mapping network embeds attack class information into the sample picture through the class mapping network by introducing attack class of directional attack at the encoding end. Inputting the sample picture into an initial model to be trained to obtain a sample vector; and expanding the implicit vector and the sample vector along the height and width directions in the category mapping network so as to splice feature graphs of the implicit vector and the sample vector in a channel dimension to obtain an auxiliary semantic guidance category which is strongly related to the attack category semantic information.

Specifically, the class mapping network is operated with advanced convolution to move the image within a small amplitude.

According to the embodiment of the application, the semantic information is input into the subsequent feature map through category embedding, so that the success rate of attack can be improved.

In one embodiment of the present application, the step 205 of inputting at least one surrogate model using the challenge sample to obtain a target loss value includes: inputting the challenge sample into at least one surrogate model, resulting in a loss of the at least one surrogate model.

The surrogate model is a model for outputting a prediction result, that is, after inputting a challenge sample, the surrogate model outputs a prediction category, for example, an attack category is selected as a cat category, and a sample picture is a person category, then the challenge sample should be a sample which enables the target detection model to be output as the cat category, the challenge sample is input into the surrogate model, the surrogate model outputs the target detection result, the prediction result of the surrogate model is obtained, and a difference value between the prediction result and a preset output result is calculated, and the obtained difference value is a loss value. After the loss value is calculated, the loss value can be made to be the minimum value as much as possible through iteration, gradient descent and the like, so that the attack logic is continuously optimized.

Wherein the prediction result of the substitution model comprises the class of the challenge sample and coordinates.

Inputting the countermeasure sample and the category sample constraint set into the semantic feature extraction network, and outputting a first semantic feature and a second semantic feature; calculating the distance between the first semantic feature and the second semantic feature, and taking the distance as a loss between sample constraints; a target loss value is determined based on losses between the losses of the at least one surrogate model and the sample constraints.

The semantic features output by the antagonism sample input semantic feature extraction network are taken as first semantic features, and the semantic features output by the category sample constraint set input semantic feature extraction network are taken as second semantic features.

Specifically, for example, the replacement model may be m1=yolov3, m2=fasterrcnn, m3=detr, and the loss value calculation formula of the entire replacement model is:

/>

wherein,all alpha _i The values of (a) are all greater than 0, wherein alpha ₁ For the weight of model M1, α ₂ For the weight of model M2, α ₃ For the weight of model M3, δ is the perturbation of the directed attack, +.> Model M1, model M2, model M3 and semantic feature extraction network respectively +. >Is a loss value of (2).

According to the embodiment of the application, the loss value of the semantic layer is constrained by constraining the loss value of the substitution model and calculating the loss value of the semantic layer, so that the information of the semantic layer can be attacked, the loss value is constrained to be as small as possible, the attack success rate can be improved, and the attack is enhanced.

In one embodiment of the present application, said determining a target loss value from a loss between the loss of the at least one surrogate model and the sample constraint comprises: acquiring a first loss parameter preset by each substitution model in the at least one substitution model; acquiring a second loss parameter preset by the semantic feature extraction network; a target loss value is calculated based on the first loss parameter, the second loss parameter, the loss of the at least one surrogate model, and the loss between the sample constraints. Wherein the first loss parameter and the second loss parameter are obtained and can be used for obtaining formula parameters for generating an countermeasure sample.

According to the embodiment of the application, the loss parameters are acquired, so that the formula parameters for generating the countermeasure sample can be acquired, a specific generation formula of the countermeasure sample is acquired, and the efficiency of generating the countermeasure sample is improved.

According to the target detection and directional attack method utilizing semantic priori information, semantic priori knowledge integrated from a generalized data set and human priori information is utilized, and information on a semantic layer can be attacked by utilizing the knowledge to guide scrambling. The attack method adopts a method which mainly aims at the target class of the directional attack and is other in line with the scene class as an auxiliary, the directional attack is carried out according to the scene information, the attack method does not adopt a simple generation type countermeasure method, but considers attack disturbance of different classes, and the generation type countermeasure network which embeds multi-class vectors by utilizing a condition is utilized to generate the directional disturbance, so that a victim model recognizes the attacked target as the class of the directional attack. The method can greatly improve the type of the attack, improve the speed and the efficiency of the attack, and enable an attacker to select different types to attack by using the method of embedding the type. In training, the migration capability of disturbance among a plurality of models can be increased by an integrated training method of the plurality of models.

In the embodiment of the application, the focus is on how to find the optimal attack plan, and the current detection model is more focused on the whole semantic combination, so if the categories in the data set are to be Target attack of (2) is category->The analysis category +.>Scene and scene category information. By analyzing the information of the scene, a class combination of disturbance attacks can be constructed. The classes of attacks include the class of directed attacks and the class of assisted attacks that involve the scene. At this time, an attack plan is specified through semantic priori information, and a category sample constraint set is searched through different attack plans, wherein the constraint set can provide a high-level semantic information guide and can constrain generated disturbance to be close to the directed attack category semantics. The semantic guidance method constrains semantic differences of the challenge sample and the attack class auxiliary attack class through a semantic extraction network.

In the embodiment of the application, the constraint of semantic information is not performed before the final attack plan P is determined, and the sample constraint of the semantic layer is performed only after the optimal attack plan is determined. Inputting an original imageDirectional attack category->Attack plan P. Original image->And the decoding end flows from the encoding end of the generating network. Directional attack categoryFirstly, entering an embedded class mapping network W for class mapping, wherein the network can make +.>Embedded into the generation network. At the encoding end, the target class of the directed attack is introduced first >Outputting implicit vector of specific object using mapping network W>In the mapping network W the vectors will be spread in the height and width directions, will +.>The feature graphs of the map are spliced in the channel dimension, so that category information with strong correlation can be obtained. The method can embed the category information into the feature map by splicing the feature vector of the picture with the feature vector of the target label. At this time, inputting the feature map with the category information into the subsequent network to generate a network to recover the resolution of the feature map by using the decoding end, and using +.>Generating a network output>An operation that helps to reject illegal values in the disturbance. And the Gaussian convolution operation is used for moving the image in a small amplitude, so that the success rate of attack is improved:

wherein,to output the result, n and m represent the number of rows and columns of pixels, ε is a preset value, m _ij For the ith row, the jth column pixel value,/->Is a gaussian convolution operation.

Through the above process, the generated challenge samples for the network are represented as follows:

will obtain an challenge sampleInputting a substitution model M, wherein the substitution model outputs a prediction result including the category +.>Coordinate information->At this time, a loss function of the substitution model in the plan is calculated, and an optimal attack plan is obtained by a genetic algorithm.

The attack plan P with the lowest loss function of the substitution model can be obtained through multiple iterations, and the obtained countermeasure sampleInput semantic feature extraction network->Output semantic feature +.>(first semantic feature above). Sample constraint set defined in attack plan P +.>Input semantic feature extraction network->Output semantic feature +.>(second semantic features above). Algorithm constraint semantic feature->And semantic feature->The distance between the two parts is guided by the semantic level, so that the attack is enhanced.

The application solves the following optimization problems to solve the parameters of the generated model:

wherein the method comprises the steps ofRepresenting a set of targeted attack categories, +.>Category information representing surrogate model predictions. In generating network->In reasoning, input image and directed attack category +.>Disturbance delta of network output directed attack and +.>Adding the desired result of the directed attack +.>

Specifically, referring to fig. 3, fig. 3 is a flowchart of a specific algorithm of the model training method.

The embodiment of the application also provides a method for generating the countermeasure sample based on the model training method, which comprises the following steps: the challenge sample is generated using the model obtained by training using the model training method in any of the embodiments described above.

According to the embodiment of the application, the challenge sample meeting the actual application requirement is generated through the trained model, so that the actual application rate of the challenge sample can be improved, and the aggressiveness of the challenge sample is improved.

The embodiment of the application also provides a model evaluation method, which comprises the step of evaluating the robustness of the target detection model by utilizing a plurality of countermeasure samples generated by the countermeasure sample generation method.

According to the embodiment of the application, a large number of countermeasure samples meeting the requirements can be generated through the trained model to evaluate the target detection model, so that the evaluation efficiency of the target detection model is higher, the model evaluation is more accurate, and the robustness of the model is improved.

In order to facilitate better implementation of the model training method provided by the embodiment of the application, the embodiment of the application also provides a device based on the model training method. Wherein the meaning of the nouns is the same as that in the model training method, and specific implementation details can be referred to the description in the embodiment of the model training method.

The model training device in the embodiment of the application has the function of realizing the model training method corresponding to the embodiment. The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above, which may be software and/or hardware.

Referring to fig. 4, fig. 4 is a schematic structural diagram of a model training apparatus provided by an embodiment of the present application, where the model training apparatus may be applied to a computing device in a scene where model training is required, and specifically, the model training apparatus 400 may include a selection module 401, a construction module 402, a first generation module 403, a second generation module 404, a calculation loss module 405, and a determination module 406, which are specifically as follows:

a selection module 401 for selecting an attack category and attack logic.

A construction module 402, configured to construct a model to be trained according to the attack logic, where the attack logic includes the attack category.

The first generation module 403 is configured to input a preset sample picture and the attack category into a model to be trained, and generate candidate countermeasure disturbance.

And a second generating module 404, configured to obtain a countermeasure sample according to the sample picture and the candidate countermeasure disturbance.

A calculation loss module 405, configured to input at least one surrogate model using the challenge sample to obtain a target loss value.

And the determining module 406 is configured to update the candidate countermeasure disturbance when the target loss value does not meet the preset condition, and redetermine the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and take a model to be trained, which is currently generated as a model after training.

In one embodiment of the present application, the selecting module 401 is specifically configured to:

selecting an attack category;

the selecting module 401 is specifically configured to:

In one embodiment of the present application, the construction module 402 is specifically configured to:

The selecting module 401 is specifically configured to:

In one embodiment of the present application, the calculation loss module 405 is specifically configured to:

the calculation loss module 405 specifically is configured to:

In one embodiment of the present application, the calculation loss module 405 is further configured to:

In the embodiment of the application, the attack category and the attack logic are selected by the selection module 401; constructing a model to be trained according to the attack logic by a construction module 402, wherein the attack logic comprises the attack category; inputting a preset sample picture and the attack category into a model to be trained through a first generation module 403, and generating candidate countermeasure disturbance; obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance through a second generation module 404; the calculation loss module 405 inputs at least one substitution model by using the countermeasure sample to obtain a target loss value, the determining module 406 updates the candidate countermeasure disturbance when the target loss value does not meet a preset condition, and redetermines the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and the model to be trained which currently generates the candidate countermeasure disturbance is used as a trained model. According to the method and the device, the target loss value is calculated based on at least one substitution model, whether the current countermeasure sample meets the requirement or not is checked based on the target loss value, if the current countermeasure sample does not meet the requirement, the candidate countermeasure disturbance is updated, and after the countermeasure sample meets the requirement, the model which is currently generated as the countermeasure sample is the trained model.

The model training apparatus in the embodiment of the present application is described above from the point of view of the modularized functional entity, and the model training apparatus in the embodiment of the present application is described below from the point of view of hardware processing, respectively.

It should be noted that, the physical devices corresponding to the first generating module 403 and the second generating module 404 shown in fig. 4 may be a transceiver, a radio frequency circuit, a communication module, an input/output (I/O) interface, etc., and the physical device corresponding to the determining module 406 may be a processor.

The apparatus shown in fig. 4 may have a structure as shown in fig. 5, and when the model training apparatus shown in fig. 5 has a structure as shown in fig. 5, the processor and the transceiver in fig. 5 can implement the same or similar functions as the determining module 406, the first generating module 403, and the second generating module 404 provided in the foregoing apparatus embodiment corresponding to the apparatus, and the memory in fig. 5 stores a computer program that needs to be invoked when the processor executes the foregoing model training method.

When the computing device in the embodiment of the present application is a terminal device, the embodiment of the present application further provides a terminal device, as shown in fig. 6, for convenience of explanation, only the portion related to the embodiment of the present application is shown, and specific technical details are not disclosed, please refer to the method portion of the embodiment of the present application. The terminal device may be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), a Point of Sales (POS), a vehicle-mounted computer, and the like, taking the terminal device as an example of the mobile phone:

Fig. 6 is a block diagram showing a part of the structure of a mobile phone related to a terminal device provided by an embodiment of the present application. Referring to fig. 6, the mobile phone includes: radio Frequency (RF) circuitry 1010, memory 1020, input unit 1030, display unit 1040, sensor 1050, audio circuitry 1060, wireless fidelity (wireless fidelity, wiFi) module 1070, processor 1080, and power source 1090. Those skilled in the art will appreciate that the handset configuration shown in fig. 6 is not limiting of the handset and may include more or fewer components than shown, or may combine certain components, or may be arranged in a different arrangement of components.

The following describes the components of the mobile phone in detail with reference to fig. 6:

the RF circuit 1010 may be used for receiving and transmitting signals during a message or a call, and particularly, after receiving downlink information of a base station, the signal is processed by the processor 1080; in addition, the data of the design uplink is sent to the base station. Generally, RF circuitry 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low noise amplifier (Low NoiseAmplifier, LNA), a duplexer, and the like. In addition, the RF circuitry 1010 may also communicate with networks and other devices via wireless communications. The wireless communications may use any communication standard or protocol including, but not limited to, global system for mobile communications (GlobalSystem of Mobile communication, GSM), general Packet radio service (General Packet RadioService, GPRS), code division multiple access (Code Division Multiple Access, CDMA), wideband code division multiple access (Wideband Code Division Multiple Access, WCDMA), long term evolution (Long Term Evolution, LTE), email, short message service (Short Messaging Service, SMS), and the like.

The memory 1020 may be used to store software programs and modules that the processor 1080 performs various functional applications and data processing of the handset by executing the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 1020 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state memory device.

The input unit 1030 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the handset. In particular, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1031 or thereabout using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a predetermined program. Alternatively, the touch panel 1031 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 1080 and can receive commands from the processor 1080 and execute them. Further, the touch panel 1031 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 1030 may include other input devices 1032 in addition to the touch panel 1031. In particular, other input devices 1032 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a track ball, a mouse, a joystick, etc.

The display unit 1040 may be used to display information input by a user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041, and alternatively, the display panel 1041 may be configured in the form of a Liquid crystal display (Liquid CrystalDisplay, LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 1031 may overlay the display panel 1041, and when the touch panel 1031 detects a touch operation thereon or thereabout, the touch panel is transferred to the processor 1080 to determine a type of touch event, and then the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in fig. 6, the touch panel 1031 and the display panel 1041 are two independent components to implement the input and input functions of the mobile phone, in some embodiments, the touch panel 1031 and the display panel 1041 may be integrated to implement the input and output functions of the mobile phone.

The handset may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 1041 and/or the backlight when the mobile phone moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the handset are not described in detail herein.

Audio circuitry 1060, a speaker 1061, and a microphone 1062 may provide an audio interface between a user and a cell phone. Audio circuit 1060 may transmit the received electrical signal after audio data conversion to speaker 1061 for conversion by speaker 1061 into an audio signal output; on the other hand, microphone 1062 converts the collected sound signals into electrical signals, which are received by audio circuit 1060 and converted into audio data, which are processed by audio data output processor 1080 for transmission to, for example, another cell phone via RF circuit 1010 or for output to memory 1020 for further processing.

Wi-Fi belongs to a short-distance wireless transmission technology, and a mobile phone can help a user to send and receive e-mails, browse web pages, access streaming media and the like through a Wi-Fi module 1070, so that wireless broadband Internet access is provided for the user. Although fig. 6 shows Wi-Fi module 1070, it is understood that it does not belong to the necessary constitution of the handset, and can be omitted entirely as required within the scope of not changing the essence of the invention.

Processor 1080 is the control center of the handset, connects the various parts of the entire handset using various interfaces and lines, and performs various functions and processes of the handset by running or executing software programs and/or modules stored in memory 1020, and invoking data stored in memory 1020, thereby performing overall monitoring of the handset. Optionally, processor 1080 may include one or more processing units; alternatively, processor 1080 may integrate an application processor primarily handling operating systems, user interfaces, applications, etc., with a modem processor primarily handling wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 1080.

The handset further includes a power source 1090 (e.g., a battery) for powering the various components, optionally in logical communication with the processor 1080 via a power management system, such as for managing charge, discharge, and power consumption by the power management system.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which will not be described herein.

In an embodiment of the present application, the processor 1080 included in the mobile phone further has a control unit for executing the above model training method executed by the model training device.

Referring to fig. 7, fig. 7 is a schematic diagram of a server structure according to an embodiment of the present application, where the server 1100 may have a relatively large difference due to different configurations or performances, and may include one or more central processing units (in english: central processing units, in english: CPU) 1122 (for example, one or more processors) and a memory 1132, and one or more storage media 1130 (for example, one or more mass storage devices) storing application programs 1142 or data 1144. Wherein the memory 1132 and the storage medium 1130 may be transitory or persistent. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 1122 may be provided in communication with a storage medium 1130, executing a series of instruction operations in the storage medium 1130 on the server 1100.

The Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158, and/or one or more operating systems 1141, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like.

The steps in the model training method in the above embodiment may be based on the structure of the server 1100 shown in fig. 7. For example, the CPU 1122 may perform the following operations by calling instructions in the memory 1132:

selecting an attack category and attack logic; constructing a model to be trained according to the attack logic, wherein the attack logic comprises the attack category; inputting a preset sample picture and the attack category into a model to be trained through an input-output interface 1158, and generating candidate countermeasure disturbance; obtaining a countermeasure sample according to the sample picture and the candidate countermeasure disturbance; inputting at least one substitution model by using the countermeasure sample to obtain a target loss value; and when the target loss value does not meet the preset condition, updating the candidate countermeasure disturbance, and re-determining the target loss value according to the candidate countermeasure disturbance until the target loss value meets the preset condition, and taking the model to be trained for generating the candidate countermeasure disturbance currently as a model after training.

In one embodiment, the selected attack category and attack logic may also select an attack category via the central processor 1122; acquiring initial attack logic aiming at the model to be trained; and solving the optimal attack logic through a genetic algorithm and the initial attack logic to obtain the attack logic aiming at the model to be trained.

In one embodiment, the initial attack logic further comprises an auxiliary semantic guidance category strongly correlated with the attack category semantic information, and a category sample constraint set belonging to the auxiliary semantic guidance category; the initial attack logic for the model to be trained is obtained, and an auxiliary semantic guidance category strongly related to the attack category semantic information can be determined through a central processor 1122 through preset semantic priori knowledge; calculating a category sample constraint set belonging to an auxiliary semantic guidance category through a preset semantic information extraction network; and determining initial attack logic aiming at the model to be trained according to the attack category, the auxiliary semantic guidance category and the category sample constraint set.

In one embodiment, the model to be trained is constructed according to the attack logic, and the attack category can be embedded into a category mapping network through the central processor 1122 to perform category mapping, so as to embed the attack category into the model to be trained; the auxiliary semantic guidance category strongly related to the attack category semantic information is determined through the preset semantic priori knowledge, and the sample picture can be input into the category mapping network through an input/output interface 1158 to perform category mapping, so as to output an implicit vector of a preset specific target; inputting the sample picture into an initial model to be trained to obtain a sample vector; and expanding the implicit vector and the sample vector along the height and width directions in the category mapping network so as to splice feature graphs of the implicit vector and the sample vector in a channel dimension to obtain an auxiliary semantic guidance category which is strongly related to the attack category semantic information.

In one embodiment, the target loss value is obtained by inputting the challenge sample into at least one surrogate model, and the loss of the at least one surrogate model is obtained by inputting the challenge sample into the at least one surrogate model through the input-output interface 1158; inputting the countermeasure sample and the category sample constraint set into the semantic feature extraction network, and outputting a first semantic feature and a second semantic feature; calculating the distance between the first semantic feature and the second semantic feature, and taking the distance as a loss between sample constraints; a target loss value is determined based on losses between the losses of the at least one surrogate model and the sample constraints.

In one embodiment, the determining the target loss value according to the loss between the loss of the at least one surrogate model and the sample constraint may further obtain a first loss parameter preset by each surrogate model in the at least one surrogate model through the input-output interface 1158; acquiring a second loss parameter preset by the semantic feature extraction network; a target loss value is calculated based on the first loss parameter, the second loss parameter, the loss of the at least one surrogate model, and the loss between the sample constraints.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, apparatuses and modules described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.

In the embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the flow or functions according to the embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

The above description has been made in detail on the technical solutions provided by the embodiments of the present application, and specific examples are applied in the embodiments of the present application to illustrate the principles and implementation manners of the embodiments of the present application, where the above description of the embodiments is only for helping to understand the methods and core ideas of the embodiments of the present application; meanwhile, as for those skilled in the art, according to the idea of the embodiment of the present application, there are various changes in the specific implementation and application scope, and in summary, the present disclosure should not be construed as limiting the embodiment of the present application.

Claims

1. A method of model training, the method comprising:

selecting an attack category and attack logic;

2. The model training method of claim 1, wherein the selected attack class and attack logic comprises:

selecting an attack category;

acquiring initial attack logic aiming at the model to be trained;

3. The model training method of claim 2, wherein the initial attack logic further comprises an auxiliary semantic guidance category strongly correlated with the attack category semantic information, and a category sample constraint set belonging to the auxiliary semantic guidance category;

4. A model training method according to claim 3, characterized in that said constructing a model to be trained from said attack logic comprises:

5. The model training method of claim 1, wherein said inputting at least one surrogate model with the challenge sample results in a target loss value comprising:

6. The model training method of claim 5, wherein the determining a target loss value from the loss between the loss of the at least one surrogate model and the sample constraint comprises:

7. A method of generating a challenge sample, the method comprising:

model trained using the model training method according to any one of claims 1 to 6, to generate an challenge sample.

8. A method of model evaluation, the method comprising:

the robustness of the target detection model is evaluated using a plurality of challenge samples generated by the challenge sample generation method as set forth in claim 7.

9. A model training apparatus, the apparatus comprising:

the selecting module is used for selecting attack categories and attack logics;

10. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 8.