CN112735550B - Neural network robustness rapid verification method, system, device and storage medium - Google Patents

Neural network robustness rapid verification method, system, device and storage medium Download PDF

Info

Publication number
CN112735550B
CN112735550B CN202110038415.8A CN202110038415A CN112735550B CN 112735550 B CN112735550 B CN 112735550B CN 202110038415 A CN202110038415 A CN 202110038415A CN 112735550 B CN112735550 B CN 112735550B
Authority
CN
China
Prior art keywords
neural network
input set
shell
physical examination
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110038415.8A
Other languages
Chinese (zh)
Other versions
CN112735550A (en
Inventor
郭山清
唐朋
张云若
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202110038415.8A priority Critical patent/CN112735550B/en
Publication of CN112735550A publication Critical patent/CN112735550A/en
Application granted granted Critical
Publication of CN112735550B publication Critical patent/CN112735550B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention discloses a method, a system, equipment and a storage medium for quickly verifying robustness of a neural network, wherein the method comprises the following steps: selecting physical examination data of healthy users, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing; carrying out segmentation processing on the input set to obtain a shell of the input set; taking the shell of the input set as an input value of the neural network to be verified, and solving output reachable set estimation corresponding to the shell of the input set; judging whether the output reachable set estimation meets the output limit; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained; training the neural network with a safe verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels; inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.

Description

Neural network robustness rapid verification method, system, device and storage medium
Technical Field
The present application relates to the field of neural network security technologies, and in particular, to a method, a system, a device, and a storage medium for fast verification of neural network robustness.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
The neural network achieves higher accuracy rate in the scenes of auxiliary diagnosis and treatment, health management, disease risk prediction and the like. However, neural networks are very sensitive to perturbations. When the normal data is slightly disturbed, the neural network may wrongly classify the normal data, and although the doctor does not influence his judgment, the disturbance of the normal data causes potential safety hazards in the practical use of the neural network; for example, the health examination result is judged, and the unhealthy examination result is output in error only because the robustness of the neural network is insufficient; this is not sufficient for the medical field to classify medical data by a neural network without robustness.
Therefore, the robustness of the neural network is generally verified before the neural network is actually applied, so as to improve the credibility of the neural network. The robustness can ensure that the classification of all the inputs in a certain neighborhood of normal inputs is the same, and the neural network with the robustness can effectively resist the misclassification caused by the disturbance. However, in the research of the existing verification method, when a type of neural network satisfying that the node number of each layer is not more than that of the previous layer is verified, the algorithms have redundant calculated amount.
The inventors have found that there is a lack in the prior art of a method and system for quickly verifying the robustness of a medical classification neural network.
Disclosure of Invention
In order to solve the defects of the prior art, the application provides a method, a system, equipment and a storage medium for quickly verifying the robustness of a neural network;
in a first aspect, the application provides a neural network robustness rapid verification method;
the neural network robustness rapid verification method comprises the following steps:
selecting physical examination data of a healthy user, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing;
segmenting the input set to obtain a shell of the input set;
taking the shell of the input set as an input value of the neural network to be verified, and solving output reachable set estimation corresponding to the shell of the input set;
judging whether the output reachable set estimation meets the output limit or not; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained;
training the neural network with a safe verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels;
inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.
In a second aspect, the application provides a neural network robustness rapid verification system;
a neural network robustness rapid verification system comprises:
a selection module configured to: selecting physical examination data of healthy users, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing;
a segmentation module configured to: carrying out segmentation processing on the input set to obtain a shell of the input set;
a solving module configured to: taking the shell of the input set as an input value of the neural network to be verified, and solving an output reachable set estimation corresponding to the shell of the input set;
a determination module configured to: judging whether the output reachable set estimation meets the output limit or not; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained;
a training module configured to: training the neural network with a safe verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels;
an output module configured to: inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.
In a third aspect, the present application further provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein a processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so as to make the electronic device execute the method according to the first aspect.
In a fourth aspect, the present application also provides a computer-readable storage medium for storing computer instructions which, when executed by a processor, perform the method of the first aspect.
In a fifth aspect, the present application also provides a computer program (product) comprising a computer program for implementing the method of any of the preceding first aspects when run on one or more processors.
Compared with the prior art, the beneficial effect of this application is:
the method and the device have the advantages of parallel computing and distributed computing, the speed of the robustness verification process of the neural network is higher, the verification result is more accurate, the verified neural network used for medical data classification is higher in robustness, and disturbance caused by data interference can be resisted. The output of wrong data due to the insufficient robustness of the neural network can be effectively avoided.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the application, and the description of the exemplary embodiments and illustrations of the application are intended to explain the application and are not intended to limit the application.
FIG. 1 is a flow chart of a method of the first embodiment;
FIG. 2 is a set of inputs for squares of the first embodiment;
FIG. 3 is a schematic diagram of the first embodiment of the partitioning of an input set into 25 partitions;
fig. 4 is a schematic structural diagram of the neural network of the first embodiment.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" and "comprising", and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example one
The embodiment provides a neural network robustness rapid verification method;
as shown in fig. 1, the method for quickly verifying the robustness of a neural network includes:
step (1): selecting physical examination data of healthy users, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing;
step (2): segmenting the input set to obtain a shell of the input set;
and (3): taking the shell of the input set as an input value of the neural network to be verified, and solving output reachable set estimation corresponding to the shell of the input set;
and (4): judging whether the output reachable set estimation meets the output limit or not; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained;
and (5): training the neural network with a safe verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels;
and (6): inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.
It will be appreciated that verifying the (local) robustness of a neural network is to verify that the classification of samples in a certain neighborhood of normal inputs is the same. And this neighborhood is the input set.
Illustratively, the normalizing the physical examination data includes:
suppose that normal user data includes k total of height, weight, blood pressure, etc 0 Data, regarding the data as a k 0 Dimension vector, as the center of the neighborhood, denoted c * The jth component thereof is denoted as
Figure BDA0002894318520000051
Hypothesis->
Figure BDA0002894318520000052
Is->
Figure BDA0002894318520000053
Then the normalized value is £ er>
Figure BDA0002894318520000054
From all c j (j＝1,2,…,k 0 ) The vector c formed is c * And (5) normalizing the result.
Illustratively, the setting an error operation specifically includes:
assuming that the allowable error size of the present application is e, then for data item c j (j＝1,2,…,k 0 ) In any of j ＝c j E and u j ＝c j A value between + eAre allowed. From k to k 0 K is formed by e 0 The dimension vector is denoted as r.
Illustratively, an input set is generated according to the normalized user physical examination data and the error; the method comprises the following specific steps:
the input set is a hyper-rectangle determined by | X-c ≦ r, marked as X, and the hyper-rectangle can also be written as [ l, u ≦ r]L and u are each independently of the other j And u j (j＝1,2,…,k 0 ) The vectors are formed, which are the two vertices of the X that are the lowest and highest. The hyper-rectangle is the popularization of the rectangle in a high-dimensional space, and the inequality forms and the interval forms of the hyper-rectangle are in one-to-one correspondence.
Further, the step (2): carrying out segmentation processing on the input set; the method comprises the following specific steps:
step (21): calculating the number of the partitions on each dimension of the input set according to the set width limit;
step (22): calculating the total number of the blocks obtained by segmentation according to the number of the segments in each dimension;
step (23): calculating the relative position of each block to the input set;
step (24): selecting the sequence numbers of the blocks positioned on the boundary of the input set according to the relative positions;
step (25): obtaining the blocks, namely the hyper-rectangles, positioned on the input boundary set by using the selected block serial numbers; all hyper-rectangles together constitute the shell of the input set.
Exemplarily, the step (21): calculating the number of the partitions in each dimension according to the set width limit; the method specifically comprises the following steps:
Figure BDA0002894318520000061
wherein n is j Indicating the number of segments in each dimension.
Exemplary, step (22): calculating the total number of the blocks obtained by segmentation according to the number of the segments in each dimension; the method specifically comprises the following steps:
Figure BDA0002894318520000071
wherein n is all Representing the total number of blocks resulting from the segmentation.
Exemplary, step (23): calculating the relative position of each block to the input set; the method specifically comprises the following steps:
the relative position of each partition to the input set is represented by a cartesian index.
The Cartesian index CI is a vector of non-zero integers, each component CI being j (j＝1,2,…,k 0 ) Representing the forward order of this partition in the corresponding dimension.
With the k (k =1,2, \8230;, n) all ) Taking each block as an example, calculating corresponding Cartesian index CI k 。
Sequentially calculating CI k Component (b) of
Figure BDA0002894318520000072
Let k 0 = k, calculation
Figure BDA0002894318520000073
Make->
Figure BDA0002894318520000074
And &>
Figure BDA0002894318520000075
Is established, and
Figure BDA0002894318520000076
thereby calculating the corresponding cartesian index of each block.
Illustratively, the step (24): selecting the sequence numbers of the blocks positioned on the boundary of the input set according to the relative positions; the method specifically comprises the following steps:
the patches that lie on the input set boundary, i.e. those that intersect the input set boundary.
Due to the fluteThe Cartesian index represents the forward order of the partitions in each dimension, and the Cartesian index in any dimension is 1 or n j Must be located on the boundary of the input set.
Initialization set S h Is an empty set. For each partition, according to j =1,2, \ 8230;, k 0 In the order of (1), checking
Figure BDA0002894318520000077
Or
Figure BDA0002894318520000078
Whether or not this is true.
If there is a certain j that makes the above equation hold, then k at this time is put into the set S h If k is already in the set S h If no, no operation is done); otherwise, no operation is performed.
Thus, the sequence numbers of all the partitions at the boundary of the input set, i.e., the set S, are obtained h Of (2).
Illustratively, the step (25): obtaining the blocks, namely the hyper-rectangles, positioned on the input boundary set by using the selected block serial numbers; all hyper-rectangles together form a shell of the input set; the method comprises the following specific steps:
the block itself is a hyper-rectangle; here, it is necessary to find the kth (k ∈ S) h ) A block of interval form, i.e. [ l ] k ,u k ]；
Solving for l k J (j =1,2, \ 8230; k) 0 ) The formula for each component:
Figure BDA0002894318520000081
solving for u k J (j =1,2, \ 8230; k) 0 ) The formula for each component:
Figure BDA0002894318520000082
obtaining all blocks at the boundary of the input set, i.e. super-momentsShape H k ＝[l k ,u k ],(k∈S h )。
All hyper-rectangles together constitute a shell of the input set.
As one or more embodiments, the step (3): solving output reachable set estimation corresponding to the shell of the input set; the method comprises the following specific steps:
for all H obtained k (k∈S h ) All calculate their corresponding output reachable set estimates R k (k∈S h )；
Outputting a reachable set estimate R k The union of (1) is the output reachable set estimate corresponding to the shell of the input set. The method used is interval operation.
To solve for an H k Corresponding to R k The solving process is illustrated as an example:
because the value of the node on any hidden layer of the neural network only depends on the value of the node on the previous layer, the node is calculated in a layer-by-layer propagation mode.
From H k ＝[l k ,u k ]To obtain the value range of the variable at any node of the kth block input layer, i.e.
Figure BDA0002894318520000083
Therefore, the value ranges of all nodes of the first hidden layer are obtained, and then the value ranges of all nodes of the second hidden layer, the third hidden layer and the output layer can be obtained. The hyper-rectangle formed by the value ranges of all nodes of the output layer is R k 。
Suppose that the value ranges [ l ] of all nodes on the i-1 th layer have been obtained i-1 ,u i-1 ]Then, the value range [ l ] is taken from any node j on the ith layer i,j ,u i,j ]The following equation is used:
l i,j ＝σ i,j ([w i,j ] + ·l i-1 +[w i,j ] - ·u i-1 +b i,j )
u i,j ＝σ i,j ([w i,j ] + ·u i-1 +[w i,j ] - ·l i-1 +b i,j )
wherein [ w i,j ] + Is shown to w i,j Taking a non-negative value, namely, the non-negative element is unchanged, and the negative element is zero. Similarly, [ w ] i,j ] - Represents a pair of w i,j Take a non-positive value. In each dimension, l represents the lower bound and u represents the upper bound.
Thereby obtaining an output reachable set estimate corresponding to the shell of the input set.
As one or more embodiments, the step (4): judging whether the output reachable set estimation meets the output limit; if the output limit is met, a conclusion that the neural network is safe is obtained; otherwise, a conclusion that the neural network is unsafe is obtained; the method specifically comprises the following steps:
step (3) has solved the output reachable set estimate for the input set's shell, which is all R' s k (k∈S h ) A union of (1);
therefore, each R is examined in turn k Whether all are a subset of a given output limit Y;
where Y is a given set of output layers (hyper-rectangles) in which all element classifications are the same;
if there is
Figure BDA0002894318520000091
Then, it is concluded that the neural network is safe; otherwise, it is concluded that the neural network is not secure.
A robustness verification method of a medical purpose neural network based on shell protection belongs to the field of neural network safety and credible artificial intelligence. The method comprises the following steps: step one, selecting normal input and allowable error to generate an input set; step two, dividing an input set; solving output reachable set estimation corresponding to the shell of the input set; and step four, checking whether the output set estimation meets the output limit. The method provided by the invention requires that the neural network meets the requirement that the node number of each layer is not more than that of the previous layer. Compared with the existing verification method, the method can effectively reduce the time complexity and improve the verification speed of the neural network.
The core idea of the invention is as follows: when the neural network meeting the above conditions is verified, only the points on the given neighborhood boundary of the normal input are verified, and the points inside the neighborhood are ignored, so that the time complexity can be effectively reduced.
The correctness of the above idea will be demonstrated below:
first, the theoretical basis of the present invention is described: keeping the shell property. The crust protection is a mathematical property of the partial extended set function discovered by the research of the application. The following are some definitions and theorems related to crust-preserving.
Definition (crust protection): for a set function F (X), if F (X) satisfies: if Y = F (X) and G = F (H) are given for any shell H of any simple region X and X, then G must be a shell of Y, and F (X) is said to have a shell-retaining property.
The following two definitions are used to explain the terms used in the definitions of crust protection:
definition (simple area): a simple region A in n-dimensional Euclidean space is a single connected closed set with continuous boundaries. Obviously, convex polyhedrons in n-dimensional euclidean space are all simple regions.
Definition (shell): the shell of a simple region a in n-dimensional euclidean space is any subset B of a that contains all the boundary points of a. Obviously, A itself, the boundary of A is the shell of A.
The following theorem is used to later prove that some aggregate functions are crust-preserving:
theorem 1: the shell-preserving property is transitive to function compounding, that is, for any two set functions F (X) and G (X) with shell-preserving property (at least, the definition domain of F is required to contain the value domain of G, and it is generally considered that both F and G define n-dimensional euclidean space), their compound function H (X) = F [ G (X) ] still has shell-preserving property.
And (3) deducing: if the aggregation function F (X) is compounded by a finite number of aggregation functions with the shell-preserving property, the F (X) has the shell-preserving property.
Define 4 (extended set function): for a function defined in n-dimensional Euclidean spaceThe extended set function F (X) of the numbers F (X), F (X) is a function that maps sets to sets, and the independent variable of F (X) can be a subset of any F (X) domain whose dependent variable is dependent
Figure BDA0002894318520000111
Theorem 2: for functions F (X) and G (X) defined on an n-dimensional Euclidean space, the definition domain of F is required to contain the value domain of G, the extended set functions of the extended set functions are F (X) and G (X), respectively, and H (X) = F [ G (X) ], and the extended set function of H (X) is H (X), then H (X) = F [ G (X) ].
Theorem 3: for a function F (X) defined on an n-dimensional Euclidean space, the extended set functions are recorded as F (X), and two input sets U and V meet the requirement
Figure BDA0002894318520000112
Then there is->
Figure BDA0002894318520000113
The following demonstrates a reachable set estimate for computing a given set of inputs to a neural network
Figure BDA0002894318520000114
The set function of (2) has a shell-preserving property.
The neural network contains affine transformation and activation functions, and the extended set functions of the two functions are proved to have a shell-preserving property as follows:
first, the extended set function of a particular affine transformation is demonstrated to be crust-preserving:
introduction 1: given a linear transformation
Figure BDA0002894318520000115
Marks the corresponding matrix as->
Figure BDA0002894318520000116
L (X) is an extended gather function L (X), with input gathers for any simple region +>
Figure BDA0002894318520000117
M is the boundary of I, J = L (I), N = L (M), if k i ≤k i-1 Then N is the shell of J.
And (3) deducing: given a linear transformation
Figure BDA0002894318520000118
Marks the corresponding matrix as->
Figure BDA0002894318520000119
L (X) is an extended gather function L (X), and the input gather for any simple region->
Figure BDA00028943185200001110
P is the shell of I, J = L (I), Q = L (P), if k i ≤k i-1 Then Q is the envelope of J. That is, L (X) has a shell-retaining property at this time.
Theorem 4: given an affine transformation a (x) = Wx + b, in which
Figure BDA0002894318520000121
If k is i ≤k i-1 Then the extended set function a (X) of the affine transformation a (X) has a crust-preserving property.
Secondly, the extended set function of the activation function is proved to be crust-preserving:
let us note the activation function σ of the i-th layer i Has an extended set function of D i (X). Assuming an activation function sigma on each node i,j Satisfy the requirement of monotonous non-decreasing and continuous. Definition k i A vector function c i, To
Figure BDA0002894318520000122
c i,j (x)＝y,
Wherein the vector
Figure BDA0002894318520000123
Vector->
Figure BDA0002894318520000124
Note c i,j (x) Has an extended set function of C i,j Then, there are:
Figure BDA0002894318520000125
wherein
Figure BDA0002894318520000126
Representing the composition of the function.
The following two definitions define the extended set function used to simplify the activation function as having a crust-preserving property:
define (j-dimensional straight line): to pair
Figure BDA0002894318520000127
The j-dimensional straight line where x is located is:
Figure BDA0002894318520000128
obviously, any two different /) x,j There is no intersection between them, all l x,j The union set of (A) and (B) is
Figure BDA0002894318520000129
Definition (strongly simple region): a strong simple region I is a simple region that satisfies the following condition: any of x,j Intersection with I is only possible to be one of the following 3: an empty set, a point, or a line segment.
2, leading: given a strong simple region I, J = C i,j (I) Is a strong simple region, and C i,j Has shell protecting effect.
Theorem 5: activation function sigma i Is D i (X) has a shell-retaining property.
Then, the overestimation function is proved to be crust-preserving. The overestimation function is to calculate
Figure BDA00028943185200001210
An aggregation function is introduced.
Definition (overestimation function): as used herein, an over-estimation function is a set function, given a set of inputs I, the output of the over-estimation function G (X) is the smallest hyper-rectangle containing I.
A super-rectangular shape: { x: | x-c | < r }, where n-dimensional vector c is the center of the hyper-rectangle, and r is an n-dimensional vector. A hyper-rectangle is a convex polyhedron.
Theorem 6: the overestimation function G (X) has a crust-preserving property.
Layer i f of neural network i (x) Extended set function of
Figure BDA0002894318520000131
Finally, the proofs are used for calculations
Figure BDA0002894318520000132
The aggregation function F (X) of (2) has a crust-preserving property. />
Theorem 7: the extended set function of the neural network f (x) is noted as
Figure BDA0002894318520000133
If F e With crust-retaining property, then->
Figure BDA0002894318520000134
It also has shell protecting effect.
As deduced from theorem 1, if each layer has k i ≤k i-1 Then F e (X) has a shell-retaining property.
Theorem 8: for F (X) above, the input set X and any of its shells H,
Figure BDA0002894318520000135
E＝F(H)，/>
Figure BDA0002894318520000136
whether inclusion of Y is equivalent to whether E is included in Y.
Taking a neural network which is classified into two categories (healthy/unhealthy) according to blood routine indexes as an example, the application selects a detection result of a healthy male as a normal input, and relevant parameters are shown in the following table:
TABLE 1 relevant parameters
Figure BDA0002894318520000137
Figure BDA0002894318520000141
The data in the above table are labeled "health"
The present application verifies that the prediction (classification) of the neural network does not change, i.e. remains "healthy", when a slight perturbation is added to this sample.
The method comprises the following steps: according to the data in the table, the neighborhood center is taken
c＝[0.8,0.5,0.4,0.65,0.3,0.2,0.45,0.6,0.7]，
For ease of calculation, the input set X may be determined by taking the error magnitude e = 0.1.
Step two: for convenience of illustration, without loss of generality, the present application illustrates the segmentation process by taking two-dimensional data as an example, and the segmentation method of high-dimensional data is similar.
In the following figure, a 1x1 square input set is to be split, as shown in fig. 2.
The present application selects δ =0.2, and divides the input set into 25 blocks, wherein 16 blocks are located on the boundary, as shown in fig. 3.
The present application only holds the 16 partitions that lie on the boundary (outer circle) and computes their output reachable set estimates.
Step three: without loss of generality, the present application also illustrates the process of solving the output reachable set estimate for a given input set, taking as an example a neural network with two nodes per layer, one for each input layer and one for each output layer:
fig. 4 is the structure of the neural network, with the following parameters:
Figure BDA0002894318520000142
the activation function is set to ReLU.
Suppose that the value range of x1 is [0,1], and the value range of x2 is [ -2, -1], that is
Figure BDA0002894318520000151
Then the corresponding output reachable set estimate computation procedure is as follows:
[w 1,1 ] + ＝[1,2],[w 1,1 ] - ＝[0,0],[w 1,2 ] + ＝[0,0],[w 1,2 ] - ＝[-1,0]
l 1,1 ＝σ 1,1 ([w 1,1 ] + ·l 0 +[w 1,1 ] - ·u 0 +b 1,1 )＝-3
u 1,1 ＝σ 1,1 ([w 1,1 ] + · 0 +[w 1,1 ] - ·l 0 +b 1,1 )＝0
l 1,2 ＝σ 1,2 ([w 1,2 ] + ·l 0 +[w 1,2 ] - ·u 0 +b 1,2 )＝-2
u 1,2 ＝σ 1,2 ([w 1,2 ] + ·u 0 +[w 1,2 ] - ·l 0 +b 1,2 )＝-1
i.e., the output reachable set estimate is the rectangle R { (y) 1 ,y 2 )|-3≤y 1 ≤0,-2≤y 2 ≤-1}。
The network computation process for the greater number of layers and nodes is similar.
Step four:
continuing with the example of step three, assume here that the output limit Y is rectangular { (Y) 1 ,y 2 )|-5≤y 1 5. Ltoreq. Y2. Ltoreq.4, -4. Ltoreq. Y2. Ltoreq.4, then, obviously, there are
Figure BDA0002894318520000152
Therefore this neural network is safe in contrast to other inventions
The invention can effectively reduce the time complexity and improve the running speed.
Example two
The embodiment provides a neural network robustness rapid verification system;
a neural network robustness rapid verification system comprises:
a selection module configured to: selecting physical examination data of healthy users, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing;
a segmentation module configured to: carrying out segmentation processing on the input set to obtain a shell of the input set;
a solving module configured to: taking the shell of the input set as an input value of the neural network to be verified, and solving output reachable set estimation corresponding to the shell of the input set;
a determination module configured to: judging whether the output reachable set estimation meets the output limit; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained;
a training module configured to: training the neural network with a safety verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels;
an output module configured to: inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.
It should be noted here that the selecting module, the dividing module, the solving module, the judging module, the training module and the outputting module correspond to steps S101 to S106 in the first embodiment, and the modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In the foregoing embodiments, the descriptions of the embodiments have different emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The proposed system can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules may be combined or integrated into another system, or some features may be omitted, or not executed.
EXAMPLE III
The present embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, a processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device runs, the processor executes the one or more computer programs stored in the memory, so as to make the electronic device execute the method according to the first embodiment.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory, and may provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software.
The method in the first embodiment may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
Example four
The present embodiments also provide a computer-readable storage medium for storing computer instructions, which when executed by a processor, perform the method of the first embodiment.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (4)

1. A user physical examination data classification system based on neural network robustness rapid verification is characterized by comprising the following components:
a selection module configured to: selecting physical examination data of healthy users, carrying out normalization processing and error setting operation on the physical examination data, and generating an input set according to the user physical examination data and errors after the normalization processing; physical examination data of the healthy user comprises detection data of white blood cells, neutrophils, lymphocytes, hemoglobin, red blood cells, average red blood cell volume, platelet count, average platelet volume and average platelet width;
a segmentation module configured to: segmenting the input set to obtain a shell of the input set; the input set is subjected to segmentation processing; the method comprises the following specific steps:
calculating the number of the partitions on each dimension of the input set according to the set width limit;
calculating the total number of the blocks obtained by segmentation according to the number of the segments in each dimension;
calculating the relative position of each block to the input set;
selecting the sequence numbers of the blocks positioned on the boundary of the input set according to the relative positions;
obtaining a super-rectangle by using the sorted block serial numbers; forming all the hyper-rectangles into a shell of an input set;
calculating the number of the partitions in each dimension according to the set width limit; the method specifically comprises the following steps:
Figure 917956DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 277393DEST_PATH_IMAGE002
represents the number segmented in each dimension, and>
Figure 667923DEST_PATH_IMAGE003
，/>
Figure 11180DEST_PATH_IMAGE004
the number of the data;
calculating the total number of the blocks obtained by segmentation according to the number of the segments in each dimension; the method specifically comprises the following steps:
Figure 653514DEST_PATH_IMAGE005
wherein, the first and the second end of the pipe are connected with each other,
Figure 257671DEST_PATH_IMAGE006
representing the total number of the partitioned blocks;
a solving module configured to: taking the shell of the input set as an input value of the neural network to be verified, and solving output reachable set estimation corresponding to the shell of the input set; the neural network to be verified is a neural network which is subjected to secondary classification according to the blood conventional index;
solving output reachable set estimation corresponding to the shell of the input set; the method comprises the following specific steps:
for the obtained super-rectangle
Figure 960047DEST_PATH_IMAGE007
Calculating a corresponding output reachable set estimate >>
Figure 790600DEST_PATH_IMAGE008
；
Outputting reachable set estimates
Figure 95680DEST_PATH_IMAGE008
The union of (a) is an output reachable set estimate corresponding to the shell of the input set;
a determination module configured to: judging whether the output reachable set estimation meets the output limit; if the output limit is met, the conclusion that the neural network to be verified is safe is obtained; otherwise, the conclusion that the neural network to be verified is unsafe is obtained;
a training module configured to: training the neural network with a safe verification result, wherein a training set is user physical examination data of known healthy or unhealthy labels;
an output module configured to: inputting the physical examination data of the new user into the trained neural network, and outputting the classification label of the new user.
2. The system of claim 1, wherein the neural network robustness rapid verification-based user physical examination data classification system,
calculating the relative position of each block to the input set; the method specifically comprises the following steps:
representing, by a cartesian index, a relative position of each partition to the input set;
cartesian index
Figure 695288DEST_PATH_IMAGE009
Is a vector consisting of non-zero integers, each component->
Figure 427621DEST_PATH_IMAGE010
Representing the forward order of the partitions in the respective dimensions; />
Figure 745470DEST_PATH_IMAGE003
；
By the first
Figure 464027DEST_PATH_IMAGE011
Taking several blocks as an example, the corresponding Cartesian index is calculated>
Figure 308355DEST_PATH_IMAGE012
；/>
Figure 86955DEST_PATH_IMAGE013
；
Calculate in turn
Figure 751155DEST_PATH_IMAGE012
Component of->
Figure 538982DEST_PATH_IMAGE014
，/>
Figure 847604DEST_PATH_IMAGE003
；
Order to
Figure 187318DEST_PATH_IMAGE015
Calculate->
Figure 948601DEST_PATH_IMAGE014
Make->
Figure 399174DEST_PATH_IMAGE016
And &>
Figure 296723DEST_PATH_IMAGE017
Is established, and
Figure 541759DEST_PATH_IMAGE018
(ii) a And calculating a Cartesian index corresponding to each block.
3. The system of claim 2, wherein the neural network robustness-based rapid verification-based physical examination data classification system for users,
selecting the sequence numbers of the blocks positioned on the boundary of the input set according to the relative positions; the method specifically comprises the following steps:
the blocks positioned on the boundary of the input set are blocks which have intersection with the boundary of the input set;
the Cartesian index represents the forward order of the blocks in each dimension, and the Cartesian index in any dimension is 1 or
Figure 321496DEST_PATH_IMAGE002
Is located on the boundary of the input set;
initializing a set
Figure 304322DEST_PATH_IMAGE019
Is an empty set; for each block, according to>
Figure 587535DEST_PATH_IMAGE003
Checking the order of->
Figure 144419DEST_PATH_IMAGE020
Or
Figure 4927DEST_PATH_IMAGE021
Whether the result is true or not;
if some j makes the above formula hold, put k at this time into the set
Figure 672669DEST_PATH_IMAGE019
Performing the following steps; otherwise, no operation is carried out; obtaining the serial numbers of all the blocks positioned on the boundary of the input set;
obtaining a super-rectangle by using the sorted serial numbers of the blocks; all hyper-rectangles together form a shell of the input set; the method comprises the following specific steps:
find out the first
Figure 935023DEST_PATH_IMAGE011
Block-by-block interval form>
Figure 928387DEST_PATH_IMAGE022
；
Solving for
Figure 151558DEST_PATH_IMAGE023
In a first or second section>
Figure 482045DEST_PATH_IMAGE024
The formula for each component:
Figure 739851DEST_PATH_IMAGE025
solving for
Figure 763171DEST_PATH_IMAGE026
To (1)/>
Figure 473638DEST_PATH_IMAGE024
The formula for each component:
Figure 483182DEST_PATH_IMAGE027
/>
obtain a super rectangle
Figure 985707DEST_PATH_IMAGE028
；
All hyper-rectangles together constitute a shell of the input set.
4. The system of claim 1, wherein the neural network robustness rapid verification-based user physical examination data classification system,
judging whether the output reachable set estimation meets the output limit or not; if the output limit is met, a conclusion that the neural network is safe is obtained; otherwise, a conclusion that the neural network is unsafe is obtained; the method specifically comprises the following steps:
the output reachable set corresponding to the shell of the input set is estimated as all
Figure 789715DEST_PATH_IMAGE008
The union of (1);
examine each in turn
Figure 112112DEST_PATH_IMAGE008
Whether or not both are given output limits->
Figure 925348DEST_PATH_IMAGE029
A subset of (a);
Figure 892167DEST_PATH_IMAGE029
is a given set of output layers in which all element classifications are the same;
if there is
Figure 991710DEST_PATH_IMAGE030
Obtaining the conclusion that the neural network is safe; otherwise, it is concluded that the neural network is unsafe. />
CN202110038415.8A 2021-01-12 2021-01-12 Neural network robustness rapid verification method, system, device and storage medium Active CN112735550B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110038415.8A CN112735550B (en) 2021-01-12 2021-01-12 Neural network robustness rapid verification method, system, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110038415.8A CN112735550B (en) 2021-01-12 2021-01-12 Neural network robustness rapid verification method, system, device and storage medium

Publications (2)

Publication Number Publication Date
CN112735550A CN112735550A (en) 2021-04-30
CN112735550B true CN112735550B (en) 2023-04-07

Family

ID=75590609

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110038415.8A Active CN112735550B (en) 2021-01-12 2021-01-12 Neural network robustness rapid verification method, system, device and storage medium

Country Status (1)

Country Link
CN (1) CN112735550B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111145042A (en) * 2019-12-31 2020-05-12 国网北京市电力公司 Power distribution network voltage abnormity diagnosis method adopting full-connection neural network

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111145042A (en) * 2019-12-31 2020-05-12 国网北京市电力公司 Power distribution network voltage abnormity diagnosis method adopting full-connection neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
EVALUATING ROBUSTNESS OF NEURAL NETWORKS WITH MIXED INTEGER PROGRAMMING;Vincent Tjeng et al.;《 ICLR 2019》;20191231;第1-21页 *
EVALUATING THE ROBUSTNESS OF NEURAL NETWORKS: AN EXTREME VALUE THEORY APPROACH;Tsui-Wei Weng et al.;《ICLR 2018》;20181231;第1-18页 *

Also Published As

Publication number Publication date
CN112735550A (en) 2021-04-30

Similar Documents

Publication Publication Date Title
Schulam et al. Can you trust this prediction? Auditing pointwise reliability after learning
Bartlett For valid generalization the size of the weights is more important than the size of the network
Lung-Yut-Fong et al. Homogeneity and change-point detection tests for multivariate data using rank statistics
Ma et al. Classification of automatic radar plotting aid targets based on improved fuzzy C-means
CN108596053A (en) A kind of vehicle checking method and system based on SSD and vehicle attitude classification
Zhao et al. Lung segmentation in CT images using a fully convolutional neural network with multi-instance and conditional adversary loss
Azar et al. Superior neuro-fuzzy classification systems
Ung Development of a weighted probabilistic risk assessment method for offshore engineering systems using fuzzy rule-based Bayesian reasoning approach
US11475302B2 (en) Multilayer perceptron based network to identify baseline illness risk
CN112735550B (en) Neural network robustness rapid verification method, system, device and storage medium
Puthawala et al. Universal joint approximation of manifolds and densities by simple injective flows
Ventura et al. Pseudo-likelihoods for Bayesian inference
Xing et al. Calibrated approximate Bayesian inference
CN112733941A (en) Medical use neural network robustness verification method and system based on shell protection
Manimaran et al. A review of fuzzy environmental study in medical diagnosis system
Ibrahim et al. A new three-term conjugate gradient method for training neural networks with global convergence
CN115935203A (en) Distributed clustering method, device and medium on wireless sensor network
Vanea et al. A New Graph Node Classification Benchmark: Learning Structure from Histology Cell Graphs
Barros et al. p-Adaptive C k generalized finite element method for arbitrary polygonal clouds
CN110543417B (en) Method and device for evaluating effectiveness of software test
Engliš et al. The Dixmier trace of Hankel operators on the Bergman space
Hafshejani et al. Barzilai and borwein conjugate gradient method equipped with a non-monotone line search technique and its application on non-negative matrix factorization
Mohammed Grid Refinement and Verification Estimates for the RBF Construction Method of Lyapunov Functions
Edalat Smooth approximation of Lipschitz maps and their subgradients
Vorontsov Combinatorial substantiation of learning algorithms

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant