CN114549941B - Model testing method and device and electronic equipment - Google Patents
Model testing method and device and electronic equipment Download PDFInfo
- Publication number
- CN114549941B CN114549941B CN202210157077.4A CN202210157077A CN114549941B CN 114549941 B CN114549941 B CN 114549941B CN 202210157077 A CN202210157077 A CN 202210157077A CN 114549941 B CN114549941 B CN 114549941B
- Authority
- CN
- China
- Prior art keywords
- model
- noise
- disturbance
- image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 182
- 238000000034 method Methods 0.000 claims description 35
- 238000004590 computer program Methods 0.000 claims description 10
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 8
- 238000013473 artificial intelligence Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 8
- 238000011156 evaluation Methods 0.000 description 8
- 238000010998 test method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The disclosure provides a model testing method, a model testing device and electronic equipment, relates to the technical field of image processing, and particularly relates to the technical field of artificial intelligence. The specific implementation scheme is as follows: acquiring a noise map sequence, wherein the noise map sequence comprises a target noise map; acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image; and testing the first model by using the target disturbance image to obtain a test result of the first model, wherein the test result is used for evaluating the robustness of the model. The present disclosure may improve the efficiency of model testing.
Description
Technical Field
The disclosure relates to the technical field of image processing, in particular to the technical field of artificial intelligence, and specifically relates to a model testing method and device and electronic equipment.
Background
Deep learning technology has been widely used in artificial intelligence tasks, and in the field of computer vision, deep learning has become the dominant force in automatic driving, picture auditing, monitoring and security applications. In testing the robustness of an image model, the method is generally implemented by iterating and continuously enhancing noise, and each test is performed by generating a new disturbance based on the image of the last test and new noise.
Disclosure of Invention
The present disclosure provides a method, apparatus, device, and storage medium for model testing.
According to a first aspect of the present disclosure, there is provided a model test method including:
acquiring a noise map sequence, wherein the noise map sequence comprises a target noise map;
Acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image;
And testing the first model by using the target disturbance image to obtain a test result of the first model, wherein the test result is used for evaluating the robustness of the model.
According to a second aspect of the present disclosure, there is provided a model test apparatus including:
the first acquisition module is used for acquiring a noise map sequence, wherein the noise map sequence comprises a target noise map;
The second acquisition module is used for acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image;
and the first testing module is used for testing the first model by using the target disturbance image to obtain a testing result of the first model, and the testing result is used for evaluating the robustness of the model.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
At least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform any one of the methods of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any of the methods of the first aspect.
In the embodiment of the disclosure, the first model is tested by using the target disturbance image obtained by fusing the target noise image in the noise image sequence and the pre-acquired original image, that is, in the test process of evaluating the robustness of the model, the target disturbance image used can be directly obtained according to the target noise image in the noise image sequence, and the superposition of random noise generated in each test on the image tested last time can be avoided, thereby improving the test efficiency.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow diagram of a method of model testing according to a first embodiment of the present disclosure;
FIG. 2 is one of schematic structural diagrams of a model test apparatus according to a second embodiment of the present disclosure;
FIG. 3 is a second schematic diagram of a model test apparatus according to a second embodiment of the present disclosure;
FIG. 4 is a third schematic structural view of a model test device according to a second embodiment of the present disclosure;
FIG. 5 is a fourth schematic structural view of a model test device according to a second embodiment of the present disclosure;
FIG. 6 is a fifth schematic structural view of a model test device according to a second embodiment of the present disclosure;
FIG. 7 is a sixth schematic structural view of a model test apparatus according to a second embodiment of the present disclosure;
FIG. 8 is a seventh schematic structural view of a model test device according to a second embodiment of the present disclosure;
fig. 9 is a block diagram of an electronic device for implementing a model test method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As shown in fig. 1, the present disclosure provides a model test method, including the steps of:
step S101: a noise map sequence is acquired, wherein the noise map sequence comprises a target noise map.
The noise map sequence may include a plurality of noise maps, and the plurality of noise maps may be generated by overlapping the same noise a plurality of times, or may be generated by overlapping a plurality of different noise maps, and the noise of each noise map in the noise map sequence is sequentially increased, so that the plurality of noise that is continuously enhanced is represented by the noise map sequence.
It will be appreciated that in the case where a plurality of noise maps in the noise map sequence are generated by overlapping the same noise a plurality of times, the number of times the plurality of noise maps overlap the same noise is different, for example: the first noise diagram in the noise diagram sequence is obtained by superposing the same noise once, the second noise diagram in the noise diagram sequence is obtained by superposing the same noise twice, the third noise diagram in the noise diagram sequence is obtained by superposing the same noise three times, and the like; in the case where a plurality of noise maps in the above-described noise map sequence are generated by superposition of a plurality of different noises, a plurality of different noises may be acquired in advance: noise 1, noise 2, noise 3, and the like, a first noise pattern in the noise pattern sequence is obtained by overlapping noise 1, a second noise pattern in the noise pattern sequence is obtained by overlapping noise 1 and noise 2, and a third noise pattern in the noise pattern sequence is obtained by overlapping noise 1, noise 2, and noise 3, which is not limited in this disclosure.
It will be appreciated that the target noise figure may be one or more noise figures selected from the sequence of noise figures for model testing, and that in the case where the target noise figure is an image, the anti-interference capability of the model may be tested using the image; in the case where the target noise figure is a plurality of images, the anti-interference capability of the plurality of image test models may be used, respectively.
Step S102: and acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image.
It will be appreciated that the original image may be a pre-acquired image to be identified without superimposed noise, for example: including images of objects to be identified (e.g., cats, dogs, etc.). By integrating the target noise map and the pre-acquired original image, the target noise map may represent noise to be superimposed, that is, the target disturbance image after the noise is superimposed on the original image may be acquired.
Step S103: and testing the first model by using the target disturbance image to obtain a test result of the first model, wherein the test result is used for evaluating the robustness of the model.
In some embodiments, the first model may be a model for image recognition, and the image with superimposed noise may be used to test the robustness of the model when the first model is tested. The first model may identify a target object in the original image, and the test result may be obtained based on the identification result by identifying the target disturbance image using the first model, for example: if the target result cannot be identified or the difference between the identification result and the target result is large, the noise superimposed on the target disturbance image can be used as a robustness assessment result of the first model.
In the embodiment of the disclosure, the first model is tested by using the target disturbance image obtained by fusing the target noise image in the noise image sequence and the pre-acquired original image, that is, in the test process of evaluating the robustness of the model, the target disturbance image used can be directly obtained according to the target noise image in the noise image sequence, and the superposition of random noise generated in each test on the image tested last time can be avoided, thereby improving the test efficiency.
Optionally, the acquiring the noise map sequence in step S101 may specifically include:
Acquiring a transparent graph and random noise;
And performing M times of iterative operation on the transparent graph based on the random noise to obtain a noise graph sequence comprising M noise graphs, wherein the random noise is superimposed on the transparent graph in the first time of iterative operation, the random noise is superimposed on the noise graph obtained in the M-1 time of iterative operation in the mth iterative operation, the target noise graph is a noise graph in the M noise graphs, M is an integer greater than 1, and M is an integer greater than 1 and less than or equal to M.
The random noise may be simulated noise, which is pseudo-natural, such as gaussian noise, impulse noise, additive noise, or mixed noise, and the superimposed random noise may be represented in an image form (i.e., noise map) by superimposing the random noise on a transparent map.
The noise map is obtained by superposing random noise on a transparent map, the position of each noise point in the random noise corresponds to one pixel point in the noise map, and the pixel value of each pixel point in the obtained noise map corresponds to the noise point pixel value of the random noise one by taking the example that the random noise is superposed once on the transparent map.
Based on the random noise, performing iterative operation on the transparent graph, wherein each iterative operation can obtain a noise graph, and after performing M iterative operations, different noise graphs in the obtained M noise graphs respectively correspond to the iterative times of the random noise, and the iterative times of the different noise graphs are different.
The iterative operation on the transparent graph can be understood as that the random noise is respectively superimposed on the transparent graph based on different times, specifically, a noise graph corresponding to a first iteration can be obtained by superimposing the random noise on the transparent graph once, a noise graph corresponding to a second iteration can be obtained by superimposing the random noise on the transparent graph twice, and the like, so as to obtain M noise graphs corresponding to different iterations.
The target noise map may be any one of the M noise maps, and in the process of testing the first model using the target disturbance image, a target noise map corresponding to a certain number of iterations may be determined from the M noise maps included in the noise map sequence, and the target noise map and the original image may be fused to obtain the target disturbance image.
In this embodiment, by acquiring a transparent graph and random noise, and performing M iterative operations on the transparent graph based on the random noise, a noise graph sequence including M noise graphs is obtained, and the noise graph sequences composed of noise graphs with different iteration times can be obtained, so that in the testing process of the first model, a target noise graph in the noise graph sequence can be quickly acquired, and a robustness test of the model is realized.
And the iteration times of different noise graphs in the M noise graphs included in the noise graph sequence are different, namely pixel difference values between adjacent noise graphs in the noise graph sequence are consistent, and when the noise graphs in the noise graph sequence are used for carrying out model robustness evaluation test, a test result can be quantized by using the noise difference values with consistency, so that the difference of the test result caused by different random noise generated each time is avoided.
Optionally, the acquiring the target disturbance image in step S102 may specifically include:
Obtaining a disturbance map sequence, wherein the disturbance map sequence comprises a plurality of disturbance images, the disturbance images are respectively obtained by fusing the M noise maps and the original image, and the disturbance images comprise the target disturbance image;
In step S103, the testing the first model using the target disturbance image to obtain a test result of the first model may specifically include:
acquiring target disturbance images in the disturbance images based on a preset iteration frequency interval, wherein the target disturbance images comprise N disturbance images, and N is an integer smaller than or equal to M;
and based on the iteration times corresponding to the N disturbance images, testing the first model by sequentially using the N disturbance images to obtain a test result of the first model.
It can be understood that the disturbance images in the disturbance image sequence are obtained by superposing the noise images on the original image, but the noise corresponding to the superposed noise images is sequentially enhanced. In the process of testing the first model, N disturbance images in the disturbance map sequence obtained by continuously enhancing noise can be obtained for testing, so that the robustness of the first model is evaluated.
The preset iteration number interval may be predetermined according to a test requirement, for example: in the scene that only the model robustness evaluation result is needed to be roughly obtained in model test effect display and the like, a larger preset iteration frequency interval can be set, so that a smaller number of target disturbance images are obtained in the disturbance image sequence to carry out model test, and the model test result is displayed, meanwhile, the test speed is higher, and the occupied calculation resources are smaller; in the scene of accurately obtaining the model robustness evaluation result, a smaller preset iteration frequency interval can be set, so that a larger number of target disturbance images are obtained in the disturbance map sequence to perform model test, and the accuracy of the test result is improved.
In this embodiment, based on a preset iteration number interval, a target disturbance image in the multiple disturbance images is obtained, where the target disturbance image includes N disturbance images, N is an integer less than or equal to M, and based on the iteration number corresponding to the N disturbance images, the first model is tested by sequentially using the N disturbance images, so as to obtain a test result of the first model, and different test requirements can be satisfied. Under the condition that the preset iteration frequency interval is larger, the test efficiency can be improved; under the condition that the preset iteration frequency interval is smaller, the accuracy of the test result can be improved.
Optionally, before acquiring at least part of the disturbance images in the plurality of disturbance images based on the preset iteration number interval, the method may further include:
Acquiring granularity parameters of a model test;
And determining the preset iteration frequency interval based on the granularity parameter.
The granularity parameter is matched with the actual requirement of the model test, and a larger granularity parameter indicates a coarser result required by the model test, and a smaller granularity parameter indicates a more accurate result required by the model test, for example: in the test with large granularity, only a coarser result is needed to be obtained for the robustness evaluation of the model, and a larger preset iteration frequency interval can be determined; in the test with small granularity, a more accurate result is required to be obtained for the robustness evaluation of the model, and a smaller preset iteration number interval can be determined.
In this embodiment, the model test may determine the preset iteration frequency interval based on the granularity parameter, that is, the adjustment of the number of images used for the model test is implemented, so as to adapt to the granularity of different model tests.
Optionally, based on the iteration times corresponding to the N disturbance images, testing the first model by sequentially using the N disturbance images to obtain a test result of the first model may specifically include:
And based on the iteration times corresponding to the N disturbance images, testing the first model by sequentially using the N disturbance images until the test result of the first model comprises a result that the original image cannot be identified, wherein the test result of the first model further comprises: and the pixel difference value between the disturbance image corresponding to the result of the original image which cannot be identified and the original image.
It can be understood that the pixel difference between the disturbance image and the original image is the pixel value of the noise image fused with the original image when the disturbance image is generated. And obtaining pixel difference values between the disturbance image corresponding to the result of the original image which cannot be identified and the original image, and quantifying a test result of the robustness assessment of the first model based on the pixel difference values, so that the robustness performance of the model is accurately represented by the pixel difference values.
In this embodiment, the first model may be tested by sequentially using the N disturbance images based on the iteration numbers corresponding to the N disturbance images until the test result of the first model includes a result that the original image cannot be identified, the robustness of the first model to sequentially enhanced noise may be tested, and the test result of the first model further includes: and the pixel difference value between the disturbance image corresponding to the result of the original image and the original image can be used for representing the robustness assessment result of the first model, so that the analysis and comparison of the model robustness can be realized.
Optionally, after the testing the first model by sequentially using the N disturbance images based on the iteration times corresponding to the N disturbance images to obtain a test result of the first model, the method may further include:
based on the iteration times corresponding to the N disturbance images, testing a second model by sequentially using the N disturbance images to obtain a test result of the second model, wherein the second model is the model with the same identification task as the first model;
comparing the test results of the first model with the test results of the second model to determine a comparison of the robustness of the first model and the second model.
In the process of testing the model, the N disturbance images are sequentially used for testing, namely, the N disturbance images with gradually increased noise are respectively used for carrying out robustness evaluation testing on the model, and in the process of robustness evaluation of the model, noise which cannot identify an original image result can be used as a robustness evaluation parameter of the model. In addition, the N disturbance images are sequentially used by the first model and the second model in the test process, so that the test result of the first model and the test result of the second model can be accurately compared.
In this embodiment, when evaluating the robustness of the model, N disturbance images in the disturbance map sequence may be used, the first model may be tested by sequentially using the N disturbance images based on the iteration numbers corresponding to the N disturbance images, so as to obtain a test result of the first model, and then, after sequentially using the N disturbance images based on the iteration numbers corresponding to the N disturbance images, a test result of the second model may be obtained, and the test result of the first model and the test result of the second model may be obtained based on the same disturbance images, so that the test result of the first model and the test result of the second model may be compared to determine the comparison result of the robustness of the first model and the second model.
Optionally, before the testing the first model using the target disturbance image in step S102 to obtain the test result of the first model, the method may further include:
Acquiring a pixel value of each noise image in the noise image sequence;
After the first model is tested by using the target disturbance image to obtain a test result of the first model, the method further comprises:
And outputting pixel values of the target noise image in the case that the test result includes a result that the original image cannot be recognized.
The pixel value is the noise that each noise map can be superimposed on the original image, that is, the pixel difference between the original image and a plurality of disturbance maps obtained by fusing each noise map with the original image. It will be appreciated that the pixel value of each noise figure in the above described sequence of noise figures increases in turn. Specifically, the pixel values of the noise map may be represented using a mean square error (Mean Square Error, MSE).
In the testing process of the first model, the disturbance image obtained by continuously enhancing noise and merging the original image can be sequentially used for testing the first model, and when the testing result of the first model comprises a result that the original image cannot be identified, the pixel value of the target noise image is output, so that the pixel value of the target noise image is used as a quantized value for evaluating the robustness of the first model.
In this embodiment, before the first model is tested by using the target disturbance image to obtain the test result of the first model, the pixel value of each noise image in the noise image sequence is obtained, and after the first model is tested by using the target disturbance image to obtain the test result of the first model, the pixel value of the target noise image is output under the condition that the test result includes the result that the original image cannot be identified, so that the pixel value of the target noise image can be quickly output, and the efficiency of model robustness assessment is improved.
As shown in fig. 2, the present disclosure further provides a model testing apparatus, including:
A first obtaining module 201, configured to obtain a noise map sequence, where the noise map sequence includes a target noise map;
A second obtaining module 202, configured to obtain a target disturbance image, where the target disturbance image is obtained by fusing the target noise map with a pre-obtained original image;
A first test module 203, configured to test a first model using the target disturbance image to obtain a test result of the first model, where the test result is used to evaluate robustness of the model.
Optionally, as shown in fig. 3, the first obtaining module 201 may specifically include:
a first acquiring unit 2011, configured to acquire a transparent graph and random noise;
An iteration unit 2012 configured to perform, based on the random noise, M iteration operations on the transparent graph to obtain a noise graph sequence including M noise graphs, where the first iteration operation is to superimpose the random noise on the transparent graph, the M-th iteration operation is to superimpose the random noise on the noise graph obtained by the M-1-th iteration operation, the target noise graph is a noise graph in the M noise graphs, M is an integer greater than 1, and M is an integer greater than 1 and less than or equal to M.
Optionally, as shown in fig. 4, the second obtaining module 202 may specifically include:
a second acquisition unit 2021, configured to acquire a disturbance map sequence, where the disturbance map sequence includes a plurality of disturbance images, the plurality of disturbance images are respectively obtained by fusing the M noise maps and the original image, and the plurality of disturbance images include the target disturbance image;
the first test module 203 may specifically include:
A third obtaining unit 2031, configured to obtain, based on a preset iteration number interval, a target disturbance image from the plurality of disturbance images, where the target disturbance image includes N disturbance images, and N is an integer less than or equal to M;
a test unit 2032, configured to sequentially test the first model using the N disturbance images based on the iteration numbers corresponding to the N disturbance images, so as to obtain a test result of the first model.
Alternatively, as shown in fig. 5, the test unit 2032 may specifically include:
The test subunit 20321 is configured to sequentially test the first model using the N disturbance images based on the iteration numbers corresponding to the N disturbance images until a test result of the first model includes a result that the original image cannot be identified, where the test result of the first model further includes: and the pixel difference value between the disturbance image corresponding to the result of the original image which cannot be identified and the original image.
Optionally, as shown in fig. 6, the model test apparatus 200 may further include:
The second testing module 204 is configured to sequentially test a second model using the N disturbance images based on the iteration numbers corresponding to the N disturbance images, so as to obtain a test result of the second model, where the second model is the same model as the first model identification task;
A comparison module 205, configured to compare the test result of the first model with the test result of the second model, so as to determine a comparison result of the robustness of the first model and the second model.
Optionally, as shown in fig. 7, the model test apparatus 200 may further include:
A third obtaining module 206, configured to obtain a granularity parameter of the model test;
a determining module 207, configured to determine the preset iteration number interval based on the granularity parameter.
Optionally, as shown in fig. 8, the model test apparatus 200 may further include:
A fourth obtaining module 208, configured to obtain a pixel value of each noise map in the noise map sequence;
An output module 209, configured to output a pixel value of the target noise image in a case where the test result includes a result that cannot identify the original image.
The model test device 200 provided in the present disclosure can implement each process of the embodiment of the model test method, and can achieve the same technical effects, so that repetition is avoided, and detailed description is omitted herein.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the apparatus 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 903 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.
Various components in device 900 are connected to I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, or the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 903 such as a magnetic disk, an optical disk, or the like; and a communication unit 909 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above, such as a model test method. For example, in some embodiments, the model test method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 903. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 900 via the ROM902 and/or the communication unit 909. When the computer program is loaded into the RAM903 and executed by the computing unit 901, one or more steps of the above-described model test method may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the model test method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (15)
1. A model testing method, comprising:
Acquiring a noise map sequence, wherein the noise map sequence comprises target noise maps, and the noise of each noise map in the noise map sequence is sequentially increased;
Acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image;
Testing a first model by using the target disturbance image to obtain a test result of the first model, wherein the test result is used for evaluating the robustness of the model;
Before the first model is tested by using the target disturbance image to obtain the test result of the first model, the method further comprises:
acquiring a pixel value of each noise image in the noise image sequence, wherein the pixel value of each noise image in the noise image sequence is sequentially increased;
the step of testing the first model by using the target disturbance image to obtain a test result of the first model specifically includes:
And sequentially testing the first model by using the interference image obtained by fusing the continuously enhanced noise with the original image, and outputting the pixel value of the target noise image under the condition that the test result comprises a result that the original image cannot be identified.
2. The method of claim 1, wherein the acquiring a noise figure sequence comprises:
Acquiring a transparent graph and random noise;
And performing M times of iterative operation on the transparent graph based on the random noise to obtain a noise graph sequence comprising M noise graphs, wherein the random noise is superimposed on the transparent graph in the first time of iterative operation, the random noise is superimposed on the noise graph obtained in the M-1 time of iterative operation in the mth iterative operation, the target noise graph is a noise graph in the M noise graphs, M is an integer greater than 1, and M is an integer greater than 1 and less than or equal to M.
3. The method of claim 2, wherein the acquiring the target disturbance image comprises:
Obtaining a disturbance map sequence, wherein the disturbance map sequence comprises a plurality of disturbance images, the disturbance images are respectively obtained by fusing the M noise maps and the original image, and the disturbance images comprise the target disturbance image;
the step of testing the first model by using the target disturbance image to obtain a test result of the first model comprises the following steps:
acquiring target disturbance images in the disturbance images based on a preset iteration frequency interval, wherein the target disturbance images comprise N disturbance images, and N is an integer smaller than or equal to M;
and based on the iteration times corresponding to the N disturbance images, testing the first model by sequentially using the N disturbance images to obtain a test result of the first model.
4. The method of claim 3, wherein the sequentially testing the first model using the N disturbance images based on the number of iterations corresponding to the N disturbance images to obtain a test result of the first model includes:
And based on the iteration times corresponding to the N disturbance images, testing the first model by sequentially using the N disturbance images until the test result of the first model comprises a result that the original image cannot be identified, wherein the test result of the first model further comprises: and the pixel difference value between the disturbance image corresponding to the result of the original image which cannot be identified and the original image.
5. The method of claim 3, wherein after sequentially testing the first model using the N disturbance images based on the iteration numbers corresponding to the N disturbance images to obtain the test result of the first model, further comprising:
based on the iteration times corresponding to the N disturbance images, testing a second model by sequentially using the N disturbance images to obtain a test result of the second model, wherein the second model is the model with the same identification task as the first model;
comparing the test results of the first model with the test results of the second model to determine a comparison of the robustness of the first model and the second model.
6. The method of claim 3, further comprising, prior to acquiring at least a portion of the plurality of perturbation images based on a preset iteration number interval:
Acquiring granularity parameters of a model test;
And determining the preset iteration frequency interval based on the granularity parameter.
7. A model test apparatus comprising:
the first acquisition module is used for acquiring a noise map sequence, wherein the noise map sequence comprises target noise maps, and the noise of each noise map in the noise map sequence is sequentially increased;
The second acquisition module is used for acquiring a target disturbance image, wherein the target disturbance image is obtained by fusing the target noise image and a pre-acquired original image;
The first testing module is used for testing a first model by using the target disturbance image to obtain a testing result of the first model, and the testing result is used for evaluating the robustness of the model;
The apparatus further comprises:
A fourth obtaining module, configured to obtain a pixel value of each noise map in the noise map sequence, where the pixel value of each noise map in the noise map sequence increases sequentially;
And the output module is used for testing the first model by sequentially using the interference images obtained by continuously enhancing noise fusion of the original images, and outputting the pixel value of the target noise image under the condition that the test result comprises a result that the original images cannot be identified.
8. The apparatus of claim 7, wherein the first acquisition module comprises:
A first acquisition unit configured to acquire a transparent map, and random noise;
And the iteration unit is used for executing M iteration operations on the transparent graph based on the random noise to obtain a noise graph sequence comprising M noise graphs, wherein the random noise is superposed on the transparent graph in the first iteration operation, the random noise is superposed on the noise graph obtained in the (M-1) th iteration operation in the mth iteration operation, the target noise graph is the noise graph in the M noise graphs, M is an integer greater than 1, and M is an integer greater than 1 and less than or equal to M.
9. The apparatus of claim 8, wherein the second acquisition module comprises:
The second acquisition unit is used for acquiring a disturbance image sequence, wherein the disturbance image sequence comprises a plurality of disturbance images, the disturbance images are respectively obtained by fusing the M noise images and the original image, and the disturbance images comprise the target disturbance image;
The first test module comprises:
a third obtaining unit, configured to obtain a target disturbance image from the multiple disturbance images based on a preset iteration number interval, where the target disturbance image includes N disturbance images, and N is an integer less than or equal to M;
And the testing unit is used for testing the first model by sequentially using the N disturbance images based on the iteration times corresponding to the N disturbance images so as to obtain a testing result of the first model.
10. The apparatus of claim 9, wherein the test unit comprises:
The testing subunit is configured to sequentially test the first model using the N disturbance images based on the iteration times corresponding to the N disturbance images until a test result of the first model includes a result that the original image cannot be identified, where the test result of the first model further includes: and the pixel difference value between the disturbance image corresponding to the result of the original image which cannot be identified and the original image.
11. The apparatus of claim 9, further comprising:
The second test module is used for sequentially using the N disturbance images to test a second model based on the iteration times corresponding to the N disturbance images so as to obtain a test result of the second model, wherein the second model is the model with the same identification task as the first model;
And the comparison module is used for comparing the test result of the first model with the test result of the second model to determine a comparison result of the robustness of the first model and the second model.
12. The apparatus of claim 9, further comprising:
The third acquisition module is used for acquiring granularity parameters of the model test;
And the determining module is used for determining the preset iteration frequency interval based on the granularity parameter.
13. An electronic device, comprising:
At least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210157077.4A CN114549941B (en) | 2022-02-21 | 2022-02-21 | Model testing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210157077.4A CN114549941B (en) | 2022-02-21 | 2022-02-21 | Model testing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114549941A CN114549941A (en) | 2022-05-27 |
| CN114549941B true CN114549941B (en) | 2024-06-28 |
Family
ID=81676414
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210157077.4A Active CN114549941B (en) | 2022-02-21 | 2022-02-21 | Model testing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114549941B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263872A (en) * | 2019-06-26 | 2019-09-20 | 上海鹰瞳医疗科技有限公司 | Training data processing method and processing device |
| CN113792791A (en) * | 2021-09-14 | 2021-12-14 | 百度在线网络技术（北京）有限公司 | Processing method and device for visual model |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845520B (en) * | 2016-12-23 | 2018-05-18 | 深圳云天励飞技术有限公司 | A kind of image processing method and terminal |
| CN112907552B (en) * | 2021-03-09 | 2024-03-01 | 百度在线网络技术（北京）有限公司 | Robustness detection method, device and program product for image processing model |
| CN113449783B (en) * | 2021-06-17 | 2022-11-08 | 广州大学 | Countermeasure sample generation method, system, computer device and storage medium |
-
2022
- 2022-02-21 CN CN202210157077.4A patent/CN114549941B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263872A (en) * | 2019-06-26 | 2019-09-20 | 上海鹰瞳医疗科技有限公司 | Training data processing method and processing device |
| CN113792791A (en) * | 2021-09-14 | 2021-12-14 | 百度在线网络技术（北京）有限公司 | Processing method and device for visual model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114549941A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112966742A (en) | Model training method, target detection method and device and electronic equipment | |
| CN112819169A (en) | Quantum control pulse generation method, device, equipment and storage medium | |
| CN113537374B (en) | Method for generating countermeasure sample | |
| CN113538235B (en) | Training method and device for image processing model, electronic equipment and storage medium | |
| CN114202026B (en) | Multi-task model training method and device, multi-task processing method and device | |
| CN113177451A (en) | Training method and device of image processing model, electronic equipment and storage medium | |
| CN112949767A (en) | Sample image increment, image detection model training and image detection method | |
| CN113705362B (en) | Training method and device of image detection model, electronic equipment and storage medium | |
| CN111833391B (en) | Image depth information estimation method and device | |
| CN113344862A (en) | Defect detection method, defect detection device, electronic equipment and storage medium | |
| CN114417780B (en) | State synchronization method and device, electronic equipment and storage medium | |
| CN113515462B (en) | Method, apparatus, device and storage medium for testing | |
| CN114549941B (en) | Model testing method and device and electronic equipment | |
| CN115481594B (en) | Scoreboard implementation method, scoreboard, electronic equipment and storage medium | |
| CN116229211A (en) | Sample generation method, model training method, object detection method and device | |
| CN113781653B (en) | Object model generation method and device, electronic equipment and storage medium | |
| CN114581730A (en) | Training method of detection model, target detection method, device, equipment and medium | |
| CN114998649A (en) | Training method of image classification model, and image classification method and device | |
| CN113313049A (en) | Method, device, equipment, storage medium and computer program product for determining hyper-parameters | |
| CN114093006A (en) | Training method, device and equipment of living human face detection model and storage medium | |
| CN116416500B (en) | Image recognition model training method, image recognition device and electronic equipment | |
| CN114677564B (en) | Training sample generation method, deep learning model training method and device | |
| CN111523452B (en) | Method and device for detecting human body position in image | |
| CN116486197B (en) | Training method of image detection model, image detection method and image labeling method | |
| CN116386026A (en) | Training method of point cloud 3D detection model and point cloud detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |