JPWO2020142110A5

JPWO2020142110A5 -

Info

Publication number: JPWO2020142110A5
Application number: JP2021527195A
Authority: JP
Publication date: 2023-06-12
Anticipated expiration: 2039-04-23

Description

図２３Ｂにテスト／訓練データセット２３６０の例を示す。同図に簡潔に触れると、テスト／訓練データセット２３６０は、入力２３６１と、想定出力２３６３とを含む。概して、想定入力は、入力値（１または複数）２３６１－ｎを含み得る。ｎは正の整数である。例えば、入力値（１または複数）２３６１－１、入力値（１または複数）２３６１－２、入力値（１または複数）２３６１－３、入力値（１または複数）２３６１－４、入力値（１または複数）２３６１－５から入力値（１または複数）２３６１－Ｎが、入力２３６１について示される。各入力値２３６１－ｎについて、関連付けられた想定出力２３６３が示される。想定出力２３６３はそれぞれ、可視クラス２２１０または隠れクラス２２２０のいずれかに対応する。これに対する入力の分類のために、推論モデル２３２２が訓練される。一例として、可視クラス２２１０は、ペン、ナイフ、剣、定規、ハサミ、および鉛筆など、推論システム２３２２により分類されるオブジェクトのカテゴリに対応し得る。隠れクラス２２２０は、例えばペンを模したナイフ、定規を模した剣など、潜在的なプロービングを示し得るカテゴリに対応し得る。いくつかの例では、隠れクラス２２２０は、入力として許可されていない、ブラックリストに登録されているオブジェクトに対応し得る。隠れクラスに関連付けられた入力値（１または複数）２３６１－ｎは、推論モデル２３２２の誤分類を引き起こす意図で変換された入力に対応し得る。例えば、銃の画像と、蛇口の画像のモーフィングにより、例示的入力値（１または複数）２３６１－ｎが生成され得る。
An example of a test/training dataset 2360 is shown in FIG. 23B. Briefly referring to the figure, test/training dataset 2360 includes inputs 2361 and expected outputs 2363 . In general, hypothetical inputs may include input value(s) 2361-n. n is a positive integer. For example, input value(s) 2361-1, input value(s) 2361-2, input value(s) 2361-3, input value(s) 2361-4, input value(1 2361-5 through input value(s) 2361-N are shown for input 2361. For each input value 2361-n, the associated expected output 2363 is shown. Each expected output 2363 corresponds to either a visible class 2210 or a hidden class 2220 . An inference model 2322 is trained to classify the inputs to this. As an example, visibility class 2210 may correspond to categories of objects classified by reasoning system 2322, such as pens, knives, swords, rulers, scissors, and pencils. A hidden class 2220 may correspond to categories that may indicate potential probing, such as a knife resembling a pen, a sword resembling a ruler, and the like. In some examples, hidden class 2220 may correspond to blacklisted objects that are not allowed as inputs. The input value(s) 2361-n associated with the hidden class may correspond to inputs that have been transformed with the intention of causing inference model 2322 to misclassify. For example, morphing an image of a gun and an image of a faucet may produce exemplary input value(s) 2361-n.

Claims

a circuit and a memory coupled to the circuit, the memory, when executed by the circuit, causing the circuit to:
receiving input data from an input device;
generating output data including indications of either visible or hidden classes based in part on running an inference model on said input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on determining that the output data does not include the hidden class indication ;
generating obfuscated output including the visible class indication in response to determining that the output data includes the hidden class indication;
providing the obfuscated output to the output consumer in response to the determination that the output data includes an indication of the hidden class;
and a memory that stores instructions to cause the to execute.

2. The apparatus of claim 1, wherein the visible classes correspond to categories of objects classified by the inference system, and the hidden classes correspond to categories of objects indicative of potential probing.

The memory, when executed by the circuit, causes the circuit to execute sending an alert to a monitoring system based on the determination that the output data includes an indication of the hidden class. 3. The device of claim 1 or 2 , further storing.

The memory, when executed by the circuit, causes the circuit to:
collecting forensic data including indications of at least one of the input data and the input device in response to the determination that the output data includes indications of the hidden class;
4. The apparatus of claim 3, further storing instructions for performing: providing the forensic data to the surveillance system in response to the determination that the output data includes indications of the hidden class.

5. Apparatus according to any one of the preceding claims, wherein said visible class comprises a plurality of visible classes and said output data comprises an indication of said hidden class or one of said plurality of visible classes.

6. A method according to any one of claims 1 to 5, wherein said hidden class comprises a plurality of hidden classes and said output data comprises an indication of one of said visible classes or one of said plurality of hidden classes. device.

7. The apparatus of claim 6, wherein at least one of the plurality of hidden classes corresponds to a blacklisted object that is disallowed as the input data .

The memory, when executed by the circuit, causes the circuit to:
identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to the determination that the output data includes an indication of the hidden class; and
7. The apparatus of claim 6, further storing instructions for performing: generating obfuscated output including the identified one indication of the plurality of visible classes.

circuit and
a memory coupled to the circuit, the memory, when executed by the circuit, causing the circuit to:
receiving input data from an input device;
generating output data including an indication of either one of a plurality of visible classes or one of more hidden classes based in part on running an inference model on the input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on determining that the output data does not include the hidden class indication;
identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to determining that the output data includes an indication of the hidden class; and
generating obfuscated output including the identified one indication of the plurality of visible classes;
A memory that stores instructions to execute
A device comprising

6. Apparatus according to any one of the preceding claims, comprising a Trusted Execution Environment (TEE) comprising said circuit and said memory.

to the circuit ,
a procedure for receiving input data from an input device;
generating output data including indications of either visible or hidden classes based in part on running an inference model on said input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on a determination that the output data does not contain an indication of the hidden class;
generating obfuscated output including the visible class indication in response to determining that the output data includes the hidden class indication;
providing the obfuscated output to the output consumer in response to the determination that the output data includes an indication of the hidden class;
A program to run the

12. The program product of claim 11, wherein the visible classes correspond to categories of objects classified by the inference system and the hidden classes correspond to categories of objects that indicate potential probing.

13. The method according to claim 11 or 12 , for causing the circuit to further perform the procedure of sending an alert to a monitoring system based on the determination that the output data contains indications of the hidden class. program as described.

in said circuit,
collecting forensic data including indications of at least one of the input data and the input device in response to the determination that the output data includes indications of the hidden class;
providing the forensic data to the surveillance system in response to the determination that the output data includes the hidden class indication;
14. The program according to claim 13 , further causing the execution of

15. A program according to any one of claims 11 to 14, wherein said visible class comprises a plurality of visible classes and said output data comprises an indication of said hidden class or one of said plurality of visible classes.

16. A method according to any one of claims 11 to 15 , wherein said hidden class comprises a plurality of hidden classes and said output data comprises an indication of one of one or more visible classes of said plurality of hidden classes. program.

17. The program product of claim 16 , wherein at least one of the plurality of hidden classes corresponds to a blacklisted object that is not allowed as the input data .

in said circuit,
identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to the determination that the output data includes an indication of the hidden class; and
generating obfuscated output including the identified one indication of the plurality of visible classes;
17. The program according to claim 16 , further causing the execution of

to the circuit,
a procedure for receiving input data from an input device;
generating output data comprising an indication of either one of a plurality of visible classes or one of a plurality of hidden classes based in part on running an inference model on said input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on a determination that the output data does not contain an indication of the hidden class;
identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to the determination that the output data includes an indication of the hidden class; and
generating obfuscated output including the identified one indication of the plurality of visible classes;
A program to run the

16. A program according to any one of claims 11 to 15 , wherein said circuit is part of a Trusted Execution Environment (TEE).

A non-transitory computer-readable storage medium storing the program according to any one of claims 11 to 20 .

receiving input data from an input device;
generating output data including indications of either visible or hidden classes based in part on running an inference model on said input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on a determination that the output data does not contain an indication of the hidden class ;
generating obfuscated output including the visible class indication in response to determining that the output data includes the hidden class indication;
providing the obfuscated output to the output consumer in response to the determination that the output data includes an indication of the hidden class;
A computer-implemented method comprising :

23. The computer-implemented method of claim 22, wherein the visible classes correspond to categories of objects classified by the inference system, and the hidden classes correspond to categories of objects indicative of potential probing.

24. The computer-implemented method of claim 22 or 23 , comprising sending an alert to a monitoring system based on the determination that the output data contains indications of the hidden class.

collecting forensic data including indications of at least one of the input data and the input device in response to the determination that the output data includes indications of the hidden class;
25. The computer-implemented method of claim 24 , comprising providing the forensic data to the surveillance system in response to the determination that the output data includes indications of the hidden class.

26. The computer-implemented method of any one of claims 22-25, wherein the visible class comprises a plurality of visible classes, and wherein the output data comprises an indication of one of the hidden class or the plurality of visible classes. .

27. The computer-implemented method of claim 26, wherein the hidden class comprises a plurality of hidden classes, and wherein the output data comprises an indication of one of the plurality of hidden classes or one of the plurality of visible classes.

identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to the determination that the output data includes an indication of the hidden class; and
and generating the obfuscated output including the identified one indication of the plurality of visible classes.

receiving input data from an input device;
generating output data including an indication of either one of a plurality of visible classes or one of more hidden classes based in part on running an inference model on the input data;
determining whether the output data includes an indication of the hidden class;
providing the generated output data to an output consumer based on a determination that the output data does not contain an indication of the hidden class;
identifying one of the plurality of visible classes associated with the one of the plurality of hidden classes indicated by the output data in response to determining that the output data includes an indication of the hidden class; stages and
generating obfuscated output including the identified one indication of the plurality of visible classes;
A computer-implemented method comprising: