JP2019121141A - Device, secure element, program, information processing system and information processing method - Google Patents

Abstract

To provide a device or the like capable of securely executing processing using results of machine learning.SOLUTION: A device 1 includes: a storage unit that stores a learned model having a plurality of arithmetic elements, the learned model being a learned model generated by machine learning; an execution unit that executes arithmetic processing based on the learned model; and a secure unit that is a component more secure than the execution unit, and executes arithmetic processing related to a part of the arithmetic elements of the learned model in accordance with an arithmetic request from the execution unit. The execution unit outputs a final arithmetic result using a result of the arithmetic processing performed by the secure unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a device, a secure element, a program, an information processing system, and an information processing method.

  In recent years, IoT (Internet of Things) technology has rapidly spread and various devices are connected to the Internet. On the other hand, with the expansion of devices connected to the Internet, security problems resulting from device vulnerabilities have become apparent.

  In addition, interest in machine learning represented by deep learning as well as IoT is increasing. Along with this, machine learning technology is applied to cloud computing related to IoT, the cloud server which is a data center performs machine learning, builds a learned model, provides learning results to the IoT device, and performs various processing A system to perform is being studied (for example, Patent Document 1).

  On the other hand, in cloud computing where all data and functions are integrated on the cloud, there is a problem that latency (processing speed) delay can not be avoided because the communication distance between the user and the cloud server becomes long. is there. In order to address this problem, edge computing technology that shortens the communication distance and improves latency has attracted attention by performing a part of the processing performed by the cloud server on the client side closer to the user.

  Attention has also been drawn to the complementation of machine learning and edge computing described above. That is, by delegating a part of the machine learning operation to the IoT device which is the client, it is an idea that the operation result is transmitted to the user earlier than in cloud computing. For example, a cloud server performs machine learning and distributes learning results to an IoT device, and the IoT device develops a learned model, which is a learning result, on a memory, and executes various processes based on the learned model.

Unexamined-Japanese-Patent No. 2017-142654

  However, as described above, it is predicted from the security problem in the IoT device that a problem such as theft or imitation by a third party of the learned model developed on the device will be apparent.

  In one aspect, it is an object of the present invention to provide a device or the like capable of safely executing processing using machine learning results.

  A device according to one aspect is a learned model generated by machine learning, and a storage unit that stores a learned model having a plurality of arithmetic elements, and an execution unit that executes arithmetic processing based on the learned model And a secure unit which is more secure than the execution unit, and executes an operation process related to the operation element of a part of the learned model according to the operation request from the execution unit, the execution unit The final calculation result is output using the calculation result of the secure unit.

  A secure element according to one aspect is a secure element mounted on a device, which is a learned model generated by machine learning from the device, and an arithmetic processing based on the learned model having a plurality of arithmetic elements And an output unit configured to output an operation result to the device. It is characterized by

  A program according to one aspect is a learned model generated by machine learning in a device equipped with a secure element, storing a learned model having a plurality of computing elements in a storage unit, and using the learned model. A program for executing processing based on the operation processing, which requests the secure element to perform operation processing related to the operation element of a part of the learned model, and using the operation result by the secure element, It is characterized by outputting a final calculation result.

  A program according to one aspect is a trained model generated by machine learning in a device, wherein a trained model having a plurality of computing elements is held in a first execution environment, and computation based on the trained model A program that executes processing for executing processing in the first execution environment, and relates to the computing element of a part of the learned model in a second execution environment that is more secure than the first execution environment. An arithmetic processing is performed, and a final arithmetic result is output using the arithmetic result in the second execution environment.

  An information processing system according to one aspect is an information processing system including a device and a management apparatus capable of communicating with the device, wherein the device is a learned model generated by machine learning, and a plurality of the information processing system are generated. A storage unit for storing a learned model having an arithmetic element, an execution unit for executing arithmetic processing based on the learned model, and a component that is more secure than the execution unit according to an arithmetic request from the execution unit; And a secure unit that executes arithmetic processing related to part of the arithmetic elements of a learned model, the execution unit outputting a final arithmetic result using the arithmetic result by the secure unit, and the secure unit Notifies the management apparatus of the number of executions of the arithmetic processing, or information indicating that the arithmetic processing has been executed, and the management apparatus communicates from the secure unit. Based on, characterized by managing the number of executions.

  An information processing method according to one aspect is a learned model generated by machine learning, and executes a storage unit that stores a learned model having a plurality of computing elements, and an arithmetic process based on the learned model. The execution unit requests the component more secure than the execution unit to perform operation processing on the operation element of the part of the learned model, to the device including the execution unit; The arithmetic processing according to the part of the arithmetic elements may be performed, and the execution unit may execute a process of outputting a final arithmetic result using the arithmetic result of the secure component.

  In one aspect, processing using machine learning results can be performed safely.

It is a block diagram showing an example of composition of an IoT system. It is explanatory drawing which shows an example of the record layout of operation management DB. It is an explanatory view for explaining expansion processing of an inference model. It is explanatory drawing which shows an example of the basic structure of an inference model. It is an explanatory view about application processing of a learning parameter. It is an explanatory view about generation of a student model using a teacher model. It is an explanatory view for explaining inference processing. It is an explanatory view for explaining operation contents which a device main part and a secure element perform, respectively. It is explanatory drawing regarding the update process of the frequency | count of inference execution, and permission information. It is a flowchart which shows an example of the process sequence of the expansion process of an inference model. It is a flowchart which shows an example of the process sequence of an inference process. It is a flowchart which shows an example of a process procedure of the update process of the frequency | count of inference execution, and permission information. FIG. 18 is an explanatory diagram of inference processing according to the second embodiment; It is an explanatory view for explaining operation contents which a device main part, a secure element, and a server perform, respectively. FIG. 16 is a flowchart illustrating an example of a processing procedure of inference processing according to Embodiment 2. FIG. FIG. 16 is a flowchart illustrating an example of a processing procedure of inference processing according to Embodiment 2. FIG. It is a flowchart which shows the process sequence of the inference process which concerns on an applied Example. FIG. 18 is a block diagram showing an example of configuration of an IoT system according to a third embodiment. It is explanatory drawing regarding the update process of an inference model. It is a flowchart which shows an example of the process sequence of an update process of an inference model. It is explanatory drawing regarding the inference prohibition process based on the execution frequency of an inference process. It is a flowchart which shows an example of the process sequence of an inference prohibition process. It is a functional block diagram which shows operation | movement of the device of the form mentioned above.

Hereinafter, the present invention will be described in detail based on the drawings showing the embodiments.
Embodiment 1
FIG. 1 is a block diagram showing an example of the configuration of an IoT system. In the present embodiment, an inference process is performed in which an inference model 121, which is a product of machine learning, is installed in a device 1 which is an IoT terminal, and the device 1 infers appropriate output data from input data using the inference model 121. Describe the form in which In the present specification, “inference” is used as a term representing the whole process using machine learning results, and “inference model” is used as a term representing a learned model generated by machine learning.

  The IoT system has a device 1 and a server 3. The device 1 is an electronic device connected to the network N which is the Internet, and various devices such as a monitoring camera, a multifunctional terminal, an ECU (Electronic Control Unit) mounted on a vehicle, an abnormality monitoring device for production facilities and infrastructure facilities, etc. Can be In the present embodiment, the device 1 is described as a surveillance camera.

  The server 3 is a management device that manages the operation status of the device 1 and is communicably connected to the device 1 via the network N. In the present embodiment, the server 3 performs machine learning to generate the inference model 121, and distributes data of the generated inference model 121 (the latest model parameter 341 shown in FIG. 1) to the device 1. The device 1 develops the inference model 121 on the memory of the own device based on the data and performs inference processing. For example, when the device 1 is a surveillance camera, the device 1 infers (estimates) what object is included in the captured image, and notifies the inference result to the outside.

  Also, the server 3 collects data on the operation status of the device 1 based on the inference model 121, and manages the operation status of the device 1 on a database. As described later in detail, the server 3 permits the number of times of execution of inference processing executed in each device 1 in addition to the type and version information of the inference model 121 installed in each device 1, and inference processing in each device 1 Manage information such as whether to

  The device 1 has a device body 10 and a secure element (secure unit) 20. The device body 10 is a body portion of the device 1 configured by a SoC (System on Chip) and implementing many or all of the functions of the device 1. The secure element 20 is hardware physically or logically separated from the device body 10, and is a chip having tamper resistance against external attacks. The secure element 20 has a non-volatile memory inside, and securely stores data. The secure element 20 may be configured to be removable from the device 1 like a universal integrated circuit card (UICC). The device body 10 and the secure element 20 are mutually connected by a standard such as, for example, ISO (International Organization Standardization) 7816, SPI (Serial Peripheral Interface).

The device body 10 includes a control unit 11, a storage unit 12, an input / output I / F 13, a communication unit 14, an input unit 15, and an imaging unit 16.
The control unit 11 includes an arithmetic processing unit such as one or more central processing units (CPUs), a micro-processing unit (MPU), and a graphics processing unit (GPU), and reads out the program P1 stored in the storage unit 12 By performing the processing, various information processing and control processing related to the device 1 are performed. The storage unit 12 includes a random access memory (RAM), a read only memory (ROM), and the like, and stores a program P1 necessary for the control unit 11 to execute arithmetic processing and other data. The storage unit 12 also stores an inference model 121 built based on the inference model parameters 271 stored in the secure element 20, as described later.

  The input / output I / F 13 is an interface for performing input and output of information with the secure element 20, and as described above, performs input and output of information according to a standard such as ISO 7816 and SPI. The communication unit 14 includes a processing circuit for performing processing related to Internet communication, an antenna, and the like, and transmits and receives information with the server 3 and the like through the network N. The input unit 15 is an input interface that receives an input of information from the user, and may be in various forms such as a mechanical key, a touch panel, a voice input microphone, or the like according to the type of the device 1. The imaging unit 16 is an imaging mechanism having a Complementary Metal Oxide Semiconductor (CMOS) sensor or the like, and captures an image.

The secure element 20 includes a read unit 21, an authentication unit 22, an operation unit 23, a counter 24, a communication path establishment unit 25, an input / output I / F 26, and a storage unit 27.
The reading unit 21 reads data from the storage unit 27 and outputs the data to the device body 10. The authentication unit 22 confirms authentication information input from the device main body 10 as described later. The arithmetic unit 23 executes arithmetic processing based on the inference model 121. The counter 24 counts the number of times of execution of inference processing in the device 1. The communication path establishment unit 25 establishes a secret communication path 41 (see FIG. 7 and the like) in which the communication content is encrypted with the server 3. The input / output I / F 26 performs data input / output with the input / output I / F 13 of the device body 10.

  The storage unit 27 is a non-volatile memory, and stores an inference model parameter 271, an inference model 121, a permission flag 272, and a unique ID 273. The inference model parameter 271 is a parameter necessary for the device body 10 to expand the inference model 121 as described later. The inference model 121 is the same data as model data stored in the storage unit 12 of the device body 10, and is data of a learned model generated by machine learning. The permission flag 272 is permission / disapproval information indicating that execution of inference processing in the own device (device 1) is permitted or prohibited, and is set in accordance with an instruction from the server 3 as described later. The unique ID 273 is individual identification information unique to the secure element 20, and is, for example, a serial number of the secure element 20 or the like.

The server 3 includes a control unit 31, a main storage unit 32, a communication unit 33, and an auxiliary storage unit 34.
The control unit 31 includes arithmetic devices such as one or more CPUs and MPUs, and performs various information processing and control processes related to the server 3. The main storage unit 32 is a volatile memory such as a RAM, and temporarily stores data necessary for the control unit 31 to execute processing. The communication unit 33 includes a processing circuit that performs processing related to communication, and communicates with the device 1 and the like via the network N.

  The auxiliary storage unit 34 is a large capacity memory, a hard disk or the like, and stores a program P2 necessary for the server 3 to perform processing and other data. Further, the auxiliary storage unit 34 stores the latest model parameter 341 and the operation management DB 342. The latest model parameter 341 is data of the latest inference model 121 distributed to the device 1, and is a parameter necessary for the device 1 to deploy the inference model 121. The operation management DB 342 is a database that manages the operation status of the device 1.

  The auxiliary storage unit 34 may be an external storage device connected to the server 3. Also, the server 3 may be a multi server consisting of a plurality of computers, or may be a virtual machine virtually constructed by software.

  FIG. 2 is an explanatory view showing an example of the record layout of the operation management DB 342. As shown in FIG. The operation management DB 342 includes an ID column, an execution count column, an execution date column, an execution permission column, a type column, and a version column. The ID column stores a unique ID 273 which is identification information of each device 1 (secure element 20). The execution number column stores the number of executions of inference processing performed by each device 1 in association with the unique ID 273. The execution date and time column stores the date and time when the inference process was performed in each device 1 in association with the unique ID 273. The execution permission column stores permission / refusal information indicating that each device 1 permits or prohibits inference processing in association with the unique ID 273. The type column stores the type of the inference model 121 applied (installed) to each device 1 in association with the unique ID 273. The version column stores version information of the inference model 121 applied to each device 1 in association with the unique ID 273.

FIG. 3 is an explanatory diagram for explaining expansion processing of the inference model 121. As shown in FIG. FIG. 3 conceptually illustrates how the device body 10 reads the inference model parameter 271 from the secure element 20 and develops the inference model 121 on the memory based on the parameter.
As described above, the secure element 20 stores the inference model parameters 271 for expanding the inference model 121 generated by machine learning in the storage unit 27. In the present embodiment, the inference model 121 is described as a neural network generated by deep learning. Note that the machine learning algorithm is not limited to deep learning, and may be, for example, regression method, decision tree learning, Bayesian method, clustering, etc. Further, if the inference model 121 is limited to a neural network Instead, they may be linear models, decision trees, Bayesian networks, etc.

  For example, the server 3 periodically updates (learns) the inference model 121 and distributes the latest model parameter 341 to the device 1. The device 1 stores the latest model parameter 341 distributed from the server 3 in the secure element 20 as the inference model parameter 271. The machine learning may be performed mainly by the server 3 that is the distribution source of the parameter, or another computer may perform machine learning and store the learning result in the server 3.

  The inference model parameter 271 is a data group including a hyper parameter 271 a, a learning parameter 271 b, and version information 271 c. The hyper parameter 271 a is a setting value manually set to cause the computer to perform machine learning, and is a parameter that can uniquely identify the basic structure of the inference model 121. For example, when the inference model 121 is a neural network, the hyperparameter 271a is used for the number of inputs input to each neuron layer, the number of outputs output from each neuron layer, interlayer connection between neurons, and arithmetic processing of each neuron Information indicating an activation function or the like.

The learning parameter 271 b is a learning value learned by a computer by machine learning under the network structure defined by the hyper parameter 271 a. For example, when the inference model 121 is a neural network, the learning parameter 271b is a parameter such as a weighting coefficient or a bias value of each neuron. The server 3 receives the input of the teacher data with the correct value, and learns the learning parameter 271b to be applied to each neuron.
Machine learning may be unsupervised learning, semi-supervised learning, or the like. Machine learning may be reinforcement learning, and in this case, the inference model parameter 271 may be a Q value (Q (s, a); s is a state, a is an action).

  The version information 271 c is information indicating the number of versions of the inference model 121. As described above, the server 3 periodically updates the inference model 121 and distributes the latest model parameters 341. The version information 271 c indicates the number of versions according to the update.

The processing processes indicated by reference signs P1 to P8 in FIG. 3 will be described in order. It is assumed that the inference model 121 is not developed in the device body 10 at the initial stage.
The control unit 11 of the device body 10 requests the secure element 20 to read out the inference model parameter 271 (P1), for example, when the device 1 starts up or when the operation of an application involving inference starts.

  When a read request is received, the read unit 21 of the secure element 20 first performs operator authentication to determine whether the read request is valid. The reading unit 21 requests the authenticating unit 22 to confirm the authentication information necessary for authenticating the operator (the user of the device 1 or the like) (P2). When the request is received, the authentication unit 22 requests the device body 10 to output the authentication information (P3).

  When the output request for the authentication information is received, the control unit 11 of the device body 10 outputs the authentication information to the secure element 20 (P4). The authentication information is, for example, a PIN (Personal Identification Number) code input via the input unit 15, a password, biometric information and the like. The authentication information is not limited to these, as long as the legitimacy of the operator can be appropriately confirmed. The authentication unit 22 performs authentication by collating the authentication information acquired from the device body 10 with the authentication information (not shown) held in advance by the secure element 20.

  If the authentication fails, the secure element 20 does not output the inference model parameter 271 and ends the series of processes. If the authentication is successful, the authentication unit 22 holds that the authentication is completed in the secure element 20 and notifies the device body 10 that the authentication is completed.

  When the notification of the authentication completion is received, the control unit 11 of the device main body 10 makes a read request of the parameter again (P5). When the reading request is received, the reading unit 21 of the secure element 20 reads the inference model parameter 271 from the storage unit 27 (P6). The reading unit 21 outputs the read parameter to the device body 10 (P7).

  The control unit 11 of the device body 10 develops the inference model 121 on the memory based on the parameters acquired from the secure element 20 (P8). Specifically, as described below, the control unit 11 determines the basic structure of the neural network based on the hyper parameter 271 a of the inference model parameters 271, and determines each neuron of the neural network whose basic structure has been determined. The inference model 121 is developed by applying the learning parameter 271b to the program.

  FIG. 4 is an explanatory view showing an example of the basic structure of the inference model 121. As shown in FIG. FIG. 4 conceptually illustrates how the basic structure of the neural network is determined based on the hyper parameter 271a. The hyper parameter 271 a is data defining the number of inputs, the number of outputs, interlayer coupling and the like of each neuron layer, and is defined such that the structure can be uniquely identified for each neuron layer.

  In the example of FIG. 4, since the number of inputs of the 0th layer (input layer) is defined as “3”, the control unit 11 of the device main body 10 assumes input values x0 to x2. Further, since the number of outputs of the zeroth layer is “3”, the control unit 11 arranges three neurons. Further, since the interlayer coupling is “all coupling”, the control unit 11 inputs the output value of all neurons in the zeroth layer to all neurons in the next layer (first layer). Furthermore, since the activation function of the 0th layer is defined by ReLU (Rectified Linear Unit), the control unit 11 calculates the output value using the ReLU function in each neuron of the 0th layer.

  Likewise, the control unit 11 determines the structure in each neuron layer in the same manner. Thus, the control unit 11 determines the basic structure of the neural network from the input to the output as shown in FIG.

FIG. 5 is an explanatory diagram of an application process of the learning parameter 271b. FIG. 5 illustrates how a learning parameter 271b according to the machine learning result is applied to the basic structure of the neural network shown in FIG.
For example, in the secure element 20, learning parameters 271b to be applied to each neuron layer are held corresponding to the 0th to 3rd neuron layers defined by the hyper parameter 271a. Specifically, the secure element 20 stores parameters such as weighting factors and bias values to be applied to each neuron, and the control unit 11 applies the parameters to each neuron.

  In the tables of FIG. 3 and FIG. 5, the value of the table portion indicated by “from / to” corresponds to the weighting factor. Note that "from" indicates input values x0 to x2 as input sources, or neurons in the anterior layer, and "to" indicates neurons to be input targets. For example, in the 0th layer, when (from, to) = (0, 0), the value in the table is 0.1. This value indicates the weighting factor when the zeroth input (x0) is input to the zeroth neuron whose neuron number is "0". Therefore, the control unit 11 calculates the weight when the input value x0 is input to the zeroth neuron as 0.1. Likewise, the control unit 11 sets the weight when the input value x0 is input to the first neuron to 0.2 and the weight when the input value x0 is input to the second neuron. The weight when the input value x1 is input to the 0-th neuron is set to 0.1,.

  Also, “bias” in the tables of FIG. 3 and FIG. 5 indicates a bias value to be applied to the input value after multiplication of the weighting coefficient in each neuron. For example, the bias values in the zeroth layer are all 0.6. Therefore, the control unit 11 sets a bias value of 0.6 for each neuron in the zeroth layer.

  Likewise, the control unit 11 also sets weighting coefficients and bias values for the first to third neuron layers. Thus, the control unit 11 develops a learning parameter in the entire network, and generates a neural network that calculates final output values y0 to y2 from the first input values x0 to x2.

  The device body 10 performs predetermined inference processing using the inference model 121 (neural network) developed as described above. Although detailed illustration and description are omitted, for example, when the device 1 is a surveillance camera, the device body 10 acquires an image captured by the imaging unit 16 as input data, and inputs the input data to the inference model 121. , And perform processing of estimating (inferring) an object included in the image. The device body 10 inputs values such as chromaticity and luminance of each pixel of the captured image as input values into the inference model 121, and calculates output values. For example, the output value is a value indicating the probability that the object included in the image is a specific object for which the feature amount has been learned at the time of machine learning. The device body 10 outputs the output value to the server 3 or another device. For example, when the suspicious object is specified from the image, the device body 10 notifies that the suspicious object has been found.

  Although the device 1 reads out both the hyper parameter 271 a and the learning parameter 271 b from the secure element 20 and develops the inference model 121 in the above, the present embodiment is not limited to this. For example, in the device 1, the hyper parameter 271a is held in the device body 10, the learning parameter 271b is stored in the secure element 20, and the device body 10 reads only the learning parameter 271b from the secure element 20. May be expanded. As described above, only part of the inference model parameters 271 may be stored in the secure element 20.

  As described above, the device 1 stores the inference model parameter 271 in the secure element 20 physically or logically separated from the device body 10, reads out the inference model parameter 271 from the secure element 20, and performs inference Expand the model 121. By holding the inference model parameter 271 in a more secure component than the device body 10, the possibility of theft, imitation, etc. of the inference model 121 by a third party can be reduced.

  However, protecting the data of the inference model 121 itself may impair the security for the reason described below.

FIG. 6 is an explanatory diagram regarding generation of a student model using a teacher model. An algorithm for generating a learned model called distillation will be described with reference to FIG.
Distillation is equivalent to inference while reducing the size of the learned model by using the output of the existing learned model (neural network in the example of FIG. 6) as teacher data for other smaller models. Is a method of generating a student model that can

  The upper part of FIG. 6 conceptually illustrates a general deep learning method. In a general learning method, the training data to which the correct value (label) is added to the input value is given to the neural network, and the weighting coefficient, bias value, etc. are adjusted so that the output value of the neural network approaches the correct value ( learn. For example, when a learning model for image recognition is assumed, correct values of output values y0 and y1 that indicate whether an object shown in an input image is a specific object (for example, a dog or a cat) are given. In teacher data, since it is previously defined whether a certain input image is a dog or a cat, the correct value is given as a value of 0 or 1. On the other hand, the output value of the neural network calculated from the input value is a value of 0 to 1 expressed by the probability. The computer that has obtained the teacher data sets weighting coefficients, bias values, etc. so that the output values y0 and y1 calculated with probability values of 0 and 1 approach 0 or 1 which is the correct value (in FIG. Learn with point leader).

  On the other hand, in the learning method by distillation, a learned neural network is used as a teacher model, and the input and output values of the teacher model are used as teacher data, and the teacher data is made to learn to a student model smaller than the teacher model. . The lower side of FIG. 6 conceptually illustrates a learning method by distillation. In the example shown in FIG. 6, the student model has fewer neuron layers than the teacher model.

  Input and output values are given to the student model when inference processing is performed using the teacher model. Here, as the correct value of the output values y0 and y1, not the binary value of 0 or 1 indicating the inference result of dog or cat but the probability value output from the teacher model is a general learning method It is different. For example, as shown in the table on the lower side of FIG. 6, probability values of 0 to 1 are given to the student model as the correct values of the output values y0 and y1.

  In the distillation, using the output value of the teacher model represented by the probability as described above as the correct value, the weighting coefficient, the bias value, etc. are adjusted so as to minimize the difference from the output value of the student model. As a result, a learned model capable of performing equivalent inference while reducing the number of operation steps more than the teacher model is generated.

  According to the distillation algorithm, only input and output values of the teacher model are used as teacher data, and hyper parameters 271a defining the basic structure of the teacher model or learning parameters 271b of the teacher model are not referred to. Therefore, even if the inference model parameter 271 is black boxed as in the present embodiment, there is a possibility that the inference model 121 may be mimicked by extracting input / output values at the time of inference processing execution from the device body 10.

  Therefore, the device 1 according to the present embodiment solves the above-mentioned problem by executing part of the inference processing not by the device body 10 but by the secure element 20.

  FIG. 7 is an explanatory diagram for explaining an inference process. FIG. 8 is an explanatory diagram for explaining the contents of operations performed by the device body 10 and the secure element 20. In FIG. 7, the device main body 10 executes a part of operation of the inference model 121 and outputs the operation result to the secure element 20, and the secure element 20 executes the operation of the remaining part of the inference model 121 to finally It is conceptually illustrated that the inference result is returned to the device body 10. FIG. 8 shows a specific example of the arithmetic processing described with reference to FIG.

Processing processes indicated by reference signs P21 to P42 in FIG. 7 will be sequentially described using FIG. In FIG. 7, for convenience, the input / output I / F 13 of the device body 10 and the input / output I / F 26 of the secure element 20 are illustrated by a dotted rectangular frame.
First, the control unit 11 of the device body 10 inputs input data to be inferred into the inference model 121, and executes a part of operation of the inference model 121 (P21). For example, when the inference model 121 is a neural network, the inference model 121 has a plurality of neuron layers, that is, a plurality of operation elements. The control unit 11 performs an operation of a former element including an input layer among a plurality of arithmetic elements constituting the inference model 121.

  In the example of FIG. 8, in the neural network in which the device body 10 includes 0th to 3rd layers, calculation of the 0th layer, which is the input layer, and the next first layer is performed. The device body 10 inputs the input values x0 to x2 (for example, pixel values of the input image) relating to the input data into the 0th layer, performs the operation of the 0th layer and the 1st layer, and calculates That is, the operation result of the former element is obtained.

  When the calculation of the former element is completed, the control unit 11 of the device body 10 outputs the calculation result of the former element to the secure element 20 and requests the secure element 20 to calculate the latter element of the inference model 121 excluding the former element. (P22). Specifically, the control unit 11 outputs, together with the calculation result of the former element, specification information for specifying the latter element to be operated by the secure element 20, and requests the calculation. The designation information is information that allows the secure element 20 to identify an element to be operated by itself, and is, for example, in the case of a neural network, information indicating a layer number of a neuron layer to be operated.

  In the example of FIG. 8, the device body 10 inputs the calculation result of the first layer to the secure element 20 as an input value, and the calculation of the second layer and the third layer, which are subsequent elements of the inference model 121, is secure element 20. To request. The device body 10 combines the layer number “2” of the start layer of the arithmetic processing and the layer number “3” of the end layer so that the secure element 20 can determine the neuron layer to be calculated. Notify and instruct the operation specifically.

  When an operation request is received from the device body 10, first, the authentication unit 22 of the secure element 20 performs operator authentication, and confirms whether or not the operation request is from a valid operator (P23). Specifically, as at the time of expansion of the inference model 121, the authentication unit 22 acquires authentication information such as a PIN code and a password from the device body 10, and authenticates the operator based on the authentication information. If the authentication fails, the authentication unit 22 notifies the device body 10 that the authentication has failed, and stops the inference process. In addition, for example, when operator authentication has already been performed, the authentication process may be skipped. If the authentication is successful, the authentication unit 22 passes the calculation result of the former element obtained from the device body 10 and the designation information of the latter element to the calculation unit 23, and requests the calculation (P24).

  When an operation request is received from the authentication unit 22, the operation unit 23 first causes the counter 24 to count the number of times of execution of the inference process (operation process) to check whether the number of times of inference execution has reached a certain number (P25 ). For example, the counter 24 holds the limited number of times of inference execution number, and decrements the remaining number each time an operation request is received from the device main body 10. The limit number of times of inference execution is different for each secure element 20 of each device 1. As will be described later, the server 3 individually assigns a limited number of times to the secure element 20 of each device 1 and centrally manages the number of inference execution times of each device 1 on the operation management DB 342. The counter 24 checks whether the remaining number has reached 0 or not, thereby confirming whether or not the limit number given by the server 3 has been reached. If the number of executions reaches a predetermined number, the arithmetic unit 23 does not execute the arithmetic processing, notifies the device main body 10 that the number of executions has reached a predetermined number, and stops the inference processing. As a result, it is possible to prevent a situation in which a large amount of input / output values are exploited by execution of inference processing due to unauthorized access from the outside.

  If the number of executions has not reached the predetermined number, the operation unit 23 refers to the permission flag 272 and confirms whether inference processing is permitted (P26). As described above, the permission flag 272 is permission information indicating permission or prohibition of inference processing. For example, the permission flag 272 is managed by the server 3 on the operation management DB 342 in the same manner as the number of inference executions, and the secure element 20 sets the permission flag 272 to “permit” or “prohibit” according to an instruction from the server 3 . If the permission flag 272 is "prohibit", the operation unit 23 does not perform the operation, and notifies the device main body 10 that the inference process is prohibited to stop the process.

  When the permission flag 272 is “permitted”, the operation unit 23 refers to the data of the inference model 121 stored in the storage unit 27 and executes the operation process of the subsequent element requested from the device body 10 (see FIG. P27). For example, when the inference model 121 is a neural network, the operation unit 23 performs an operation of a subsequent element including an output layer among a plurality of operation elements (neuron layers) constituting the inference model 121.

  In the example of FIG. 8, it is illustrated that the secure element 20 performs the calculation of the second layer among the 0th to third layers and the third layer which is the output layer. The secure element 20 inputs the calculation result of the first layer calculated by the device main body 10 to the second layer which is the start layer of the arithmetic processing designated from the device main body 10. The secure element 20 executes operations of the second and third layers to obtain output values y0, y1 and y2. As described above, in the present embodiment, the secure element 20 performs the operation of the subsequent element of the inference model 121, and calculates the output values y0, y1, and y2 corresponding to the inference result.

  The operation unit 23 outputs the operation result of the post-stage element to the device body 10 (P28). Here, the arithmetic unit 23 performs a fraction process of converting the probability value into a binary value of 0 or 1 without directly outputting the output value of the inference model 121 expressed by the probability to the device body 10, and the converted value Is output to the device body 10.

  For example, as shown in FIG. 8, a conversion layer for converting the probability value represented by 0 to 1 into a binary value of 0 or 1 is prepared as an operation element constituting the inference model 121. The secure element 20 inputs the output values y0, y1, y2 output from the output layer (third layer) to the conversion layer. The secure element 20 rounds the output values y0, y1, y2 in the conversion layer and outputs a value of 0 or 1.

  As described above, the secure element 20 expresses the output value of the inference model 121 as a binary value of 0 or 1 without outputting it as a probability value and outputs it. Therefore, even if the output value is exploited from the device body 10, implication of the inference model 121 becomes difficult because it is not expressed by the probability.

  When the calculation result of the subsequent element is acquired from the secure element 20, the control unit 11 of the device body 10 outputs an inference result related to a series of inference processing using the acquired calculation result. For example, when the device 1 is a surveillance camera, the control unit 11 outputs the inference result obtained by inferring the object included in the input image to the outside. Thus, the device body 10 makes it difficult to imitate the inference model 121 by using the operation result in the secure element 20 as the inference result.

  Furthermore, in the present embodiment, after the execution of the inference process, the secure element 20 notifies (reports) the server 3 that the inference process has been executed, so that the server 3 manages the number of times of execution of the inference process in each device 1. . First, the counter 24 of the secure element 20 delivers the counted number of inference execution times to the communication path establishment unit 25 (P29). Then, the communication path establishment unit 25 establishes an end-to-end confidential communication path 41 between the device 1 and the server 3 (P30).

  The secret communication path 41 is, for example, a communication path in which the communication content is encrypted by a protocol of TLS (Transport Layer Security). The communication path establishment unit 25 establishes the secret communication path 41 with the server 3 via the communication unit 14 which is a communication interface with the network N in the device main body 10.

  The secret communication path 41 may be any type as long as the communication content can be concealed, and for example, a protocol such as SSL (Secure Sockets Layer) may be adopted. Further, although the secret communication path 41 is established through the physical communication means (communication unit 14) of the device main body 10 in the above, the secure element 20 can establish the secret communication path 41 with the server 3 If the secure element 20 itself has physical communication means (LAN, Wi-Fi (registered trademark), etc.), the secure element 20 directly establishes the secret communication path 41 with the server 3. It is also good.

  The communication path establishment unit 25 notifies the server 3 of the number of inference execution times counted up to the present via the secret communication path 41 (P41). Specifically, the communication channel establishment unit 25 sets the unique ID 273, which is individual identification information of the secure element 20, and the current value (remaining number of times) of the counter 24 together with the current execution date and time to execute the inference processing. Notify The control unit 31 of the server 3 stores the notified inference execution count in the operation management DB 342 (P42). Specifically, the control unit 31 stores the number of remaining notified inference processes and the execution date in the operation management DB 342 in association with the unique ID 273 notified from the secure element 20.

  As described above, the secure element 20 notifies the server 3 of the inference execution count, and the server 3 manages the execution count of each device 1 (secure element 20). Thereby, the number of times of inference execution in each device 1 is centrally managed.

  FIG. 9 is an explanatory diagram of the process of updating the number of times of inference execution and the permission / refusal information. As mentioned above, in the present embodiment, the server 3 manages the number of times of execution of inference processing in each device 1 and permission information of the inference processing. Here, using FIG. 9, the server 3 communicates with each device 1 as one of the specific management methods of the inference execution frequency and the permission / refusal information, and the inference execution frequency and permission / refusal information held in the secure element 20 The process of updating the In FIG. 9, for convenience of illustration, the communication unit 14 of the device body 10 and the communication unit 33 of the server 3 are illustrated by dotted rectangular frames.

  For example, when the administrator of this system updates the number of inference execution times and permission information linked to the unique ID 273 of each device 1 in the operation management DB 342, the server 3 notifies each device 1 of the updated contents and sends it to the secure element 20. Update the number of inference execution times and permission information that is held.

  For example, the server 3 refers to the operation management DB 342 by batch processing to confirm whether there is a device 1 for which the number of times of inference execution or permission / refusal information has been changed (P41). If there is a device 1 for which the number of times of inference execution or permission information has been changed, the server 3 identifies the unique ID 273 of the device 1.

  The server 3 establishes the secret communication path 41 with the secure element 20 of the device 1 of the identified unique ID 273 (P42). Then, the server 3 transmits the updated content regarding the number of times of inference execution or permission / refusal information to the secure element 20 via the secret communication path 41 (P43).

  When the update content is received from the server 3, the communication channel establishment unit 25 of the secure element 20 stores the update content in the secure element 20, and updates the number of times of inference execution or permission / refusal information (P44). For example, as illustrated in FIG. 9, the communication channel establishment unit 25 sets the remaining number of inference processing held by the counter 24 or the value of the permission flag 272 held in the storage unit 27 to the value received from the server 3. Overwrite.

  As described above, the secure element 20 synchronizes the inference execution count and permission information set in the operation management DB 342 with the inference execution count and permission information that the secure element 20 holds. The secure element 20 determines the propriety of the inference process according to the number of times of inference execution and permission / refusal information instructed from the server 3 as described above. This enables remote control of the secure element 20 from the server 3.

  Although the number of times of inference execution and permission / refusal information are updated in accordance with the manual setting by the administrator in the above, the present embodiment is not limited to this. For example, the server 3 may communicate with the secure element 20 and automatically reset the remaining number to a predetermined default value when the remaining number becomes 0 by notification from the secure element 20. As described above, the server 3 can remotely control the number of inference execution times and the permission / refusal information held by the secure element 20, and the timing of update and the process content are not particularly limited.

FIG. 10 is a flowchart showing an example of the processing procedure of the expansion processing of the inference model 121. The processing content of the expansion processing of the inference model 121 will be described based on FIG.
For example, the device 1 executes the following processing at a predetermined timing, such as when the device 1 is activated or when an application involving inference processing is activated. The control unit 11 of the device body 10 requests the secure element 20 holding parameters necessary for developing the inference model 121 to read the parameters (step S11). As described above, the secure element 20 is a chip having tamper resistance, is physically separated from the device body 10, and securely holds parameters for the inference model.

  When receiving the parameter read request from the device body 10, the secure element 20 requests the device body 10 to output authentication information required for operator authentication (step S12). When receiving the output request of the authentication information, the control unit 11 of the device main body 10 receives the input of the authentication information through the input unit 15, and outputs the input to the secure element 20 (step S13). The authentication information is, for example, a PIN code, a password, biometric information, etc., but may be any information that can properly confirm the legitimacy of the operator.

  The secure element 20 performs operator authentication based on the authentication information (step S14). The secure element 20 determines whether the operator authentication is successful (step S15). If the authentication fails (S15: NO), the secure element 20 does not read the parameter and ends the series of processes. If the authentication is successful (S15: YES), the secure element 20 reads the inference model parameter 271 from the storage unit 27 (step S16). For example, when the inference model 121 is a neural network generated by deep learning, the secure element 20 reads out hyper parameters 271a defining the basic structure of the neural network and learning parameters 271b applied to each neuron layer. The secure element 20 outputs the read parameter to the device body 10 (step S17).

  When receiving the output of the parameter from the secure element 20, the control unit 11 of the device body 10 develops the inference model 121 in the storage unit 12 based on the parameter (step S18). Specifically, the control unit 11 determines the basic structure of the neural network according to the hyper parameter 271a, and develops the inference model 121 by applying the learning parameter 271b to each neuron of the neural network of the determined basic structure. The control unit 11 ends the series of processes.

FIG. 11 is a flowchart illustrating an example of a process procedure of inference processing. The processing content of the inference processing based on the inference model 121 will be described based on FIG.
The control unit 11 of the device body 10 acquires input data to be input to the inference model 121 (step S31). For example, when the device 1 is a surveillance camera, the control unit 11 acquires image data captured by the imaging unit 16. The control unit 11 executes arithmetic processing of the preceding elements constituting the inference model 121 (step S32). For example, when the inference model 121 is a neural network, the control unit 11 executes an operation of one or more neuron layers including the input layer of the neural network.

  The control unit 11 outputs the calculation result of the former element, and requests the secure element 20 to perform the arithmetic process of the latter element of the inference model 121 different from the former element (step S33). Specifically, the control unit 11 adds the information (for example, the layer number of the neuron layer, etc.) specifying the operation element to be operated by the secure element 20 to the secure element 20 together with the operation result of the former element operated by the device body 10. Output.

  When an operation request is received from the device body 10, the secure element 20 performs an authentication process to confirm whether the operation request is a genuine operator (step S34). For example, the secure element 20 queries the device body 10 for authentication information such as PIN information and a password, and the device body 10 returns authentication information. The secure element 20 performs an authentication process based on the authentication information.

  The secure element 20 determines whether the authentication is successful (step S35). When it is determined that the authentication is successful (S35: YES), the secure element 20 counts the number of times of execution of inference processing (calculation processing) in the device 1 (step S36). The secure element 20 determines whether the counted number of executions has reached a certain number (step S37). The number of times used as the determination reference is the limited number of times of inference processing permitted for the device 1, and the server 3 manages the limited number of times for each device 1 (secure element 20).

  If it is determined that the number of executions has not reached the fixed number (S37: NO), the secure element 20 checks the permission flag 272 and determines whether inference processing (calculation processing) based on the inference model 121 is permitted. (Step S38). The permission flag 272 is permission / disapproval information indicating that inference processing in the device 1 is permitted or prohibited, and the server 3 manages permission or prohibition for each device 1 as in the number of times of inference execution.

  If it is determined that the authentication is not successful (S35: NO), if it is determined that the number of executions has reached a certain number (S37: YES), or if it is determined that the inference process is not permitted (S38: NO) The secure element 20 notifies the device body 10 that the execution of the requested arithmetic processing is refused (step S39). When the notification of operation refusal is acquired from the secure element 20, the control unit 11 of the device main body 10 stops the inference processing (step S40), and ends the series of processing.

  If it is determined that the inference process is permitted (S38: YES), the secure element 20 executes an arithmetic process of the subsequent element requested from the device body 10 (step S41). For example, when the inference model 121 is a neural network, the secure element 20 inputs, as an input value, the operation result of the preceding element acquired from the device body 10 to the neuron layer specified from the device body 10. The neuron layer designated from the device body 10 is one or more neuron layers constituting a neural network, and is a neuron layer including an output layer. The secure element 20 performs operations on each neuron up to the output layer and calculates the final output value.

  The secure element 20 performs a rounding process on the calculation result (output value) in step S41, and converts the calculation result (step S42). The secure element 20 outputs the converted operation result to the device body 10 (step S43). When the calculation result is acquired from the secure element 20, the control unit 11 of the device body 10 outputs a final inference result (calculation result) using the calculation result (step S44), and ends the series of processing.

  After executing the process of step S43, the secure element 20 establishes the secret communication path 41 in which the communication content is encrypted with the server 3 (step S45). The secure element 20 notifies the server 3 of the inference execution count counted in step S36 via the secret communication path 41 (step S46). Specifically, the secure element 20 notifies the server 3 of the unique ID 273 of the secure element 20 and the execution date and time of the inference process, along with the counted number of times of inference execution. When the notification of the number of inference execution times is acquired from the secure element 20, the control unit 31 of the server 3 manages the number of inference execution times notified and the execution date and time in association with the unique ID 273 of the device 1 (secure element 20). It is stored in the DB 342 (step S47). The control unit 31 ends the series of processes.

FIG. 12 is a flow chart showing an example of the procedure of updating the number of times of inference execution and permission / refusal information. Based on FIG. 12, a process in which the server 3 updates the count value of the number of inference execution times held in the secure element 20 of each device 1 and the permission / refusal information will be described.
For example, the server 3 starts a series of processing by batch processing. The control unit 31 of the server 3 checks the operation management DB 342, and determines whether there is a device 1 for which the number of inference execution times or permission / refusal information has been updated (step S71). As described above, the operation management DB 342 stores the number of times of inference execution of each device 1 and permission information indicating permission or prohibition of inference processing in association with the unique ID 273 of each device 1. For example, the server 3 receives an update input of each data from the administrator of this system, and stores the update value in the operation management DB 342. The control unit 31 refers to the operation management DB 342 in batch processing to detect an update.

  If it is determined that there is no updated device 1 (S71: NO), the control unit 31 ends the series of processing without performing any particular processing. If it is determined that there is a device 1 that has been updated, the control unit 31 specifies the unique ID 273 of the device 1 for which the number of times of inference execution or permission information has been updated (step S72).

  The control unit 31 establishes the secret communication path 41 with the secure element 20 of the device 1 indicated by the unique ID 273 (step S73). The control unit 31 transmits the updated number of executions or permission information to the secure element 20 via the opened secret communication channel 41 (step S74).

  When the number of executions or permission information is received from the server 3, the secure element 20 of the device 1 updates the number of executions or permission information held by itself to the received number of executions or permission information (step S 75). The secure element 20 ends the series of processes.

  Although the secure element 20 notifies the server 3 of the number of executions immediately after the execution of the inference process, the timing of the notification is not limited to immediately after the execution, and may be performed independently. Also, although the secure element 20 notifies the count value (remaining number) of the number of inference execution times, it may notify only the fact indicating that it has been executed, and the server 3 may separately count the number of execution times. Furthermore, although the execution date and time notified from the secure element 20 is stored in the operation management DB 342 in the above, when the notification of the number of executions is received, the server 3 adds a time stamp and stores this in the operation management DB 342 May be That is, transmission of the execution date and time from the secure element 20 is not essential.

  In the above, the device body 10 is in charge of the former element of the inference model 121, and the secure element 20 is in charge of the latter element operation. However, it is an optional design matter which processing entity calculates which operation element. There is no particular limitation. As will be described later, in some embodiments, a design may be envisioned that directs all operations to the secure element 20. In addition, since the device body 10 and the secure element 20 each perform only part of the inference model 121, at least a part of data of the inference model 121 which itself performs the operation may be stored. It is not necessary to hold the data of Further, although the device body 10 designates the computing element operated by the secure element 20 in the above, for example, the computing element which each is in charge may be fixed in advance. In this case, the device body 10 is a computing element It is not necessary to output to the secure element 20 designation information (for example, a layer number of a neuron layer) that designates. That is, the device main body 10 only needs to input the calculation result calculated at least by itself or the input data received from the control unit 11 to the secure element 20 and request the calculation processing.

  As described above, according to the first embodiment, the device 1 executes the computation of a part of the computation elements of the inference model 121 with the secure element 20 that is more secure than the device body 10. Thereby, the device 1 can safely execute processing based on the machine learning result.

  Further, according to the first embodiment, the device 1 executes the operation of the former element of the inference model 121 in the device body 10, and the operation of the latter element in which the final output value is obtained, in the secure element 20. . Thereby, appropriate measures can be taken against the above-described problems of distillation, and safety can be enhanced.

  Further, according to the first embodiment, the secure element 20 performs rounding of the output value represented by the probability, and returns the converted output value to the device body 10. Thereby, even if the output value is extracted from the device body 10, it is possible to avoid the extraction of the probability value necessary for the distillation, and the security can be further improved.

  Further, according to the first embodiment, the secure element 20 performs operator authentication, and executes arithmetic processing only when the authentication is successful. This can further enhance the safety.

  Further, according to the first embodiment, by limiting calculation processing when the number of times of execution of inference processing (calculation processing) reaches a certain number of times, excessive execution of calculation processing due to an unauthorized operation from the outside can be prevented. And improve safety.

  Further, according to the first embodiment, the server 3 can centrally manage the number of inference execution times in each device 1 by notifying the server 3 of the counted number of inference execution times or that the inference process has been executed. it can.

  Further, according to the first embodiment, the server 3 can appropriately manage the operation status of the device 1 as an execution history by not only notifying the number of executions but also notifying together the execution date and time. .

  Also, according to the first embodiment, remote control of inference processing in the device 1 can be performed by updating the number of inference execution times counted by the secure element 20 from the server 3.

  Further, according to the first embodiment, the secure element 20 having high security holds the permission / disapproval information (permission flag 272), and by determining whether the inference is possible or not according to the permission / permission information, an unauthorized operation is performed from the outside. We can respond appropriately to the situation.

  Further, according to the first embodiment, by setting the permission information held by the secure element 20 according to the instruction of the server 3, remote control of the device 1 (secure element 20) from the server 3 becomes possible.

  Further, according to the first embodiment, by performing communication between the server 3 and the secure element 20 via the secret communication path 41, it is possible to safely transmit and receive the number of executions, permission / refusal information, etc. It is possible to prevent an instruction or notification from being falsified on the communication channel.

Second Embodiment
In the first embodiment, the form has been described in which part of operation processing of the inference model 121 is executed in the secure element 20 in the device 1. In this embodiment, a mode will be described in which the server 3 outside the device 1 performs part of the arithmetic processing to be executed by the secure element 20. The same reference numerals as in the first embodiment denote the same parts as in the first embodiment, and a description thereof will be omitted.

  FIG. 13 is an explanatory diagram of inference processing according to the second embodiment. The process of the inference process according to the present embodiment will be described based on FIG. In addition, since the process shown to code | symbol P21-P26 is the same as that of Embodiment 1, the same code | symbol is attached | subjected here and description is abbreviate | omitted. In the present embodiment, processing processes indicated by reference numerals P41 to P49 will be sequentially described.

  After confirming the number of times of inference execution and the permission flag 272 (P25, P26), the operation unit 23 of the secure element 20 processes some of the operation elements of the inference model 121 for which the device body 10 requests the operation. The arithmetic processing is executed (P41). For example, when the inference model 121 is a neural network, the operation unit 23 executes the operation on only a part of neuron layers out of the plurality of neuron layers specified from the device body 10 except at least the output layer.

  FIG. 14 is an explanatory diagram for describing the contents of operations performed by the device body 10, the secure element 20, and the server 3 respectively. FIG. 14 illustrates how the device body 10 performs the operation of the zeroth layer, the secure element 20 performs the operation of the first layer, and the server 3 performs the operations of the second and third layers.

  As shown in FIG. 14, in the present embodiment, the device body 10 performs the calculation of the zeroth layer, and passes the calculation result of the zeroth layer to the secure element 20 as an input value. Then, the device body 10 designates the remaining first to third layers as neuron layers to be calculated, and requests the secure element 20 to execute the calculation processing.

  When receiving the operation request from the device body 10, the secure element 20 executes the operation of the neuron layer specified by the device body 10. Here, the secure element 20 does not operate on all designated neuron layers, but only on some neuron layers. For example, as illustrated in FIG. 14, the secure element 20 performs the operation only on the first layer which is the operation start layer among the first to third layers specified from the device body 10.

  Then, the secure element 20 requests the server 3 to perform the operation of the remaining second and third layers of the neuron layer specified by the device body 10 in which the self does not perform the operation. Specifically, as shown in FIG. 14, the secure element 20 requests the calculation result of the first layer to be an input value, specify the second layer and the third layer as calculation targets, and execute such calculation. .

  Returning to FIG. 13, after executing the arithmetic processing, the arithmetic unit 23 delivers the arithmetic result to the communication path establishment unit 25 (P42). The communication path establishment unit 25 establishes the secret communication path 41 with the server 3 (P43). Then, the communication path establishment unit 25 transmits the calculation result calculated by the calculation unit 23 to the server 3 and requests to execute a part of the calculation of the inference model 121 (P44). Specifically, the communication path establishment unit 25 transmits, to the server 3, specification information for specifying the calculation element to be calculated by the server 3 and the unique ID 273 of the secure element 20 together with the calculation result calculated by the calculation unit 23. .

  When an operation request is received, the control unit 31 of the server 3 first counts the number of inference executions in the device 1 of the request source, and confirms whether the number of inference executions has reached a certain number (P45). As described in the first embodiment, the operation management DB 342 stores the inference execution count (remaining count) in each device 1 in association with the unique ID 273 of the secure element 20. The control unit 31 refers to the unique ID 273 acquired from the secure element 20, and decrements the remaining number associated with the unique ID 273. Then, the control unit 31 confirms whether the remaining number of times has reached 0, that is, a predetermined number of times. The control unit 31 stores the decremented value together with the execution date and time in the operation management DB 342, and updates the inference execution count.

  If the number of times has reached a predetermined number, the control unit 31 does not execute the arithmetic process, and notifies the device 1 that the number of times of execution has reached a predetermined number to stop the inference process. On the other hand, if the number of executions has not reached the predetermined number, then the control section 31 checks whether or not the inference process in the request source device 1 is permitted (P46). Specifically, as described above, the control unit 31 refers to the permission information associated with the unique ID 273 from the operation management DB 342 to determine permission or prohibition of the inference process. If the inference process is not permitted, the control unit 31 does not execute the arithmetic process, notifies the device 1 that the inference process is prohibited, and stops the process.

  When the inference process is permitted, the control unit 31 executes an arithmetic process of part of the inference model 121 requested from the secure element 20 (P47). Specifically, as described above, the control unit 31 uses the calculation result obtained from the secure element 20 as an input value, and one or more neuron layers including the output layer of the neural network designated from the secure element 20. Perform the operation of

  Further, as in the first embodiment, the control unit 31 performs a rounding process on the output value from the output layer of the neural network, and outputs the probability-expressed output value as a binary value of 0 or 1. (See FIG. 14).

  The control unit 31 transmits the converted output value as the calculation result to the secure element 20 via the secret communication path 41 (P48). The secure element 20 returns the calculation result obtained from the server 3 as an output value for the calculation request from the device body 10 (P49). The control unit 11 of the device body 10 outputs an inference result using the output value returned from the secure element 20.

  Although the secure element 20 and the server 3 share the calculation processing requested from the device main body 10 in the above description, the present embodiment is not limited to this, and the secure element 20 does not perform any special calculation. The processing request may be transferred to the server 3 to execute the processing. That is, as long as the secure element 20 or the server 3 can execute the arithmetic processing requested from the device main body 10, it does not limit which of the operations is to be burdened.

FIG. 15 and FIG. 16 are flowcharts showing an example of the processing procedure of the inference processing according to the second embodiment. The processing content of the inference processing according to the present embodiment will be described based on FIG. 15 and FIG.
With reference to the permission flag 272, when it is determined that the inference process is permitted (S38: YES), the secure element 20 executes the following process. The secure element 20 executes an operation on some of the operation elements of the inference model 121 specified from the device body 10 (step S201). For example, when the inference model 121 is a neural network, the secure element 20 executes the operation of one or more neuron layers including at least the operation start layer designated from the device body 10. In the case where the secure element 20 does not execute the arithmetic process, the process of step S201 may be skipped.

  The secure element 20 establishes the secret communication path 41 with the server 3 (step S202). The secure element 20 transmits the calculation result in step S201 to the server 3 via the secret communication channel 41, and requests the server 3 to execute part of the calculation processing of the inference model 121 (step S203). Specifically, the secure element 20 transmits specification information for specifying the calculation element (neuron layer) to be calculated by the server 3 and the unique ID 273 of the secure element 20, together with the calculation result calculated by itself.

  When an operation request is received from the secure element 20, the control unit 31 of the server 3 counts the number of times of execution of inference processing in the device 1 that is the transmission source of the operation request (step S204). Specifically, the control unit 31 reads out the inference execution count (remaining count) of the device 1 associated with the unique ID 273 based on the unique ID 273 received from the secure element 20 with reference to the operation management DB 342. The control unit 31 counts the number of executions by decrementing the read remaining number. The control unit 31 stores the counted number of times of execution in the operation management DB 342 together with the current execution date and time.

  The control unit 31 determines whether the counted number of inference execution times has reached a predetermined number (step S205). If it is determined that the predetermined number of times has not been reached (S205: NO), the control unit 31 refers to the operation management DB 342 and determines whether inference processing at the request source device 1 is permitted ( Step S206). Specifically, as in step S204, the control unit 31 refers to the permission / disapproval information (permission flag) associated with the unique ID 273 to confirm whether the device 1 has been permitted or prohibited to perform the inference process. When it is determined that the inference process is permitted (S206: YES), the control unit 31 shifts the process to step S210.

  If it is determined that the number of times of inference execution has reached a certain number (S205: YES), or if it is determined that inference processing is not permitted (S206: NO), the control unit 31 rejects the operation request, The secure element 20 is notified via the secret communication path 41 (step 207). The secure element 20 transfers the notification to the device body 10 (step S208). When the notification is acquired, the device body 10 stops the inference process (step S209), and ends the series of processes.

  Moving to FIG. 16, in the case of YES in step S206, the control unit 31 of the server 3 executes a part of the operation of the inference model 121 according to the request from the secure element 20 (step S210). Specifically, the control unit 31 inputs the calculation result (input value) acquired from the secure element 20 into the designated neuron layer, and calculates the final output value.

  The control unit 31 performs a rounding process on the calculation result of step S210 and converts an output value (step S211). The control unit 31 transmits the converted output value (operation result) to the secure element 20 (step S212). The secure element 20 transfers the output value acquired from the server 3 to the device body 10 (step S213). The control unit 31 outputs an inference result using the output value acquired from the secure element 20 (step S214), and ends the series of processes.

  In the above, although the server 3 performs the operation of the remaining operation element (neuron layer including the output layer) after the secure element 20 performs the operation, for example, the secure element 20 remains after the server 3 performs the operation. The calculation of the calculation element may be performed. That is, it is only necessary that the server 3 can execute the operation of at least a part of the operation elements requested by the device body 10, and the context of the operation process is not particularly limited.

  As described above, according to the second embodiment, by causing the server 3 to execute a part of the arithmetic processing based on the inference model 121, for example, it is possible to cope with the case where the computing capacity is insufficient with only the secure element 20. it can.

  According to the second embodiment, it is possible to share the inference process with the device body 10, the secure element 20, and the server 3 and perform the process, but in addition, the process to be performed by the device body 10 is included. Also, instructing the secure element 20 to execute all but not a part of the inference process and performing the inference process by only the two of the secure element 20 and the server 3 can also be shown as an application example of the present embodiment. .

  FIG. 17 is a flowchart showing the procedure of inference processing according to the applied embodiment. FIG. 17 illustrates a flowchart that substitutes for FIG. In this case, the control unit 11 requests the secure element 20 to process the inference model without performing the operation (step S32, refer to FIG. 15) of the former stage element in the first and second embodiments in the device body 10 (step S33). Specifically, the secure element 20 designates input data received by the control unit 11 as the secure element while designating a former element (for example, information indicating the first layer of the neural network) as the information for specifying the computing element to be computed by the secure element 20. Output to 20.

  As described above, which computing element the device body 10 and the secure element 20 calculate is an arbitrary design matter. Therefore, it can be easily understood that the implementation in which only the secure element 20 and the server 3 perform the calculation in cooperation with each other is intentionally included in the scope of the present embodiment.

Third Embodiment
The first embodiment has described a mode in which part of arithmetic processing of the inference model 121 is executed in the secure element 20 mounted on the device 1. In the present embodiment, an embodiment will be described in which arithmetic processing is performed in a trusted execution environment (TEE) that is virtually built in the device 1.
FIG. 18 is a block diagram showing a configuration example of the IoT system according to the third embodiment. The device 1 according to the present embodiment uses, for example, a technology called TrustZone (registered trademark) to execute an execution environment of software (OS, application, etc.) as a normal execution environment (REE; Rich Execution Environment) 51 , And the trusted execution environment 52.

  The normal execution environment 51 is an execution environment of a general-purpose OS 511 that is widely and generally used, and is an execution environment that has no special functional restrictions other than the access to the trusted execution environment 52 being restricted. The general-purpose OS 511 is software that performs the function of the OS in the normal execution environment 51, and provides various OS functions including control of hardware connected to the device 1 in response to a request from the application 512. The control unit 11 executes basic and general processing of the device 1 by executing the application 512 on the general-purpose OS 511.

  The trusted execution environment 52 is an independent execution environment provided separately from the normal execution environment on the same SoC in order to isolate security functions. In the trusted execution environment 52, access from the normal execution environment 51 is limited, and executable functions are also limited. Note that the trusted execution environment is not limited to acronym like TEE, and it is separated from the normal execution environment 51, and any execution environment that is more secure in terms of security may be used. Good. The device 1 secures security by placing software and data to be secured in the trusted execution environment 52 and restricting access from outside of the normal execution environment 51 and the device 1.

  As described above, the normal execution environment 51 is restricted from accessing the trusted execution environment 52, and the normal execution environment 51 can not recognize the existence of the trusted execution environment 52. In order to call a process to be executed in the trusted execution environment 52 from the normal execution environment 51, it is necessary to go through a secure monitor 53 implemented on software.

  The trusted OS 521 is software that performs the function of the OS in the trusted execution environment 52, and provides the OS function centering on the security function in response to a request from the application 522. The control unit 11 executes the application 522 on the trusted OS 521 to execute security-critical processing including arithmetic processing based on the inference model 121.

  In the present embodiment, it is not an essential matter whether the various functions of the device 1 are implemented by the OS or the application, and it is a design matter that the implementer should select as appropriate. Description of sharing will be omitted.

  As shown in FIG. 18, in the present embodiment, the control unit 11 of the device 1 arranges the inference model parameter 271, the permission flag 272, the unique ID 273 and the like in the trusted execution environment 52. The control unit 11 reads out the inference model parameters 271 arranged in the trusted execution environment 52, and develops the inference model 121 in the normal execution environment 51.

  When executing the inference process, the control unit 11 performs a calculation process on part of the inference model 121 in the normal execution environment 51 and then passes the operation result to the trusted execution environment 52. The control unit 11 executes counting of the number of times of inference execution, checking of the permission flag 272, and partial arithmetic processing of the inference model 121 in the trusted execution environment 52, and returns the operation result to the normal execution environment 51. The control unit 11 outputs an inference result using the returned operation result.

  As described above, according to the third embodiment, the device 1 constructs the normal execution environment 51 and the trusted execution environment 52 that is more secure than the normal execution environment 51, and part of the inference model 121 in the trusted execution environment 52. Execute the arithmetic processing of As described above, even if the secure element 20 is not mounted, the security of the inference process can be secured by the configuration on software.

  As described above, the device 1 has a component (secure unit) that is more secure than a component that executes an operation related to inference processing, and can execute (or execute) operation processing of part of the inference model 121 in the secure component. As in mode 2, the partial calculation of the inference model 121 can be requested to the server 3 and the calculation result output from the server 3 can be returned to the normal execution environment 51). The secure component may be a hardware-isolated secure element 20 or the like, or a software-isolated trusted execution environment 52 or the like.

  The second embodiment is the same as the first embodiment except that the trusted execution environment 52 is implemented instead of the secure element 20, and thus the detailed illustration and the description will be omitted in the present embodiment.

Embodiment 4
In the present embodiment, an aspect will be described in which the inference model 121 stored in the device 1 is updated.
FIG. 19 is an explanatory diagram of the updating process of the inference model 121. FIG. 19 conceptually illustrates that the secure element 20 communicates with the server 3 and updates the inference model parameter 271.

  The secure element 20 according to the present embodiment includes the updating unit 28. The updating unit 28 acquires the latest model parameters 341 from the server 3 and updates the parameters stored in the storage unit 27.

  The processing processes indicated by reference numerals P51 to P58 in FIG. 19 will be described in order. First, the device body 10 requests the secure element 20 to confirm to the server 3 whether the inference model 121 has been updated to the latest data (P51). When the confirmation request is received, the updating unit 28 of the secure element 20 requests the communication channel opening unit 25 to open the secret communication channel 41 (P52). When the request is received, the communication path establishment unit 25 establishes the secret communication path 41 (P53).

  The updating unit 28 requests the server 3 to update the inference model parameter 271 via the secret communication path 41 (P54). Specifically, the updating unit 28 transmits version information 271 c of the current inference model parameter 271 held in the secure element 20 and the unique ID 273 of the own device.

  When receiving the update request from the secure element 20 (device 1), the server 3 determines whether the inference model 121 has been updated in the device 1 as the request source (P55). Specifically, the server 3 collates the version information 271 c transmitted from the secure element 20 with the version information 341 c (see FIG. 19) of the latest model parameter 341. If the version information matches the latest one, the inference model 121 of the device 1 has been updated to the latest model, and the server 3 does not perform any particular processing, and ends the series of processing. On the other hand, if the version information does not match the latest version, the server 3 transmits the latest model parameter 341 to the device 1 via the secret communication path 41 (P56). In addition, when transmitting the latest model parameter 341, the server 3 stores update information indicating that the inference model 121 of the device 1 has been updated in the operation management DB 342 in association with the unique ID 273 received together with the version information 271c. (P57). For example, the server 3 stores the current date and time in the operation management DB 342 as the final update date and time. The server 3 may store information that has been updated on the database, and may store information other than date and time as update information, such as storing version information 341c of the latest model, for example. .

  The update unit 28 of the secure element 20 stores the latest model parameter 341 transmitted from the server 3 in the storage unit 27 and updates the parameter (P58). For example, the updating unit 28 outputs the parameter to the device body 10 immediately after downloading the parameter, and the device body 10 updates (reconstructs) the inference model 121 based on the parameter. Note that the updating may be performed at any timing such as the next activation of the device 1 or the next execution of the application without updating the inference model 121 immediately after downloading the parameter. As described above, the data stored in the IoT device can be updated to the latest one, and it is possible to meet the update needs, such as preventing obsolescence.

  Although a series of processes are started according to a request from the device main body 10 in the above, the present embodiment is not limited to this. For example, update is performed when the remaining number of inference processes becomes 0, or the server 3 detects the device 1 operating in the old version from the operation management DB 342, and the parameter update request from the server 3 to the device 1 The update process of the inference model 121 may be started at any timing such as

FIG. 20 is a flowchart illustrating an example of the processing procedure of the process of updating the inference model 121. The contents of processing performed by the IoT system according to the present embodiment will be described based on FIG.
The control unit 11 of the device body 10 requests the secure element 20 to confirm with the server 3 whether it is the latest model of the inference model 121 (step S401). When the request is received, the secure element 20 establishes the secret communication path 41 with the server 3 (step S402). The secure element 20 issues an update request for the inference model parameter 271 via the secret communication path 41 (step S403). For example, as described above, the secure element 20 transmits the version information 271 c of the parameter stored in the storage unit 27 and the unique ID 273 of the own device to the server 3.

  If the update request has been received, the control unit 31 of the server 3 determines whether the inference model parameter 271 has been updated in the device 1 that has made the update request (step S404). For example, the control unit 31 determines whether the version information 341c of the latest model matches the version information 271c transmitted from the secure element 20. If it is determined that the update has been completed (S404: YES), the control unit 31 ends the series of processing.

  When it is determined that the update is not completed (S404: NO), the control unit 31 transmits the latest model parameter 341 to the secure element 20 via the secret communication path 41 (step S405). When the latest model parameter 341 is transmitted, the control unit 31 stores update information indicating that the inference model 121 of the device 1 has been updated in the operation management DB 342 (step S406), and ends the series of processing.

  When the latest model parameter 341 is received from the server 3, the secure element 20 stores the parameter in the storage unit 27 (step S407). The secure element 20 outputs the newly stored latest model parameter 341 to the device body 10 (step S408). The secure element 20 ends the series of processes.

  When the latest model parameter 341 is output from the secure element 20, the control unit 11 of the device body 10 deploys the latest inference model 121 on the memory based on the parameter (step S409). The control unit 11 ends the series of processes.

  Although the version information 271c and 341c are illustrated and described as being the version number, that is, the number indicating the version in the above description, the present embodiment is not limited to this. For example, the version information 271 c and 341 c may be information representing a time such as a date when the parameter was updated. Also, for example, the server 3 may compare the hash value of the inference model 121 developed in the device 1 with the latest inference model 121 to determine whether or not the latest version. As described above, the server 3 only needs to update the inference model parameter 271 based on the information that can determine the new and old of the inference model 121, and the version information 271c and 341c are not limited to numbers.

  As described above, according to the fourth embodiment, by distributing the latest model parameter 341 from the server 3 to the device 1, the latest inference model 121 can be developed in the device 1, and obsolescence of the model, etc. It can meet the need for updating. Particularly, in the present embodiment, as in the first embodiment, the parameter is temporarily stored (stored) in the secure element 20, and the device body 10 reads the parameter from the secure element 20 and uses the inference model on the memory (storage unit 12). Expand 121. As a result, the possibility of exploiting parameters that can mimic the inference model 121 can be reduced, and security can be ensured.

Fifth Embodiment
In the present embodiment, a mode will be described in which the server 3 detects an abnormality based on the execution frequency of the inference process in the device 1 and prohibits the inference process.
FIG. 21 is an explanatory diagram of inference prohibition processing based on the execution frequency of the inference processing. An outline of the present embodiment will be described based on FIG.

  As described in the first embodiment, the server 3 acquires the count value of the inference execution number from the secure element 20, and manages the execution number on the operation management DB 342. In particular, since the server 3 additionally acquires the inference execution date and time from the secure element 20 and stores it in the operation management DB 342, the time series change in the number of executions in each device 1, that is, the frequency of inference processing is unified can do.

  In the present embodiment, the server 3 detects fraud analysis from the outside of the device 1 based on the frequency of inference processing. As described with reference to FIG. 6, when attempting to distill the inference model 121, it is necessary to cause the device 1 to perform inference processing and acquire (exploit) a large amount of inference result data. Such fraudulent activity takes place in a relatively short time, and the number of inference executions increases dramatically while the fraudulent activity is being conducted. In other words, the frequency of inference processing is characterized by fraud. The server 3 detects an unauthorized analysis from the outside by determining an increase in the frequency of inference processing.

  In the example of FIG. 21, the frequency of inference processing is significantly increased in the device 1 in which the remaining number of inference processing is indicated in bold. For example, in the device 1 in which the unique ID 273 is "100002", the remaining number is exhausted in a few minutes. Further, even in the device 1 with the unique ID 273 of “100003”, more than 100 inference operations are performed in a few minutes.

  When the inference processing frequency is equal to or higher than a predetermined frequency as the check policy, the server 3 determines that the fraud analysis is being performed. The frequency serving as the criterion is, for example, 10 times / minute, but the numerical value is not particularly limited. The server 3 specifies, from the operation management DB 342, the device 1 which is executing the inference process at 10 times / minute or more.

  The server 3 instructs the identified device 1 to stop the inference process. For example, the server 3 changes the permission information on the operation management DB 342 from “permitted” to “prohibited”, and transmits the changed permission information to the device 1 via the secret communication path 41. The secure element 20 changes the permission flag 272 to "prohibit" according to the permission information transmitted from the server 3, and stops the subsequent inference processing.

  In addition, the server 3 stops the execution of the inference processing not only for the device 1 with high inference processing frequency but also for other devices 1 to which the inference model 121 of the same type as the device 1 is applied (expanded). To tell.

  As described above, the server 3 distributes the inference model parameters 271 to the plurality of devices 1 to execute inference processing based on the inference model 121. In this case, it is assumed that an attacker analyzes a plurality of devices 1 in parallel and analyzes them. Therefore, the server 3 first prohibits inference processing in the device 1 that can be an analysis target in the near future by an attacker, and prevents unauthorized analysis in advance.

  For example, when the frequency of inference processing in the two or more devices 1 (secure element 20) becomes a predetermined frequency or more as the check policy, the server 3 applies another inference model 121 of the same type as the device 1 (secure Instructs element 20) to stop inference processing. Specifically, as shown in FIG. 21, the server 3 transmits permission information to the secure element 20 of the corresponding device 1 via the secret communication path 41, and changes the permission flag 272 from "permit" to "prohibit". Change the setting.

  Although grouping is performed according to the type of the inference model 121 and inference processing is simultaneously prohibited in the above description, the grouping criterion is not limited to the type of the inference model 121. For example, the server 3 may perform grouping in accordance with the physical arrangement place of the device 1, the type of component (for example, whether it is the trusted execution environment 52 relatively prepared for analysis than the secure element 20) or the like. That is, the server 3 associates the devices 1 with each other according to a predetermined check policy on the operation management DB 342, and prohibits the inference processing of the associated plurality of devices 1 according to the frequency of inference processing in any of the devices 1 I wish I could.

FIG. 22 is a flowchart of an example of the process procedure of the inference prohibition process. The processing content of the IoT system according to the present embodiment will be described based on FIG.
For example, the server 3 executes the following processing by batch processing. The control unit 31 of the server 3 calculates the execution frequency of the inference process in each device 1 with reference to the operation management DB 342 (step S501). As described above, each device 1 notifies the server 3 of the number of times of execution of the inference process along with the execution time. The server 3 stores the number of times of execution of the inference process in each device 1 in the operation management DB 342, including the execution time of the inference process. The server 3 calculates the execution frequency of the inference process in the device 1 from the execution time at each inference time stored in the operation management DB 342.

  The control unit 31 compares the execution frequency of each device 1 with a predetermined threshold value, and determines whether there is a device 1 that is performing inference processing with an execution frequency equal to or higher than the threshold value (step S502). If it is determined that there is no device 1 whose execution frequency is equal to or higher than the threshold (S502: NO), the control unit 31 ends the series of processing.

  When it is determined that there is a device 1 whose execution frequency is equal to or higher than the threshold (S502: YES), the control unit 31 specifies the inference model 121 applied (installed) to the device 1 (step S503). Then, the control unit 31 refers to the management DB 341, and identifies another device 1 to which the inference model 121 of the same type as the identified inference model 121 is applied (step S504).

  The control unit 31 transmits an instruction to prohibit the inference process to the device 1 whose execution frequency is equal to or higher than the threshold and the other device 1 to which the inference model 121 of the same type as the device 1 is applied (step S505). The control unit 31 stores (updates) in the management DB 341 permission / disapproval information to the effect that the inference processing has been prohibited for each device 1 (step S506), and ends the series of processing.

  When the prohibition instruction is received from the server 3, the secure element 20 of the device 1 sets permission / disapproval information indicating that the inference process is prohibited according to the instruction (step S 507). For example, the secure element 20 changes the setting of the permission flag 272 from “permitted” to “prohibited”. The secure element 20 ends the series of processes.

  As described above, according to the fifth embodiment, by prohibiting the inference process in the device 1 according to the execution frequency of the inference process (calculation process), it is possible to detect fraud analysis and take appropriate measures. .

  Further, according to the fifth embodiment, not only the device 1 with a high frequency of inference execution but also the inference processing of other devices 1 associated with the device 1 are prohibited together, whereby fraud analysis of the inference model 121 is performed. Can be more effectively prevented.

Sixth Embodiment
FIG. 23 is a functional block diagram showing the operation of the device 1 in the form described above. When the control unit 11 executes the program P1, the device 1 operates as follows. The storage unit 221 is a learned model generated by machine learning, and stores the learned model having a plurality of operation elements. The execution unit 222 executes arithmetic processing based on the learned model. The secure unit 223 is a component that is more secure than the execution unit 222, and executes an operation process related to the operation element of a part of the learned model according to the operation request from the execution unit 222. The execution unit 222 outputs a final calculation result using the calculation result by the secure unit 223.

  The sixth embodiment is as described above, and the other parts are the same as the first to fifth embodiments, so the corresponding parts are denoted with the same reference numerals and the detailed description thereof will be omitted.

  It should be understood that the embodiments disclosed herein are illustrative in all respects and not restrictive. The scope of the present invention is indicated not by the meaning described above but by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

1 device 10 device main body 11 control unit 12 storage unit 121 inference model 20 secure element (secure unit)
23 operation unit 27 storage unit 271 parameters for inference model 272 permission flag (permission or rejection information)
273 Unique ID
51 Normal execution environment 52 Trusted execution environment (secure part)
3 Server (management device)
31 control unit 34 auxiliary storage unit 341 parameter for latest model 342 operation management DB

Claims (27)

A storage unit storing a learned model having a plurality of operation elements, which is a learned model generated by machine learning;
An execution unit that executes arithmetic processing based on the learned model;
A component that is more secure than the execution unit, and executes an operation process related to the operation element of a part of the learned model according to the operation request from the execution unit;
A device characterized in that the execution unit outputs a final calculation result using the calculation result by the secure unit. The execution unit is
Performing an operation relating to the operation element of a part of the learned model,
Outputting, to the secure unit, an operation result related to the part of the operation elements, and information for designating the operation element different from the part of the operation elements as an operation target;
The secure unit
Performing an operation relating to the specified operation element using the operation result output from the execution unit;
The device according to claim 1, wherein an operation result of the specified operation element is returned to the execution unit. The secure unit
Performing a rounding process on the operation result represented by the probability, and converting the operation result to be returned to the execution unit;
The device according to claim 2, wherein the converted operation result is returned to the execution unit. It has an acquisition unit that acquires authentication information required for operator authentication,
The device according to any one of claims 1 to 3, wherein the secure unit executes the arithmetic processing when the operator authentication based on the authentication information succeeds. The secure part is a secure element having tamper resistance, or a trusted execution environment in which access from the execution environment in which the execution part performs processing is restricted. Device described in. The secure unit
Requesting execution of part or all of the arithmetic processing requested from the execution unit from a management apparatus capable of communicating with a device;
Obtain the calculation result from the management device,
The result of the arithmetic processing executed by the secure unit using the operation result acquired from the management apparatus or the acquired operation result is returned to the execution unit. Or a device according to clause 1. The secure unit
When an operation request from the execution unit is received, the number of executions of the operation process is counted,
The device according to any one of claims 1 to 5, wherein when the number of executions reaches a predetermined number, the arithmetic processing is limited. The device according to claim 7, wherein the secure unit notifies the management apparatus capable of communicating with the device of the counted number of times of execution or information indicating that the calculation process has been performed. The device according to claim 8, wherein the secure unit notifies of the execution date and time when the arithmetic processing is performed. The secure unit
The device according to claim 8 or 9, wherein the predetermined number of times is updated according to an instruction from the management device. The secure unit
Holding permission information indicating permission or prohibition of the arithmetic processing;
The device according to any one of claims 1 to 5, wherein when the permission information indicating permission of the operation processing is held, the operation processing is executed. The device according to claim 11, wherein communication is performed with a management device that manages the operation status of the device, and the permission information is set according to an instruction from the management device. The secure unit
Obtain parameters necessary for deploying the latest learned model from a management device capable of communicating with the device,
Outputting the acquired parameter to the execution unit;
The device according to any one of claims 1 to 5, wherein the execution unit develops the latest learned model in the storage unit based on the parameter acquired from the secure unit. The communication system according to any one of claims 6, 8 to 10, 12 and 13, wherein the secure unit performs communication with the management apparatus via a secret communication path in which communication content is encrypted. Device described. A secure element mounted on the device,
A receiving unit that receives, from the device, a request for execution of arithmetic processing based on a learned model that is a learned model generated by machine learning and has a plurality of arithmetic elements;
An operation unit that executes an operation process related to the operation element of a part of the learned model according to the received execution request;
An output unit that outputs an operation result to the device. A request unit that requests a management apparatus capable of communicating with the device to execute part or all of the arithmetic processing requested from the device;
The secure element according to claim 15, further comprising: an acquisition unit configured to acquire an operation result from the management device. On devices equipped with a Secure Element,
Storing in a storage unit a learned model generated by machine learning and having a plurality of operation elements;
A program for executing processing to execute arithmetic processing based on the learned model,
Requesting the secure element to perform an arithmetic process related to the arithmetic element of a part of the learned model;
A program that outputs a final calculation result using the calculation result of the secure element. To the device
A trained model generated by machine learning, which has a plurality of operation elements, is held in a first execution environment,
A program that executes processing for executing arithmetic processing based on the learned model in the first execution environment,
In a second execution environment that is more secure than the first execution environment, an arithmetic process related to the arithmetic element of a part of the learned model is executed.
A program that outputs a final calculation result using the calculation result in the second execution environment. An information processing system comprising a device and a management device capable of communicating with the device, the information processing system comprising:
The device is
A storage unit storing a learned model having a plurality of operation elements, which is a learned model generated by machine learning;
An execution unit that executes arithmetic processing based on the learned model;
A component that is more secure than the execution unit, and executes an operation process related to the operation element of a part of the learned model according to the operation request from the execution unit;
The execution unit outputs a final calculation result using the calculation result by the secure unit,
The secure unit notifies the management apparatus of the number of executions of the arithmetic processing or information indicating that the arithmetic processing has been executed.
The information processing system, wherein the management apparatus manages the number of times of execution based on a notification from the secure unit. The secure unit
When the operation request from the execution unit is received, the number of executions is counted,
The information processing system according to claim 19, wherein the management apparatus is notified of the counted number of times of execution. The secure unit, when executing the arithmetic process, notifies the management apparatus of information indicating that the arithmetic process has been executed.
The information processing system according to claim 20, wherein the management device counts the number of times of execution based on a notification from the secure unit. The information processing system according to any one of claims 19 to 21, wherein the secure unit executes the arithmetic processing when the number of times of execution has not reached a predetermined number of times. The secure unit notifies the management apparatus of the number of times of execution or information indicating that the arithmetic processing has been performed, together with identification information of the device.
The information processing apparatus according to any one of claims 19 to 22, wherein the management apparatus stores information indicating the number of times of execution or that the arithmetic processing has been performed in a storage unit in association with the identification information. Information processing system. The secure unit notifies the management apparatus of the number of times of execution or information indicating that the arithmetic process has been performed, together with the date and time of execution of the arithmetic process.
The information processing apparatus according to any one of claims 19 to 23, wherein the management device stores information indicating the number of times of execution or that the arithmetic processing has been performed, in a storage unit together with the execution date and time. system. The management device is
A determination unit that determines whether the execution frequency of the arithmetic processing is equal to or higher than a threshold based on the number of times of execution and the date and time of execution;
The information processing system according to claim 24, further comprising: an instruction unit that instructs the secure unit to prohibit the arithmetic processing when the determination unit determines that the execution frequency is equal to or higher than a threshold. The storage unit stores a plurality of the devices in association with each other,
The information processing system according to claim 25, wherein the instruction unit instructs the plurality of devices associated with one another to prohibit the arithmetic processing. What is claimed is: 1. A device comprising: a storage unit for storing a learned model having a plurality of computing elements, the learning model being generated by machine learning, and an execution unit for executing computing processing based on the learned model.
The execution unit requests the component more secure than the execution unit to perform an arithmetic process related to the arithmetic element of a part of the learned model;
The secure component executes arithmetic processing relating to the part of the arithmetic elements,
An information processing method comprising: executing the processing of outputting a final calculation result using the calculation result of the secure component.

Images