JP7138498B2 - Data identification device, system, program and method capable of controlling access of identification requester - Google Patents

Description

The present invention relates to techniques for identifying specific objects using data, particularly image data.

Currently, there are high expectations for the realization of smart cities that provide various innovative services using IT technology and contribute to improving the living standards of residents. For example, the appropriate use of smart cameras for surveillance can prevent or suppress the occurrence of crimes in city areas and maintain safe public spaces.

FIG. 5 shows a typical conventional surveillance system using such camera images. According to FIG. 5, a plurality of surveillance cameras 9 are installed, and image (video) data captured by each camera is encrypted and uploaded to the cloud 8 at all times. Also, in the cloud 8, a trained model for image identification is built in advance.

Next, the cloud 8 receives the encrypted image to be requested for identification (target image) as a queue (request) from the requesting terminal 7, and applies the learned model to the received image data group. Generate object identification results. This object identification result can be, for example, the image of the object (or VDO frame) and the position information of the object (ie the background image) at that point in time.

Finally, the cloud 8 encrypts the generated object identification result and transmits it to the terminal 7 which is the source of the request. As a result, the cloud 8 has answered the question from the terminal 7 as to where the specific target is.

Here, in the system of FIG. 5, data exchanged between the camera 9, the cloud 8, and the terminal 7 are all encrypted, and interception of data related to the identification target by an attacker (for example, an eavesdropper) can be prevented. It's becoming Incidentally, the encryption key for this encryption will be managed by the cloud 8 .

Protection of privacy for such identification objects is a very important issue in image monitoring systems. For example, Non-Patent Document 1 discloses a technique of adopting secure visual object coding in an image monitoring system to protect the privacy of cues and objects. In this technique, a person (human object) in a video frame is encoded such that an unauthorized user can see the object but cannot see details such as the color of the object.

In addition, for example, Non-Patent Document 2 discloses that a feature amount is extracted from face image data of a person using Eigenface, which is one of the feature extraction methods, and encryption processing is performed on this feature amount using a homomorphic encoding method. to protect privacy.

Incidentally, in the systems of Non-Patent Documents 1 and 2 described above, image identification processing is performed by one server. In this respect, for example, in the technology disclosed in Non-Patent Document 3, processing is performed to separate the foreground, which is the image portion corresponding to the object, from the obscure portion of the object caused by the subject (blur motion blur). Such image data is divided into a plurality of files and stored in a plurality of different servers.

K. Martin and K.N. Plataniotis, "Privacy Protected Surveillance Using Secure Visual Object Coding" IEEE Transactions on Circuits and Systems for Video Technology: Special Issue on Video Surveillance, Vol. 18, No. 8, 2008, pp.1152-1162 Z. Erkin, Martin. Franz, and J. Guajardo, "Privacy-preserving face recognition" In: Goldberg I., Atallah M.J. (eds) Privacy Enhancing Technologies (PETS) 2009, Lecture Notes in Computer Science book series, Vol 5672. Springer, Berlin, Heidelberg, 2009 L. Du and Y. Li, "Privacy preserving for human objects in video surveillance via visual cryptography", Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), 2014, pp.80-85

However, the actual situation is that even the conventional technology described above still has a problem in ensuring data security that sufficiently protects the privacy of an identification target.

For example, in the image monitoring system shown in FIG. 5, if the cloud 8 allows a virus or the like to intrude, the privacy of queues and targets cannot be protected after all. That is, an intruder can also obtain information about who the target is and where it is located.

Also, in the techniques disclosed in Non-Patent Documents 1 and 2, image identification processing is performed within one server after all. In other words, the identification result for the identification target is always generated in the server, and the attack target can be narrowed down to the server, making it easier for the attacker to attempt interception.

On the other hand, in the technique described in Non-Patent Document 3, image data is indeed divided into a plurality of files and processed by a plurality of different servers. However, in the first place, this technology cannot specify who the target is, and ultimately cannot answer the question of the identification requester.

Therefore, the present invention provides a data identification system capable of separating data identification processing from identification model construction processing to improve data security, a data identification device, a data identification program, and data identification capable of realizing the system. The purpose is to provide an identification method.

According to the present invention, a data acquisition unit that generates or acquires data to be subjected to identification processing;
An identification model for the identification process received as a response to a request from an external server , and constructed by this server using data related to the identification object supplied from the requester of the identification process as learning data data identification means for performing the identification process , which is a process related to identification of the identification target, on data to be the identification process using a model;
an access control means for transmitting information related to the identification result to a request source of the identification process when the identification result obtained by the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
The data identification means performs the identification process on the data to be subjected to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. A data identification device is provided.

In the data identification device according to the present invention, the server is a node in a communication network using ICN (Information-Centric Networking) technology, and the request source sends the signed data related to the identification object to this sending to a server and receiving from the server the name or identifier of the identification model constructed by the data relating to the identification object;
The data identification device transmits a request packet designating the name or identifier of the identification model notified from the request source, and receives the identification model constructed by the server and with the signature. further having
The data identification means may perform the identification process using the identification model determined to be suitable for use by matching the signature attached to the identification model with the notified name or identifier of the identification model. preferable.

According to the present invention, a data acquisition unit that generates or acquires data to be subjected to identification processing;
An identification model for the identification process constructed by an external server as a node in a communication network using ICN technology , and learns data related to the identification object supplied from the requester of the identification process. a device communication interface that receives the identification model constructed as data for use as a response to the transmitted request packet ;
data identification means for performing the identification process , which is a process related to identification of the identification target, on the data to be the identification process using the received identification model;
an access control means for transmitting information related to the identification result to a request source of the identification process when the identification result obtained by the identification process satisfies a predetermined condition ;
The device communication interface transmits a request packet specifying the name or identifier of the identification model received by the requester from the server and specifying the name or identifier of the identification model notified from the requester.
There is provided a data identification device characterized by:
Further, in the data identification apparatus according to the present invention, the server receives the data related to the identification object to which the signature is attached from the request source, and uses the data related to the identification object as learning data to obtain the signed data. building the discriminative model that is
a device communication interface receiving the identity model constructed and signed by the server in response to the outgoing request packet;
The data identification means may perform the identification process using the identification model determined to be suitable for use by matching the signature attached to the identification model with the notified name or identifier of the identification model. preferable.
According to the present invention, furthermore, data comprising at least one requesting device from which the request is made, at least one server as described above for building the identification model, and at least one data identification device as described above. An identification system is provided.

Further, as another embodiment of the data identification system according to the present invention, the requesting device is a requesting source for transmitting the name or identifier of the identification model together with the public key to the data identification device as the requesting source for the identification processing. Having a communication interface, the device communication interface may transmit the received information related to the identification result encrypted with the public key to the requesting device that requested the identification processing. preferable.

Further, as a specific example of the data identification system according to the present invention, the data to be subjected to the identification process and the data related to the identification object are image data, and data acquisition of the data identification device is performed. The unit includes a camera that generates the image data, and the access control means of the data identification device, when obtaining the identification result that the image data includes the identification target, identifies the partial image data including the identification target. , is preferably transmitted to the requesting device that requested the identification processing.

Further, in a specific example of handling this image data, the data identification device further has background separation means for separating background data from the image data,
The data identification means of the data identification device performs the identification process on the image data separated for the background data,
The device communication interface transmits the image partial data including the identification target and the background data to the requesting device that requested the identification processing,
Preferably, the requesting device further comprises image synthesizing means for combining the received image partial data including the identification target and the background data to generate image data including the identification target.

According to the present invention, there is also provided a program that causes a computer installed in a data identification device having a data acquisition unit that generates or acquires data to be subjected to identification processing to function,
An identification model for the identification process received as a response to a request from an external server , and constructed by this server using data related to the identification object supplied from the requester of the identification process as learning data data identification means for performing the identification process , which is a process related to identification of the identification target, on data to be the identification process using a model;
causing a computer to function as access control means for transmitting information related to the identification result to a requester of the identification process when the identification result obtained by the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
The data identification means performs the identification process on the data to be subjected to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. A data identification program is provided.

According to the present invention, there is further provided a data identification method in a computer installed in a data identification device having a data acquisition unit that generates or acquires data to be subjected to identification processing,
An identification model for the identification process received as a response to a request from an external server , and constructed by this server using data related to the identification object supplied from the requester of the identification process as learning data a first step of performing the identification process , which is a process relating to the identification of the identification target, on the data to be the identification process using the model;
a second step of transmitting information related to the identification result to a request source of the identification process when the identification result of the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
In the first step, the identification process is performed on the data to be subjected to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. A data identification method is provided.

According to the present invention, data security can be improved by separating data identification processing from identification model construction processing.

1 is a schematic diagram illustrating one embodiment of a data identification system according to the present invention; FIG. 1 is a functional block diagram showing the functional configuration of a request source device and a node according to an embodiment of the present invention; FIG. 1 is a functional block diagram showing a functional configuration in one embodiment of a data identification device according to the present invention; FIG. 1 is a sequence diagram schematically illustrating an embodiment of a data identification method according to the present invention; FIG. 1 is a schematic diagram showing a conventional example of a surveillance system using camera images; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Data identification system]
FIG. 1 is a schematic diagram showing one embodiment of a data identification system according to the present invention.

The data identification system 5 of this embodiment shown in FIG.
(a) one or more cameras 1, which are data identification devices capable of photographing an object to be identified and tracked, and capable of identifying and tracking the object using data of the captured image (video);
(b) a terminal 3 as an identification/tracking processing request source device that requests camera 1 to perform identification/tracking processing of the object and obtains an identification result;
(c) one or more model construction servers 2, which are nodes of a communication network, for constructing identification models used in identification/tracking processing performed by the camera 1 using learning data obtained from the terminal 3; and

Here, the objects to be tracked include various objects such as people, animals, vehicles such as automobiles, and other movable physical objects, as long as they can be photographed. Also, the place where the image is taken is not particularly limited. For example, it may be outdoors, such as a large geographic area where the general public, spectators, commuters, shoppers, workers, pedestrians, runners, etc. may appear to be tracked, as well as businesses, schools, stores, and shopping. It can also be indoors, such as inside a mall.

Similarly, according to FIG. 1, in the data identification system 5 of this embodiment, specifically,
(A) The terminal 3, as an “authorized identification processing requester” for the identification target, applies FHE ( Fully Homomorphic Encryption (fully homomorphic encryption) processing is performed to generate "encrypted learning data", and the "encrypted learning data" is transmitted to the model building server 2,
(B) The model construction server 2 constructs a "discriminative model" as a "function provider" using the received "encrypted learning data", and transmits the constructed "discriminative model" to Yes) to camera 1,
(C) The camera 1, as a "data owner", performs FHE processing on the data generated by itself (in FIG. 1, captured image (video) data), and the received "identification It is characterized by performing identification processing using a model.

Here, the FHE process is a process capable of obtaining an encrypted result by performing algebraic operations including addition and multiplication on the data encrypted by the process without decryption. In this data identification system 5, the model construction server 2 constructs a "discrimination model" using FHE-processed learning data. Identification processing is performed using already identified data to be identified.

This allows the confidentiality of training and testing data to be maintained at their respective processing locations, while entrusting the execution of machine learning algorithms to intermediate nodes for secure identification processing at the terminal. It becomes.

Furthermore, in this embodiment, the communication network of the data identification system 5 is constructed using ICN (Information-Centric Networking) technology. ICN is a technology that allows content to be shared within a network, and enables network functions to operate based on content names rather than servers (locations) to deliver content to requesters. This shortens the content download time.

Specifically, in the ICN architecture, each intermediate router caches a copy of the content and forwards the content to the requestor. That is, the requester can obtain a copy of the content from an intermediate router independently of the server associated with the content owner or publisher. Here, the request source transmits an interest packet, which is a request packet, and acquires a data packet, which is content data, but both are routed according to the content name.

Such content name routing and in-network caching makes it possible to perform content retrieval more efficiently and faster than in conventional IP-based networks.

In this way, the data identification system 5 in the embodiment in which the model building server 2 is an intermediate node in a communication network using ICN technology has the following features:
(D) The terminal 3 converts the "signature" for proving the integrity of the "identification model" constructed by the "encrypted learning data" and the access authority to the model to the "encrypted learning data". Together, it is further transmitted to the model construction server 2,
(E) The model construction server 2 transmits the constructed "identification model name or identifier" to the terminal 3,
(F) The terminal 3 transmits (broadcasts) the acquired "identification model name or identifier" to the cameras 1 (all in FIG. 1) as the source of the identification processing request,
(G) Each camera 1 that has received the "identification model name or identifier" designates the "identification model name or identifier" and transmits an "identification model acquisition request" (interest packet),
(H) The model building server 2 that has received the "acquisition request for the identification model" transmits the "identification model" to which the corresponding "signature" is attached to each camera 1 that is the source of the request,
(I) In each camera 1, the "signature" attached to the received "identification model" corresponds to the "name or identifier of the identification model" received from the terminal 3 that requested the identification process. is determined, and the identification process is performed using the "discrimination model" related to the determination of correspondence.

In this way, the camera 1 compares the "identification model name or identifier" obtained from the requester with the "signature" obtained from the model construction source, and the "identification model" obtained according to the requester's instruction. Since the propriety of use is determined, as a result, it becomes possible to reliably perform the identification processing desired by the requester.

In addition, the camera 1, as a data identification device according to the present invention,
(a) generating or acquiring data to be subjected to identification processing (in FIG. 1, image (video) data is generated by photographing);
(b) Using the "identification model" received from the outside (the model building server 2 in FIG. 1), performing identification processing on the generated or acquired data,
(c) When the identification result of the identification processing satisfies a predetermined condition, information related to the identification result is transmitted to the source of the identification processing request (the terminal 3 in FIG. 1).

Here, various settings are possible for the "predetermined condition" in (c) above. ) may be set.

In this way, the camera 1 uses an externally constructed "identification model" to perform data identification processing separately from identification model construction processing. Furthermore, without complicating the access management control to the camera 1 (camera group), it is possible to transmit the data identification result to be reported to the legitimate requester only to the requester. As a result, data security can be improved. That is, in this embodiment, it is possible to more securely generate identification result information related to privacy such as "who is where" requested by the terminal 3, and to more reliably notify only the terminal 3 which is the source of the request. .

In this embodiment, the camera 1 "generates" the data of (a) above by itself. It may be a device connected by a wired communication network that is closed to the In this case, the device "obtains" the data from the data generation means.

Further, in the camera 1 and the data identification system 5 according to the present invention, it is not essential to perform FHE processing on identification data and to use a communication network using ICN technology as a base. However, by adopting these techniques, it is possible to significantly improve data security as described above, and to perform more efficient data identification processing (i.e., using less communication bandwidth and lower delay). of.

Furthermore, in the data identification system 5 of this embodiment, there is one model building server 2 as an intermediate node capable of model building, but it is also possible to install a plurality of them. In this case, for example, the terminal 3 may appropriately select and use the server 2 located closer to the group of cameras 1 installed in an area where the identification target is likely to exist.

Further, although it is possible to install only one camera 1 in the system 5, it is preferable to install a plurality of cameras 1 as in the present embodiment. For example, by using a large number of cameras 1 to cover a wider range, it becomes possible to more reliably generate and identify image data including an identification target.

For example, in a monitoring service for multiple users in a smart city, a distributed processing system is configured by installing a large number of cameras 1 in the city, and appropriate sharing of monitoring cameras is possible without leaking information related to user privacy. of. Traditionally, such surveillance service platforms have operated under a central management system in the form of a police cloud. In this case, naturally, only the data owner could receive the monitoring service.

Incidentally, this data identification system 5 can satisfy the following W3 privacy model on the premise that the user (terminal 3) requesting the identification target monitoring service has authority regarding the identification target.
(a) The user (terminal 3) owns the target image. The user knows "who the target is" and wants to know "where the target is".
(b) The data owner (camera 1) does not know "who is the target", but can know whether the target is in the camera or not.
(c) The function provider (model building server 2) does not know 'who the subject is', 'where the subject is' and the 'decryption/encryption key'.

In this case, in this data identification system 5, the function provider (model construction server 2) is a third party, and the encrypted data is encrypted without exchanging the encryption key between the user/third party and between the data owner/third party. It becomes possible to outsource the construction of data-based identification models to third parties.

That is, in this data identification system 5, the training process is performed by a third party separate from the testing process, and the identification model is built on the FHE-processed encrypted data, so that the user/ No need to exchange encryption keys between third parties. Furthermore, by signing the identity model, the testing process is performed in a secure manner at the data owner who obtained the identity model. Also, the data owner does not need to send the generated/acquired data or the identification result to the third party. This ensures that the third party never knows "who it is," and that data privacy is consistently protected as a result.

Image (video) data is the object of processing in the data identification system 5 described above, and the camera 1, which is a data identification device, generates or acquires image (video) data. However, the data generated/obtained by the data identification device according to the present invention and handled by the system is not limited to image (video) data. For example, the data identification device may include a microphone, and the entire system may be configured to process audio data capable of identifying an audio generator (including a person, equipment, and the like) to be identified. Furthermore, it is possible to have an embodiment in which image data, audio data, and other sensing data are processed in parallel and used for identification processing.

[Request source device (terminal 3)]
FIG. 2 is a functional block diagram showing the functional configuration in one embodiment of the request source device and node according to the present invention.

As shown in the upper functional block diagram of FIG. 2, the terminal 3 of this embodiment includes a terminal communication interface (IF) 301 as a request source communication interface, a target image storage unit 302, and a touch panel display (TP/DP ) 303 and processor memory.

Here, this processor memory stores an embodiment of the data identification requester program according to the present invention, and has a computer function, and by executing this data identification requester program, Performs information processing as a data identification requester. For this reason, the terminal 3 may be a device or unit dedicated to the data identification requester, but may be, for example, a smart phone, a tablet or notebook computer, or a personal computer loaded with the data identification requester program according to the present invention. (PC) or the like is also possible.

Furthermore, this processor memory includes a target image management unit 311, a feature amount extraction unit 312, a labeling unit 313, an FHE (Fully Homomorphic Encryption) processing unit 314, a signature management unit 315, a model verification unit 321, It has a public key management unit 322 , a decryption processing unit 331 , an image synthesis unit 332 , a communication control unit 341 and an input/output control unit 342 . It should be noted that these functional components can be regarded as the functions of the data identification request source program stored in the processor memory. Also, the flow of processing in which the functional components of the terminal 3 are connected by arrows in FIG.

Similarly, in FIG. 2, the target image management unit 311 stores image data (target image data ) and image data that are not included are stored in the target image storage unit 302 or taken out from the target image storage unit 302 and managed appropriately.

The feature amount extraction unit 312 extracts feature amounts from the target image data input from the target image management unit 311 . Specifically, the target image data is converted into a feature quantity (feature vector) by a feature quantity extraction method such as HOG (Histograms of Oriented Gradients) or Eigenface. This feature amount is an amount that characterizes the target image data.

The labeling unit 313 labels the image data input from the target image management unit 311 . For example, a label "1" may be given to target image data, and a label "0" may be given to image data not including an identification target. Incidentally, as a specific labeling method, it is also preferable for the labeling operator to determine and input label values while viewing image data sequentially displayed on the touch panel display 303, for example.

The FHE processing unit 314 takes in the extracted feature amount and the corresponding label for each target image data, and applies FHE (fully homomorphic encryption) processing to these data to generate encrypted learning data. do.

The signature management unit 315 manages a signature for certifying the integrity of the identification model constructed by the encrypted learning data generated by the FHE processing unit 314 and the access authority to the model. This signature may be obtained, for example, by sending the target image data and the public encryption key to a TPA (Trusted Party Agency, a third-party organization) and obtaining the certificate from the TPA.

Similarly in FIG. 2, the communication control unit 341 instructs the terminal communication interface 301 to
(a) encryption learning data generated by the FHE processing unit 314;
(b) send the corresponding signature to the model building server 2;

The communication control unit 341 also outputs the model name or model identifier (ID) of the constructed identification model received from the model construction server 2 via the terminal communication interface 301 to the model verification unit 321 . The model verification unit 321 verifies/confirms that the model construction server 2 has constructed a model to be identified based on the input model name or ID, and returns the verification/confirmation result to the communication control unit 341 .

Next, the communication control unit 341 instructs the terminal communication interface 301 to
(a) the model name or ID;
(b) Transmit (broadcast) the public key managed by the public key management unit 322 to all the cameras 1;

Furthermore, the communication control unit 341 decrypts the encrypted identification result encrypted with the public key, which is received from the camera 1 (that has decided to report the identification result to the requester) via the terminal communication interface 301. Output to unit 331 . Next, the decryption processing unit 331 performs decryption processing on the input encrypted identification result using the corresponding secret key held in advance, and outputs the decrypted identification result to the image combining unit 332 .

The image synthesizing unit 332 uses the decoded identification result and the background image data (background data) transmitted together from the camera 1 to generate image data including the identification target. Specifically, the decoded identification result is foreground data, that is, the image portion to be identified (image portion data including the identification target), and the image synthesizing unit 332 combines the foreground data and the background data. are combined to generate image data including the identification target.

Further, the image synthesizing unit 332 may acquire information about the identification target based on the generated image data and the input information. For example, it is also preferable to generate the location information of the identification target in real space from the shooting range information (camera installation position information) of the camera 1 that is the source of the image data and the position of the identification target in the image data. .

Further, the image data including the identification target generated here and the information regarding the generated identification target are output to the touch panel display 303 via the input/output control unit 342, for example, and displayed, for example, in response to the user's display request. may be Also, it is preferable that the information is incorporated into an application program installed in the terminal 3 and used for various purposes. For example, it is possible to graphically display the moving route and staying position of the identification target in the real space based on the current position of the identification target.

[Node (model building server 2)]

Next, the functional configuration of the node according to the present invention will be explained. As shown in the lower functional block diagram of FIG. 2, the model construction server 2 of this embodiment is a node having a node communication interface 201, a construction model storage unit 202, and a processor/memory.

Here, this processor memory stores an embodiment of the model construction program according to the present invention, and has a computer function, and by executing this model construction program, it can be used as a model construction node. information processing. For this reason, the model building server 2 may be a device or unit dedicated to model building, but it may also be a personal computer (PC), a notebook computer, etc., on which the model building program according to the present invention is installed. It is possible.

Further, this processor memory has a signature adding section 211 , a model construction section 212 , a construction model management section 213 and a communication control section 221 . It should be noted that these functional components can be regarded as the functions of the model construction program stored in the processor memory. Also, the flow of processing in which the functional components of the model construction server 2 are connected by arrows in FIG. 2 can be understood as an embodiment of the model construction method in the model construction server 2.

Similarly, as shown in FIG. 2, the communication control unit 221 outputs the encrypted learning data and the corresponding signature received from the terminal 3 via the node communication interface 201 to the model construction unit 212 and the signature addition unit 211, respectively. do.

The model building unit 212 uses the input encrypted learning data to build a dedicated identification model for identifying the identification target specified by the request source (terminal 3). As a machine learning algorithm for constructing this discrimination model, for example, CNN (Convolutional Neural Network) can be used.

Here, since the learning data used for model construction has been subjected to FHE (fully homomorphic encryption) processing, it can be used for model construction as it is encrypted without being decrypted. Incidentally, in the testing process, the identification model constructed in this way outputs identification result data that has undergone FHE processing (as it is encrypted). Further, a signature (by the terminal 3) is added to the identification model constructed here by the signature adding unit 211. FIG.

The building model management unit 213 manages the signed building model while storing it in the building model storage unit 202 . In addition, after determining the model name or ID of the construction model and linking it to the corresponding construction model, the model name or ID is output to the communication control unit 341 so as to be notified to the requesting terminal 3 .

The communication control unit 221 causes the node communication interface 201 to transmit the model name or ID of the constructed identification model to the terminal 3 .

Also, the communication control unit 221 receives an interest packet from the camera 1 via the node communication interface 201 . This interest packet is a request packet in a communication network using ICN technology, and requests content (identification model in this embodiment) by designating a content name or its ID ("model name or ID" in this embodiment). It is a packet to Here, the communication control unit 221 obtains the identification model corresponding to the “model name or ID” from the construction model management unit 213, and transmits the signature-added identification model to the camera 1 that requested the identification model. Let

[Data identification device (camera 1)]
FIG. 3 is a functional block diagram showing the functional configuration in one embodiment of the data identification device according to the present invention.

According to FIG. 3, the camera 1 of this embodiment serves as a data identification device having a device communication interface 101, an image data storage unit 102, a data acquisition unit 111 including a camera unit 111a, and a processor memory. there is Here, the processor memory stores an embodiment of the data identification program according to the present invention, and has a computer function to execute the data identification process by executing the data identification program. do.

Furthermore, this processor memory includes a background subtraction processing unit 112, a feature amount extraction unit 113, an FHE processing unit 114, a data identification unit 115, a decryption processing unit 116, an access control unit 117, and a public key encryption unit. 118, and a communication control unit 121 including an interest requesting unit 121a and a signature verification unit 121b. It should be noted that these functional components can be regarded as the functions of the data identification program stored in the processor memory. Also, the flow of processing in which the functional components of the camera 1 are connected by arrows in FIG. 3 can be understood as an embodiment of the data identification method in the camera 1.

Similarly, as shown in FIG. 3, the communication control unit 121 receives the terminal 3, which is the source of the identification processing request, via the device communication interface 101,
(a) "model name or ID" of the identification model to be used for identification processing;
(b) receive a public key to be used for return encryption; Next, the interest request section 121a of the communication control section 121 causes an interest packet to be transmitted.

This interest packet is a request packet in a communication network using ICN technology, and in this embodiment, it is a packet that requests an identification model as content by designating the "model name or ID" of (a) above. .

Furthermore, the communication control unit 121 receives the identification model with the “signature” corresponding to the specified “model name or ID” from the model construction server 2 via the device communication interface 101 . At this time, the signature verification unit 121b of the communication control unit 121 verifies the “model name or ID” in (a) above with the “signature” obtained from the model building source, and determines whether the obtained identification model is appropriate for use. to decide.

Specifically, if signature information is included in the "model name or ID" in (a) above, and the signature verification unit 121b matches the signature information and the acquired signature, the acquired identification model may be allowed to use. Here, the identification model permitted to be used is output to the data identification unit 115 and used for identification processing.

A data acquisition unit 111 including a camera unit 111a generates and acquires image (video) data to be subjected to identification processing, and appropriately stores and manages it in the image data storage unit 102 . Here, the camera unit 111a is, for example, a photographing device compatible with visible light, near-infrared rays, or infrared rays, which includes a solid-state imaging device such as a CCD image sensor or a CMOS image sensor. A depth camera can also be used as the camera unit 111a.

Furthermore, it is also preferable that the camera unit 111a or the camera 1 as a whole is movable, and that the installation position, shooting direction, and height can be changed. Further, in this case, it may have a driving function for receiving a control signal for this change and changing the position or the like according to the control signal. Furthermore, an embodiment is also possible in which the camera unit 111a or the entire camera 1 is a part of a movable robot, drone, or the like.

The background subtraction processing unit 112 is background separation means for separating background data from the image data input from the data acquisition unit 111 . Specifically, the background subtraction processing unit 112 collects a large number of image frames in advance, and identifies a set of non-moving objects that do not exhibit movement (exceeding a predetermined stationary condition) as a background image in these image frames. , the background image portion is separated from the input image data, and the remaining image portion is used as foreground data to be subjected to identification processing. Also, background data, which is the specified background image, is output to the communication control unit 121 so as to be transmitted to the terminal 3 .

Incidentally, for example, in an embodiment in which the group of cameras is a group of surveillance cameras installed in a public area, image data is classified into two types: (a) public data and (b) private data. Here, the background data generated by the background subtraction processing unit 112 is non-sensitive data because citizens have the right to see the public environment/area, and corresponds to the public data of (a) above. Therefore, it is preferable that the background data generated by the camera 1 be transmitted to each terminal 3 and used in order to improve the utilization efficiency of the sharable data when there are a plurality of terminals 3 which are request sources.

On the other hand, the foreground data is sensitive data that enables the location of a specific identification target (for example, a specific person or a car owned by a specific individual) to be known from the position within the image frame. will be applicable. Therefore, the foreground data is provided only to the user who has the access right (in this embodiment, the requesting terminal 3). In this embodiment, as described above, the foreground data and the background data are combined in (the image synthesizing unit 332 of) the terminal 3 to which the identification result is sent, and become image data including the identification target.

Also in FIG. 3, the feature quantity extraction unit 113 extracts the feature quantity from the foreground data input from the background subtraction processing unit 112 . Specifically, it has a function corresponding to the feature quantity extraction unit 312 (FIG. 2) of the terminal 3, and converts the foreground image data into a feature quantity (feature vector) by a feature quantity extraction method such as HOG or Eigenface. This feature amount is an amount that characterizes the foreground image data.

The FHE processing unit 114 has a function corresponding to the FHE processing unit 314 (FIG. 2) of the terminal 3, and performs FHE (fully homomorphic encryption) processing on the feature amount extracted by the feature amount extraction unit 113. Generate encrypted features. Here, the FHE process is performed because the discriminative model acquired from the model building server 2 is constructed from the FHE-processed learning data as described above, so the input to the discriminative model is This is because it must be FHE processed data.

The data identification unit 115 uses the identification model acquired from the model construction server 2 to perform identification processing on the generated encrypted feature amount, and outputs the identification result to the decryption processing unit 116 . The decryption processing unit 116 decrypts the FHE of the input identification result to generate the original unencrypted identification result.

The access control unit 117 controls the access of the requester so that the information related to the identification result is transmitted to the terminal 3 that requested the identification processing only when the identification result after the decryption processing satisfies a predetermined condition. conduct. Here, as a specific predetermined condition that serves as a criterion for transmission, in the present embodiment, from a discrimination model constructed to discriminate a specific identification target, "the input feature amount is an amount corresponding to the specific target A condition is set such that an identification result stating that the For example, if the discrimination model is learned with the label "1" for image data containing the specific discrimination target and with the label "0" for image data that does not, this predetermined condition is "output The resulting label is "1".

The public key encryption unit 118 uses the public key obtained from the terminal 3 that requested the identification to encrypt the identification result that the access control unit 117 should transmit. Next, the communication control unit 121 instructs the device communication interface 101 to
(a) this encrypted identification result;
(b) Transmit the background data generated by the background subtraction processing unit 112 to the terminal 3 that is the source of the identification processing request. As a result, the identification result generated by the camera 1 can be transferred in a secure manner to the terminal 3, which is a legitimate request source to which the identification result should be notified.

[Data identification method]
FIG. 4 is a sequence diagram schematically illustrating one embodiment of the data identification method according to the invention.

(S101) The terminal 3 generates the feature amount and label of the identification target image.
(S102) The terminal 3 performs FHE processing on the generated feature amounts and labels to generate encrypted learning data.
(S103) The terminal 3 transmits the encrypted learning data and the signature to the model construction server 2.

(S104) The model construction server 2 constructs a discriminant model using the received encrypted learning data, attaches the similarly received signature to the discriminant model, and stores the model.
(S105) The model construction server 2 determines the model name of the constructed identification model and notifies the terminal 3 of it.
(S106) Based on the received model name, the terminal 3 verifies and confirms that the model for identification has been constructed by the model construction server 2 .

(S111) The terminal 3 broadcasts the model name and the public key for reply encryption to all the cameras 1 in order to request identification processing of the identification target.
(S112) Each camera 1 that has received the model name as the identification processing request transmits an interest packet (to the ICN node) designating the model name.

(S113) The model construction server 2 transmits the signature-attached identification model corresponding to the specified model name to the camera 1 that requested the model.
(S114) Each camera 1 determines whether or not the signature attached to the received identification model corresponds to the model name received from the terminal 3 requesting identification processing, It decides to use the discriminative model involved in the decision.

(S115) Each camera 1 generates image data by a photographing operation.
(S116) Each camera 1 performs a background subtraction process for separating the background from the generated image data and extracting the foreground data.
(S117) Each camera 1 generates a feature amount for the extracted foreground data.
(S118) Each camera 1 performs FHE processing on the generated feature amount.

(S119) Each camera 1 performs identification processing on the FHE-processed feature amount using the signature-verified identification model, and generates an identification result.
(S120) Each camera 1 performs decryption processing for the FHE on the generated identification result to generate an unencrypted identification result.

(S121) Each camera 1 determines whether or not the identification result satisfies a predetermined condition. ” or “0”).
(S122) The camera 1 that has determined in step S121 that the predetermined condition is satisfied (includes an identification target (label is "1")) outputs the identification result (foreground data including the identification target). is encrypted using the public key obtained from the terminal 3 .
(S123) The camera 1 transmits the encrypted identification result (foreground data including the identification target) and the background data separated in step S116 to the terminal 3 that requested the identification processing.

(S124) The terminal 3 performs decryption processing on the received encrypted identification result using the previously possessed secret key.
(S125) The terminal 3 combines the foreground data including the identification target obtained as a result of the decoding process and the background data obtained in parallel to combine the image data including the identification target, and synthesizes the image data including the identification target. , to generate information about the identification target (for example, location information of the identification target in real space).

An embodiment of a series of object identification processing in the data identification system 5 has been described above. That is, it is also possible to generate moving route information and track the identification target. In this case, it is not necessary to transmit the background data every time in step S123. For example, if the camera 1 is fixed and it is expected that the background data will not change, the background data may be transmitted only for the first time in step S123. This is because only the transmitted foreground data contains the identification object.

Incidentally, as is clear from the sequence diagram shown in FIG. 4, the identification model construction processing is executed at the node (model construction server 2), and the data used for the construction processing is It is sent after FHE processing. Therefore, the node (model building server 2) does not know the information (information about the identification target) about the data. Key generation for encrypted communication between the data owner (camera 1) and the requesting user (terminal 3) is performed by the requesting user (terminal 3). Thus, in this system, the so-called key escrow problem in which a third party can intervene to obtain a key is greatly reduced.

As described in detail above, according to the present invention, data identification processing is performed separately from identification model construction processing. Furthermore, only data identification results to be reported to the requestor are sent to the requestor without complicating access management control to the data identification device. As a result, data security can be improved.

Here, as an example of utilization of the present invention, for example, in a monitoring service for multiple users in a smart city, a distributed processing system may be configured by installing a large number of cameras as data identification devices in the city. As a result, appropriate sharing of surveillance cameras becomes possible without leaking information relating to the user's privacy.

In addition, the data generated and acquired by the data identification device according to the present invention and handled by the system is not limited to image (video) data. Audio data that can identify a generator may be processed. Furthermore, it is possible to have an embodiment in which image data, audio data, and other sensing data are processed in parallel and used for identification processing.

In the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical ideas and aspects of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 camera (data identification device)
101 device communication interface 102 image data storage unit 111 data acquisition unit 111a camera unit 112 background subtraction processing unit 113 feature amount extraction unit 114 FHE (Fully Homomorphic Encryption) processing unit 115 data identification unit 116 decryption processing unit 117 access control unit 118 public key Encryption unit 121 Communication control unit 121a Interest request unit 121b Signature verification unit 2 Model building server (node)
201 node communication interface 202 construction model storage unit 211 signature addition unit 212 model construction unit 213 construction model management unit 221 communication control unit 3 terminal (request source device)
301 terminal communication interface (request source communication interface)
302 Target image storage unit 303 Touch panel display (TP/DP)
311 target image management unit 312 feature amount extraction unit 313 labeling unit 314 FHE processing unit 315 signature management unit 321 model verification unit 322 public key management unit 331 decryption processing unit 332 image synthesis unit 341 communication control unit 342 input/output control unit 5 data Identification system 7 Terminal 8 Cloud 9 Camera

Claims (10)

a data acquisition unit that generates or acquires data to be subjected to identification processing;
An identification model for the identification process received as a response to a request from an external server , the identification constructed by the server using data related to the identification target supplied from the requester of the identification process as learning data data identification means for performing the identification process , which is a process related to identification of the identification target, on data to be the identification process using a model;
an access control means for transmitting information related to the identification result to a request source of the identification process when the identification result obtained by the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
The data identification means performs the identification process on the data subject to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. data identification device. The server is a node in a communication network using ICN (Information-Centric Networking) technology . receiving from the server the name or identifier of the identification model constructed from such data ;
The data identification device transmits a request packet designating the name or identifier of the identification model notified from the request source, and receives the identification model constructed by the server and attached with the signature. further having an interface;
The data identification means performs the identification process using the identification model determined to be suitable for use by matching the signature attached to the identification model with the notified name or identifier of the identification model. The data identification device according to claim 1, characterized by: a data acquisition unit that generates or acquires data to be subjected to identification processing;
An identification model for the identification process constructed by an external server as a node in a communication network using ICN technology , and learns data related to the identification object supplied from the requester of the identification process. a device communication interface that receives the identification model constructed as data for use as a response to the transmitted request packet ;
data identification means for performing the identification process , which is a process related to identification of the identification target, on the data to be the identification process using the received identification model;
an access control means for transmitting information related to the identification result to a request source of the identification process when the identification result obtained by the identification process satisfies a predetermined condition ;
The device communication interface transmits a request packet specifying the name or identifier of the identification model received by the requester from the server and the name or identifier of the identification model notified from the requester.
A data identification device characterized by: The server receives the signed data related to the identification object from the request source, constructs the signed identification model using the data related to the identification object as learning data,
the device communication interface receives, in response to the outgoing request packet, the identity model constructed and signed by the server;
The data identification means performs the identification process using the identification model determined to be suitable for use by matching the signature attached to the identification model with the notified name or identifier of the identification model.
4. The data identification device according to claim 3, characterized in that: at least one requesting device that is the requesting source; at least one server that constructs the identification model; and at least one data identification device according to any one of claims 1 to 4. Data identification system. The requesting device has a requesting communication interface for transmitting a public key together with the name or identifier of the identification model to the data identification device as a requester of the identification process,
3. The device communication interface transmits information related to the identification result encrypted with the received public key to the requesting device that requested the identification processing. 6. The data identification system according to 5 . The data to be subjected to the identification process and the data related to the identification target are image data,
the data acquisition unit of the data identification device includes a camera that generates the image data;
When the access control means of the data identification device obtains an identification result indicating that the image data includes the identification target, the access control means transfers the image partial data including the identification target to the requester who requested the identification processing. 7. A data identification system according to claim 5 or 6 , characterized in that the data is transmitted to the device. The data identification device further comprises background separation means for separating background data from the image data,
The data identification means of the data identification device performs the identification process on the image data from which the background data has been separated,
The device communication interface transmits the image partial data including the identification target and the background data to the requesting device that is the requesting source of the identification processing,
The requesting device further comprises image synthesizing means for combining the received image partial data including the identification target and the background data to generate image data including the identification target. 8. The data identification system according to 7 . A program that causes a computer installed in a data identification device having a data acquisition unit that generates or acquires data to be subjected to identification processing to function,
An identification model for the identification process received as a response to a request from an external server , the identification constructed by the server using data related to the identification target supplied from the requester of the identification process as learning data data identification means for performing the identification process , which is a process related to identification of the identification target, on data to be the identification process using a model;
causing a computer to function as access control means for transmitting information related to the identification result to a requester of the identification process when the identification result obtained by the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
The data identification means performs the identification process on the data subject to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. data identification program. A data identification method in a computer installed in a data identification device having a data acquisition unit that generates or acquires data to be subjected to identification processing,
An identification model for the identification process received as a response to a request from an external server , the identification constructed by the server using data related to the identification target supplied from the requester of the identification process as learning data a first step of performing the identification process , which is a process relating to the identification of the identification target, on the data to be the identification process using the model;
a second step of transmitting information related to the identification result to a request source of the identification process when the identification result of the identification process satisfies a predetermined condition;
The data related to the identification target supplied from the requester is encrypted learning data that has undergone fully homomorphic encryption processing,
In the first step, the identification process is performed on the data to be subjected to the identification process, which has been subjected to the fully homomorphic encryption process corresponding to the fully homomorphic encryption process at the request source. data identification method.

Images