US10810461B2 - Learned model generating method, learned model generating device, and learned model use device - Google Patents

Learned model generating method, learned model generating device, and learned model use device Download PDF

Info

Publication number
US10810461B2
US10810461B2 US16/097,968 US201716097968A US10810461B2 US 10810461 B2 US10810461 B2 US 10810461B2 US 201716097968 A US201716097968 A US 201716097968A US 10810461 B2 US10810461 B2 US 10810461B2
Authority
US
United States
Prior art keywords
learned model
sensing
consent
generating
acquired
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/097,968
Other languages
English (en)
Other versions
US20190095757A1 (en
Inventor
Masataka Sugiura
Yuichi Matsumoto
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Assigned to PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. reassignment PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MATSUMOTO, YUICHI, SUGIURA, MASATAKA
Publication of US20190095757A1 publication Critical patent/US20190095757A1/en
Application granted granted Critical
Publication of US10810461B2 publication Critical patent/US10810461B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G06K9/6256—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G06K9/00221—
    • G06K9/00288—
    • G06K9/6262—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Definitions

  • the present disclosure relates to a method for generating a learned model and a learned model generating device for generating a learned model by machine learning based on sensing information obtained by sensing one or more sensing objects with one or more sensing devices, and a learned model use device.
  • a technique for performing matching a sensor side metadata with an application side metadata to extract the sensor capable of providing the sensing data satisfying a request of the application is known.
  • the sensor side metadata is information on a sensor that outputs the sensing data
  • the application side metadata is information on an application that provides a service using the sensing data (see PTL 1).
  • the present disclosure is based on the problems of such a conventional technology, and the main object is to provide a method for generating a learned model and a learned model generating device capable of generating the learned model after obtaining use consent from the sensing object and a learned model use device.
  • the method of generating a learned model in the present disclosure is a method for generating a learned model by machine learning and to be performed using at least one processor based on sensing information acquired by sensing one or more sensing objects with one or more sensing devices, the method including, by the processor, acquiring, for the one or more sensing objects, information indicating use consent as to whether the sensing information of the sensing object is used for generating the learned model, creating, in the sensing information of the one or more sensing objects, learning data by processing or deleting the sensing information of the sensing object for which the use consent is not obtained, and generating the learned model by machine learning using the learning data.
  • FIG. 1 is an overall configuration diagram of a learned model providing system according to the present disclosure.
  • FIG. 2 is a block diagram illustrating a schematic configuration of a consent acquisition device.
  • FIG. 3A is a block diagram illustrating a schematic configuration of a learned model generating device.
  • FIG. 3B is a block diagram illustrating a schematic configuration of a storage of the learned model generating device.
  • FIG. 3C is a diagram illustrating an example of a compensation database.
  • FIG. 4A is a block diagram illustrating a schematic configuration of a learned model providing device.
  • FIG. 4B is a block diagram illustrating a schematic configuration of a storage of the learned model providing device.
  • FIG. 4C is a diagram illustrating an example of a learned model database.
  • FIG. 5 is a block diagram illustrating a schematic configuration of a user side device.
  • FIG. 6 is a sequence diagram illustrating an operation procedure of consent acquisition processing and anonymization processing in a learned model providing system.
  • FIG. 7 is a sequence diagram illustrating an operation procedure of learned model generation processing and electronic signature assignment processing in the learned model providing system.
  • FIG. 8 is a sequence diagram illustrating an operation procedure of learned model use request processing and signature verification processing in the learned model providing system.
  • FIG. 9 is a sequence diagram illustrating an operation procedure of learned model use report processing and compensation processing to consent in the learned model providing system.
  • FIG. 10 is an overall configuration diagram of a learned model providing system according to another embodiment.
  • FIG. 11 is a block diagram illustrating a schematic configuration of a consent acquisition device according to the other embodiment.
  • a first disclosure made to solve the above problems is a method for generating a learned model by machine learning and to be performed using at least one processor based on sensing information acquired by sensing one or more sensing objects with one or more sensing devices, the method including, by the processor, acquiring, for the one or more sensing objects, information indicating use consent as to whether the sensing information of the sensing object is used for generating the learned model, creating, in the sensing information of the one or more sensing objects, learning data by processing or deleting the sensing information of the sensing object for which the use consent is not obtained, and generating the learned model by machine learning using the learning data.
  • the learned model can be generated by machine learning using the learning data created by processing or deleting the sensing information of the sensing object from which the use consent is not obtained, it is possible to generate the learned model after obtaining the use consent from the sensing object.
  • an electronic signature is attached to the generated learned model, the electronic signature guaranteeing that processing or deleting the sensing information is performed on the sensing object from which use consent is not obtained when creating the learning data for the generated learned model.
  • the method for generating a learned model according to the second disclosure by giving the electronic signature to the generated learned model, it is possible to guarantee that the processing or deleting the sensing information is performed on the sensing object from which the use consent is not obtained when creating the learning data used to generate the learned model.
  • the sensing object is a person and the sensing information is a face image of the person, and when creating the learning data, in order to specify the face image of the person for which the use consent is not obtained, the processor acquires the face image of the person from whom the use consent is obtained.
  • the sensing object is the person and the sensing information is the face image of the person, it is possible to specify the face image of the person from whom the use consent is not obtained by using the face image of the person from whom the use consent is obtained.
  • the face image is acquired by imaging a face of the person from whom the use consent is obtained.
  • the method for generating a learned model according to the fourth disclosure it is possible to acquire the face image of the person from whom the use consent is obtained by imaging the face of the person.
  • the face image is the face image of the person who stored in a portable terminal in advance acquired from the portable terminal possessed by the person from whom the use consent is obtained through near field communication.
  • the method for generating a learned model according to the fifth disclosure it is possible to acquire the face image of the person from whom the use consent is obtained by near field communication of the face image of the person stored in the portable terminal in advance from the portable terminal possessed by the person from whom the use consent is obtained.
  • a unique identifier to the person is given in advance and the face image of the person is associated with the identifier and registered in a member database in advance, and the face image is acquired by searching the member database based on the identifier given to the person from whom the use consent is obtained.
  • the method for generating a learned model it is possible to acquire the face image of the person from whom the use consent is obtained by searching the member database based on the identifier given to the person from whom the use consent is obtained.
  • the processor when acquiring a notification that a verification result of the electronic signature given to the learned model is unverifiable from a learned model use device receiving the learned model, the processor performs invalid processing on the learned model.
  • the method for generating a learned model when acquiring the notification that the verification result of the electronic signature given to the learned model is unverifiable from the learned model use device receiving the learned model, it is possible to perform invalid processing on the learned model.
  • the processor when acquiring a notification that the learned model is used from a learned model use device receiving the learned model, gives compensation data of the sensing object from which the use consent is obtained when generating the learned model or a business operator involved in generating the learned model.
  • the method for generating a learned model when acquiring a notification that the learned model is used from a learned model use device receiving the learned model, it is possible to give the compensation data of the sensing object from which the use consent is obtained when generating the learned model or a business operator involved in generating the learned model.
  • the processor gives a unique identifier to the sensing object from which the use consent is obtained, associates the learned model generated by machine learning based on the sensing information of the sensing object to the identifier, and registers the associated the learned model to the compensation database, and when acquiring a notification that the learned model is used from a learned model use device receiving the learned model, the processor searches the compensation database based on the identifier associated with the learned model to obtain the sensing object from which the use consent is obtained when generating the learned model.
  • the method for generating a learned model when acquiring a notification that the learned model is used from a learned model use device receiving the learned model, it is possible to obtain the sensing object from which the use consent is obtained when generating the learned model by searching the sensing object from the compensation database based on the identifier given to the learned model.
  • a tenth disclosure is a learned model generating device that generates a learned model by machine learning based on sensing information obtained by sensing one or more sensing objects with one or more sensing devices, the device including a communicator that performs communication between at least one of processors and one or more sensing devices, in which the processor acquires, for the one or more sensing objects, information indicating use consent as to whether the sensing information of the sensing object is used for generating the learned model, and create, in the sensing information of the one or more sensing objects, learning data by processing or deleting the sensing information of the sensing object for which the use consent is not obtained.
  • the learned model generating device since in the sensing information of the one or more sensing objects, processing or deleting of the sensing information of the sensing object from which the use consent is not obtained can be performed, it is possible to create the learning data after obtaining the use consent from the sensing object.
  • an eleventh disclosure is a learned model generating device that generates a learned model by machine learning based on sensing information obtained by sensing one or more sensing objects with one or more sensing devices, the device including a communicator that performs communication between at least one of processors and one or more sensing devices, in which the processor, in the sensing information of the one or more sensing objects, generates the learned model by machine learning using learning data created by processing or deleting the sensing information of the sensing object from which use consent which is used for generating of the learned model is not obtained.
  • the learned model generating device since in the sensing information of the one or more sensing objects, the learned model can be generated by machine learning using the learning data created by performing processing or deleting of the sensing information of the sensing object from which the use consent is not obtained, it is possible to generate the learned model after obtaining the use consent from the sensing object.
  • an electronic signature is attached to the generated learned model, the electronic signature guaranteeing that processing or deleting the sensing information is performed on the sensing object from which use consent is not obtained when creating the learning data for the generated learned model.
  • the learned model generating device by giving the electronic signature to the generated learned model, it is possible to guarantee that the processing or deleting the sensing information is performed on the sensing object from which the use consent is not obtained when creating the learning data used to generate the learned model.
  • a thirteenth disclosure is a learned model use device that uses the learned model provided from the learned model generating device of a twelfth disclosure, the device including at least one of processors, in which the processor verifies an electronic signature given to the learned model and in a case where the verification result of the electronic signature is acceptable, configures an analysis engine using the learned model to execute analysis processing.
  • the output data is a name, a type, and an attribute of the imaged object that appears when the input data is an image, a word or a sentence that is uttered when the input data is a sound.
  • a weight value of the coupling (synaptic coupling) between nodes configuring the neural network is updated using a known algorithm (for example, in a reverse error propagation method, adjust and update the weight value so as to reduce the error from the correct at the output layer, or the like).
  • An aggregate of the weight values between the nodes on which the learning process is completed is called a “learned model”.
  • the learned model By applying the learned model to a neural network having the same configuration as the neural network used in the learning process (setting as the weight value of inter-node coupling), it is possible to output correct data with a constant precision as output data (recognition result) when inputting unknown input data, i.e., new input data not used in learning processing, into the neural network. Therefore, in a device different from a device that generates the learned model (that is, learning process), by configuring the neural network using the learned model and executing recognition processing, it is possible to perform the image recognition and voice recognition with learned recognition accuracy.
  • Learned model providing system 1 is a system that generates the learned model by machine learning based on sensing information acquired by sensing one or more sensing objects with one or more sensing devices and provides the learned model to a user via a network such as the Internet.
  • a network such as the Internet.
  • a case of generating the learned model for estimating and predicting a person attribute (for example, age, gender, or the like) and a purchasing behavior tendency of a visitor from a captured image of the visitor at a retail store will be described.
  • FIG. 1 is an overall configuration diagram of learned model providing system 1 according to the present disclosure.
  • learned model providing system 1 is configured of consent acquisition device 2 that acquires the use consent that an acquired image of visitors is to be used for generating a learned model from visitor S, a plurality of cameras 3 (sensing device) that image visitor S, learned model generating device 4 that generates the learned model by machine learning based on a captured image imaged by camera 3 , learned model providing device 5 (hereinafter, referred to as server device 5 ) that saves the learned model generated by learned model generating device 4 , learned model use device 6 (hereinafter, referred to as user side device 6 ) that receives the learned model from server device 5 , camera 7 to be used when performing processing using the learned model by the user side device 6 , and member database 8 .
  • consent acquisition device 2 that acquires the use consent that an acquired image of visitors is to be used for generating a learned model from visitor S
  • a plurality of cameras 3 (sensing device) that image visitor S
  • Learned model generating device 4 is connected to consent acquisition device 2 , camera 3 , and member database 8 via wired or wireless communication such as a local area network (LAN).
  • consent acquisition device 2 is connected to member database 8 via wired or wireless communication such as LAN.
  • Learned model generating device 4 , server device 5 , and user side device 6 are connected to each other via a network such as the Internet.
  • User side device 6 is connected to camera 7 via wired or wireless communication such as a local area network (LAN).
  • the connection form between the above-described devices is not particularly limited, and connection via wired or wireless communication such as LAN, connection via a network such as the Internet, general-purpose interface (for example, a universal serial bus (USB) interface) via a communication cable can be used.
  • USB universal serial bus
  • Consent acquisition device 2 is a general computer device and, for example, is configured integrally with a visit point giving device installed at an entrance of a store or the like for providing a visit point to visitor S via a portable terminal or a membership card possessed by visitor S.
  • the configuration form of consent acquisition device 2 is not particularly limited, and may be integrated with, for example, a cash register device, an in-store guidance device, or the like.
  • consent acquisition device 2 may not be formed integrally with other devices but may be configured alone.
  • Consent acquisition device 2 notifies visitor S (sensing object) that the use consent of the captured image (sensing information) of visitor S to be used for generating the learned model is requested on the screen, the voice, braille, or the like and receives input of permission of use consent from visitor S.
  • consent acquisition device 2 is provided with camera 25 for imaging a face of visitor S in a case where the use consent is obtained from visitor S.
  • FIG. 2 is a block diagram illustrating a schematic configuration of consent acquisition device 2 .
  • consent acquisition device 2 includes storage 21 , processor 22 , presenter 23 , input unit 24 , camera 25 , communicator 26 , and bus 27 connecting these components.
  • Storage 21 is a storage device (storage) such as a read only memory (ROM) or a hard disk, and stores various programs and various data items for realizing each function of consent acquisition device 2 .
  • Processor 22 is, for example, a central processing unit (CPU), reads various programs and various data items from storage 21 onto a random access memory (RAM) not shown, and executes each processing of consent acquisition device 2 .
  • CPU central processing unit
  • RAM random access memory
  • Presenter 23 is configured of, for example, a display such as a liquid crystal display panel, a speaker, a braille display, and the like, and is used for presenting visitor S requesting the use consent.
  • Input unit 24 is configured of an input device such as a liquid crystal touch panel, a microphone, and the like, and is used by visitor S to input permission of the use consent.
  • Camera 25 is a general imaging device such as a CCD camera, and is disposed at a position capable of imaging the face of visitor S and images the face of visitor S from whom the use consent is obtained. As far as camera 25 is capable of imaging the face of visitor S, its form, function, arrangement, quantity, and the like are not particularly limited, and it is possible to be modified variously.
  • Communicator 26 communicates with learned model generating device 4 and member database 8 via wired or wireless communication such as a LAN.
  • Camera 3 is a general imaging device such as a CCD camera, and is installed on a wall, a ceiling, or the like in various places in the store, and images the predetermined monitored area in the store as a still image or a video in time series.
  • the captured image (sensing information) captured by camera 3 is used as learning data when machine learning is performed by learned model generating device 4 .
  • Camera 3 includes a communicator (not shown), and a captured image captured by camera 3 is transmitted to learned model generating device 4 via the communicator.
  • camera 3 is capable of imaging the predetermined monitored area, its form, function, arrangement, quantity, and the like are not particularly limited, and it is possible to be modified variously.
  • Learned model generating device 4 is a general computer device, and generates the learned model by machine learning based on a captured image (sensing information) received from camera 3 .
  • the learned model is generated by deep learning using a multilayered neural network based on the captured image of camera 3 .
  • FIG. 3A is a block diagram illustrating a schematic configuration of learned model generating device 4
  • FIG. 3B is a block diagram illustrating a schematic configuration of a storage of learned model generating device 4 .
  • learned model generating device 4 includes storage 41 , processor 42 , display 43 , input unit 44 , communicator 45 , and bus 46 connecting these components.
  • Storage 41 is a storage device (storage) such as a ROM or a hard disk, and stores various programs and various data items for realizing each function of learned model generating device 4 .
  • storage 41 stores compensation database 47 for storing various data items used for compensation to be described later.
  • FIG. 3C is a diagram illustrating an example of compensation database 47 .
  • Each learned model is given a unique ID (learned model ID), and all persons included in the image or the like which is learning data used when generating the learned model (hereinafter, referred to as “consenter”) also has the unique ID (consenter ID).
  • the consenter ID of the consenter is stored with the appearance frequency in the learning data.
  • Processor 42 is, for example, the CPU, reads various programs and various data items from storage 41 onto the RAM not shown, and executes each processing of learned model generating device 4 .
  • Display unit 43 is configured of a display such as a liquid crystal display panel and is used for displaying the processing result in processor 42 and the like.
  • Input unit 44 is configured of an input device such as a keyboard, mouse, and the like, and is used for operating learned model generating device 4 .
  • the communicator 45 communicates with consent acquisition device 2 , camera 3 , and member database 8 via wired or wireless communication such as a LAN. In addition, communicator 45 communicates with server device 5 via a network such as the Internet.
  • the server device 5 is a general computer device, and stores one or more learned models received from learned model generating device 4 in learned model database 57 (see FIG. 4B ) to be described later.
  • server device 5 selects a learned model suitable for the usage request from one or more learned models stored in learned model database 57 and transmits the selected learned model to user side device 6 .
  • server device 5 may be configured as a cloud server for providing the pre-saved learned model to user side device 6 .
  • FIG. 4A is a block diagram illustrating a schematic configuration of server device 5
  • FIG. 4B is a block diagram illustrating a schematic configuration of the storage of server device 5
  • server device 5 includes storage 51 , processor 52 , display 53 , input unit 54 , communicator 55 , and bus 56 connecting these components.
  • Storage 51 is a storage device (storage) such as a read only memory (ROM) or a hard disk, and stores various programs and various data items for realizing each function of server device 5 .
  • storage 51 stores learned model database 57 for saving the learned model.
  • FIG. 4C is a diagram illustrating an example of learned model database 57 .
  • learned model database 57 in association with the learned model ID, model attribute information of the learned model, an ID (provider ID) assigned given to the device that generates and provides the learned model or the operator of the device, an ID (user ID) of the device using the learned model or the operator of the device, and a invalidation flag indicating whether the learned model is invalid are stored.
  • the model attribute information includes, for example, the learning purpose of the learned model, the learning environment, the incoming date and time of the learning data, the number of learning data items, the type of learning data, or resources and capabilities (resources and specifications) of the device used for machine learning (camera 3 and learned model generating device 4 ).
  • Processor 52 is, for example, the CPU, reads various programs and various data items from storage 51 onto the RAM not shown, and executes each processing of server device 5 .
  • Display unit 53 is configured of a display such as a liquid crystal display panel and is used for displaying the processing result in processor 52 and the like.
  • Input unit 54 is configured of an input device such as a keyboard, mouse, and the like, and is used for operating server device 5 .
  • Communicator 55 communicates with learned model generating device 4 and user side device 6 via wired or wireless communication such as the Internet.
  • server device 5 is configured as a cloud server
  • display 53 and input unit 54 are not indispensable components, and may be operated via a Web interface from a browser such as a client PC or the like via the Internet. In this case, the display and the input device of the client PC or the like substitute the functions of display 53 and input unit 54 .
  • User side device 6 is a general computer device and is used for performing image analysis processing using a learned model provided from server device 5 and processing such as new machine learning. In the present embodiments, user side device 6 performs image analysis processing using the learned model provided from server device 5 . As described above, provision of the learned model from server device 5 to user side device 6 is performed by user side device 6 transmitting a use request to server device 5 .
  • FIG. 5 is a block diagram illustrating a schematic configuration of user side device 6 .
  • user side device 6 includes storage 61 , processor 62 , display 63 , input unit 64 , communicator 65 , and bus 66 connecting these components.
  • Storage 61 is a storage device (storage) such as a ROM or a hard disk, and stores various programs and various data items for realizing each function of user side device 6 .
  • Processor 62 is, for example, the CPU, reads various programs and various data items from storage 61 onto the RAM not shown, and executes each processing of user side device 6 .
  • Display 63 is configured of a display such as a liquid crystal display panel and is used for displaying the processing result in processor 62 and the like.
  • Input unit 64 is configured of an input device such as a keyboard, mouse, and the like, and is used for operating user side device 6 .
  • Communicator 65 communicates with server device 5 via a network such as the Internet.
  • communicator 65 communicates with camera 7 via wired or wireless communication such as a LAN.
  • Camera 7 is a general imaging device such as a CCD camera and images an image (in this embodiment, an image including the face of the person) used in image analysis processing performed by user side device 6 .
  • Camera 7 includes a communicator not shown, and a captured image imaged by camera 7 is transmitted to user side device 6 via the communicator.
  • camera 7 is capable of imaging the image to be used for image analysis processing performed by user side device 6 or new machine learning, its form, function, arrangement, quantity, and the like are not particularly limited, and it is possible to be modified variously.
  • Member database 8 stores attribute information (for example, information on person attributes and information on purchase behavior attributes), face images, and the like of members of the store in association with the member ID of the member.
  • the purchase behavior history includes categories of products purchased by the member in the past, categories of products (that is, interesting products) that the member picked up in the past, and the like. Therefore, when visitor S is a member of the store, by searching the member database based on the member ID, it is possible to acquire the attribute information and the face image of visitor S. In a case where the face image of visitor S can be acquired based on the member ID in this manner, the face image stored in the member database is used as the face image of visitor S instead of acquiring the face image of visitor S by camera 25 (in this case, camera 25 is not an essential configuration).
  • the above-described devices of learned model providing system 1 are not limited to computer devices, and other information processing devices (for example, servers or the like) capable of obtaining similar functions can also be used.
  • information processing devices for example, servers or the like
  • at least a part of the functions of the each device of learned model providing system 1 may be replaced by other known hardware processing.
  • FIG. 6 is a sequence diagram illustrating an operation procedure of consent acquisition processing and anonymization processing in learned model providing system 1 .
  • the consent acquisition processing and the anonymization processing are performed by consent acquisition device 2 and learned model generating device 4 .
  • the operation procedure of the consent acquisition processing and the anonymization processing by consent acquisition device 2 and learned model generating device 4 will be described with reference to the sequence diagram of FIG. 6 .
  • a request for a use consent of using the captured image (sensing information) of visitor S (sensing object), imaged by cameras 3 installed in various places inside the store, to generate the learned model is presented in presenter 23 of consent acquisition device 2 by a screen, a sound, a braille, or the like (step ST 101 ).
  • This use consent may include consent to a fact that the generated learned model is provided to a third party.
  • visitor S operates input unit 24 of consent acquisition device 2 and inputs availability of use consent (step ST 102 ).
  • input unit 24 is configured of a liquid crystal touch panel
  • visitor S inputs availability of use consent by touching a button or an icon displayed on the liquid crystal touch panel.
  • input unit 24 is configured of a microphone
  • visitor S inputs propriety of use consent by a voice.
  • the processing proceeds to the next step ST 103 .
  • the processing is ended and the processing returns to the previous step ST 101 .
  • a unique consenter ID is given to visitor S (that is, the consenter) (step ST 103 ).
  • consent acquisition device 2 is configured integrally with the visiting point giving device, in the case where consenter S is a member of the store, when the visit point is given by a visiting point giving device, the member ID acquired from the portable terminal or the member card of the visitor may be used as the consenter ID of consenter S.
  • consent acquisition device 2 images the face of visitor S (consenter).
  • the face image of consenter S imaged by camera 25 (hereinafter, referred to as “consent face”) is associated with the consenter ID.
  • the consent face may be acquired by searching member database 8 based on member ID.
  • attribute information (for example, information on person attributes and information on purchase behavior attributes) of consenter S is given to the consent face as its metadata.
  • This attribute information may be input by a user of learned model generating device 4 by operating input unit 44 of the device 4 or in a case where consenter is a member of the store, the attribute information may be acquired from the member information registered in member database 8 in advance.
  • the consenter information including the consenter ID of the consenter, the consent face, and attribute information of the consenter is transmitted from consent acquisition device 2 to learned model generating device 4 (step ST 105 ).
  • Learned model generating device 4 saves the consenter information received from consent acquisition device 2 in storage 41 (step ST 106 ).
  • Learned model generating device 4 receives captured images imaged by cameras 3 from a plurality of cameras 3 installed in various places in the store (step ST 107 ).
  • the captured image received from camera 3 is stored in storage 41 of learned model generating device 4 .
  • learned model generating device 4 detects a face area of the person from the captured image. Learned model generating device 4 collates the face area detected from the captured image with the consent face stored in storage 41 .
  • non-consent face the face area other than the consent face (hereinafter, referred to as “non-consent face”) is specified and extracted (Step ST 108 ).
  • the consent face acquired by consent acquisition device 2 is used as data for collation for extracting a non-consent face from the captured image imaged by camera 3 .
  • the processing of extracting a non-consent face may be performed by real time processing or batch processing.
  • learned model generating device 4 performs anonymization processing on the area of the non-consent face in the captured image (step ST 109 ).
  • the anonymization processing for example, face deleting processing, mosaic processing, or masking processing can be used.
  • the captured image which is subjected to the anonymization processing is stored in storage 41 as a learning image (learning data) for use in machine learning (step ST 110 ).
  • learned model providing system 1 can create the learning data not containing the non-consent face by performing anonymization processing of the non-consent face (sensing information) of visitor S (sensing object) from which the use consent to be used for generating the learned model cannot be obtained.
  • FIG. 7 is a sequence diagram illustrating the operation procedure of the learned model generation processing and the electronic signature giving processing in learned model providing system 1 .
  • the learned model generating processing and the electronic signature giving processing are performed by learned model generating device 4 and server device 5 .
  • learned model generating device 4 and server device 5 the operation procedure of the learned model generating processing and the electronic signature giving processing by learned model generating device 4 and server device 5 will be described.
  • learned model generating device 4 performs machine learning using the learning image (learning data) saved in storage 41 and the correct data (teacher data) corresponding to the learning image, and generates a learned model (step ST 201 ).
  • the correct data is, for example, a person attribute of visitor S (for example, age, gender, or the like) and purchasing behavior attributes (for example, categories of products that purchased in the past, or a category of products that picked up in the past), and is set based on manual input by input unit 44 or the consenter information stored in storage 41 .
  • the learning processing is performed by updating the inter-node coupling parameter (weight value) of the neural network so that the error between the output data corresponding to the input learning data and the correct data becomes small based on the well-known method such as a reverse error propagation method to generate the learned model (that is, it is a set of internode coupling parameters, which may be expressed in a matrix form or may be represented by a simple numerical set) as the leaning processing result.
  • the learned model for outputting the person attributes and purchasing behavior attributes of visitor S as estimation and prediction results.
  • the learned model that is generated in such manner, since it is possible to predict the person attributes and purchasing behavior attributes of visitor S in real time, the learned model can be used in inventory management or order management of products by estimating purchasing behavior attributes of a plurality of visitors.
  • the unique model ID (learned model ID) is given to the generated learned model.
  • the model ID is saved in compensation giving database 47 in association with the consenter ID of the consenter from whom the use consent is obtained before the learning data creation used to generate the learned model (step ST 202 ).
  • step ST 203 when learned model generating device 4 creates the learning image (learning data) used for generating the learned model with respect to the generated learned model, the electronic signature that guarantees that anonymization processing of the non-consent face of visitor S from whom the use consent cannot be obtained is performed is given (step ST 203 ). Giving the electronic signature is performed based on a publicly known method using a public key (electronic signature using a public key certificate issued by a certificate authority).
  • model attribute information includes, for example, the learning purpose of the learned model, the learning environment, the incoming date and time of the learning data, the number of learning data items, the type of learning data, or resources and capabilities (resources and specifications) of the device used for machine learning (camera 3 and learned model generating device 4 ).
  • Server device 5 stores the signed learned model, public key certificate, and model attribute information received from learned model generating device 4 in learned model database 57 (step ST 205 ).
  • learned model providing system 1 can generate the learned model from the leaning image (learning data) not containing non-consent face, create the leaning image (learning data) used when generating the learned model with respect to the generated learned model, and give the electronic signature that guarantees that anonymization processing (processing or deleting) of the non-consent face (sensing information) of visitor S (sensing object) from whom the use consent cannot be obtained is performed.
  • FIG. 8 is a sequence diagram illustrating operation procedures of use request processing and the signature verification processing of the learned model in learned model providing system 1 .
  • the use request processing and the signature verification processing of the learned model are performed by server device 5 and user side device 6 .
  • operation procedures of the use request processing and the signature verification processing of the learned model by server device 5 and user side device 6 will be described with reference to the sequence diagram of FIG. 8 .
  • a user who desires to receive provision of the learned model from server device 5 inputs a request to use the learned model to user side device 6 (step ST 301 ).
  • User side device 6 to which the request for using the learned model is input from the user transmits the use request of the learned model, use attribute information of user side device 6 and usage compensation of the learned model to user side device 6 (Step ST 302 ).
  • the use attribute information includes, for example, the usage purpose of the user, the resources and capabilities (resources and specifications) of user side device 6 and camera 7 , the learning environment and the number of learning data items when using the learned model in user side device 6 .
  • the usage compensation of the learned model is, for example, a virtual currency or a point.
  • server device 5 When server device 5 receives the learned model use request, the use attribute information, and the usage compensation from user side device 6 , server device 5 uses the known matching method to perform matching the use attribute information received from user side device 6 with the model attribute information of the learned model stored in model database 57 , and selects the signed learned model matching the use attribute information (step ST 303 ). The selected learned model is transmitted to user side device 6 by attaching the corresponding public key certificate thereto (step ST 304 ).
  • user side device 6 verifies the validity of the electronic signature given to the learned model using the public key of the public key certificate attached to the learned model (step ST 305 ).
  • the analyzing engine is configured by loading the learned model into the analysis engine program (program configuring the neural network and executing analysis processing based thereon) (Step ST 306 ).
  • the captured image is input from camera 7 to user side device 6 (step ST 307 ), and a predetermined image analysis (in this embodiment, the person attribute of the visitor or a purchase behavior attribute estimation and prediction) is executed by using the input captured image (step ST 308 ), and the analysis result is output (step ST 309 ).
  • step ST 305 In a case where the verification result is invalid (NG) (step ST 305 : NG), user side device 6 selects another learned model and transmits the information indicating that the verification result is invalid and the re-request notification for requesting re-transmission by selecting the other learned model to server device 5 (step ST 310 ).
  • server device 5 When the re-request notification of the learned model is received from user side device 6 , server device 5 writes an invalidation flag in the learned model in which the verification result is NG in learned model database 57 , whereby performing the invalid processing of the learned model (Step ST 311 ). Thereafter, the processing returns to step ST 303 , matching is performed again, and a learned model is newly selected.
  • user side device 6 can receive provision of the learned model suitable for the use attribute information of user side device 6 from server device 5 .
  • the invalid processing can be performed on the learned model.
  • FIG. 9 is a sequence diagram illustrating the operation procedure of a learned model use report processing in learned model providing system 1 and compensation processing to the consenter.
  • the use report processing of the learned model and the compensation processing to the consenter are performed by learned model generating device 4 , server device 5 , and user side device 6 .
  • learned model generating device 4 the operation procedure of the learned model use report processing and the compensation processing to the consenter by learned model generating device 4 , server device 5 , and user side device 6 will be described with reference to the sequence diagram of FIG. 9 .
  • step ST 305 OK
  • user side device 6 transmits the use determination report indicating that the learned model is determined to be used and the model ID of the learned model which is determined to be used to server device 5 (step ST 401 ).
  • server device 5 When receiving the use determination report from user side device 6 , server device 5 records the use history indicating that the learned model provided to user side device 6 is used in learned model database 57 in association with the model ID of the learned model (step ST 402 ).
  • server device 5 transmits the use report indicating that the learned model provided to user side device 6 is used, the model ID of the learned model, and the compensation data to learned model generating device 4 (Step ST 403 ).
  • the compensation data is data to be given as compensation to the consenter from whom the consent for use is obtained when generating the learned model, and is, for example, data such as a virtual currency or a point.
  • learned model generating device 4 searches for the consenter from whom the use consent is obtained when generating the learned model determined to be used by user side device 6 (Step ST 404 ). Specifically, the consenter ID is obtained by searching compensation database 47 based on the model ID of the learned model determined to be used by user side device 6 . As described above, since the model ID is stored in compensation database 47 in association with the consenter ID, the consenter ID can be obtained based on the model ID.
  • Learned model generating device 4 gives the compensation data (for example, data such as virtual currency and points) to the consenter (step ST 405 ).
  • Giving the compensation data to the consenter may be performed to the portable terminal or the member card of the consenter from consent acquisition device 2 or the visit point giving device, for example, at the time of the next visiting of the consenter.
  • the method of giving the compensation data to the consenter is not limited thereto, and may be various other methods. In the case where there are a plurality of consenters to whom compensation data is to be given, it is advisable to distribute the compensation data to each consenter at a predetermined proportion. The proportion of distribution may be determined based on the contribution degree (for example, number of appearances, or the like) of the consenter in the learning image used for generating the learned model.
  • server device 5 when server device 5 receives the use determination report from user side device 6 , the use history of the learned model is recorded in learned model database 57 , and the compensation data can also be given to a consenter from whom the use consent is obtained when generating the learned model.
  • the learning image (learning data) used for the machine learning in learned model generating device 4 is generated by performing anonymization processing to the face area (non-consent face) other than the face image (consent face) of consenter S imaged by camera 25 of consent acquisition device 2 among the face areas included in the captured image imaged by camera 3 installed in various places in the store.
  • the face image (consent face) of consenter S stored in advance in the portable terminal possessed by the consenter S may be acquired from the portable terminal of consenter S by near field communication and may generate a learning image by performing anonymization processing based on the collation between the consent face and the captured image by camera 3 . In this case, as shown in FIGS.
  • consent acquisition device 2 is provided with near filed communicator 28 and performs near field communication with portable terminal 9 of consenter S via near filed communicator 28 . Thereby, the face image (consent face) of consenter S stored in advance in portable terminal 9 is acquired.
  • near field communication for example, WiFi (registered trademark), Bluetooth (registered trademark), infrared communication, or the like can be used.
  • all of the processings of the anonymization processing, the learned model generating processing, the electronic signature giving processing, and the compensation processing are performed by learned model generating device 4 .
  • at least part of each processing may be performed by server device 5 .
  • the anonymization processing and the compensation processing may be performed by learned model generating device 4
  • the learning model generating processing and electronic signature giving processing may be performed by server device 5 .
  • the compensation data is configured to be given to the consenter.
  • the consent about being used for generation of the learned model is acquired from the sensing object.
  • the anonymization processing of the captured image of camera 3 for the non-consent sensing object may be performed after acquiring an intention (non-consent) not consent to be used for generating of the learned model or selecting the consent or the non-consent to be used for generating the learned model.
  • the attribute and behavior of the person may be learned based on the sound data acquired by a microphone or positional data acquired using GPS, wireless tag, or Bluetooth (registered trademark) beacon or the like. In this case, by performing analysis using the learned model, it become possible to predict and estimate the attribute and behavior of the person from the sound or positional data, and the like.
  • the verification processing of the electronic signature given to the learned model is configured to be performed by user side device 6 .
  • server device 5 receives the learned model from learned model generating device 4
  • the verification processing of the electronic signature may be performed in server device 5 .
  • the invalidation processing of the learned model can be executed as processing closed inside server device 5 .
  • the object of machine learning is to generate the learned model for estimating attributes (for example, age, gender, or the like) of the person and the purchasing behavior tendency from the captured image of the person.
  • attributes for example, age, gender, or the like
  • the object of the machine learning is not limited to this and may be various objects in various fields such as manufacture, distribution, public service, transportation, medical care, education, or finance.
  • the method for generating a learned model, the learned model generating device, and the learned model use device according to the present disclosure are useful as the method for generating a learned model and the learned model generating device capable of generating the learned model after obtaining use consent from the sensing object and the learned model use device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Development Economics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Library & Information Science (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Computing Systems (AREA)
  • Bioethics (AREA)
  • Human Resources & Organizations (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Game Theory and Decision Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Tourism & Hospitality (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
US16/097,968 2017-02-03 2017-12-11 Learned model generating method, learned model generating device, and learned model use device Active 2038-03-07 US10810461B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2017018293 2017-02-03
JP2017-018293 2017-02-03
PCT/JP2017/044295 WO2018142764A1 (ja) 2017-02-03 2017-12-11 学習済みモデル生成方法、学習済みモデル生成装置、および学習済みモデル利用装置

Publications (2)

Publication Number Publication Date
US20190095757A1 US20190095757A1 (en) 2019-03-28
US10810461B2 true US10810461B2 (en) 2020-10-20

Family

ID=63039488

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/097,968 Active 2038-03-07 US10810461B2 (en) 2017-02-03 2017-12-11 Learned model generating method, learned model generating device, and learned model use device

Country Status (5)

Country Link
US (1) US10810461B2 (ja)
EP (1) EP3579160A1 (ja)
JP (1) JP7129598B2 (ja)
CN (1) CN109074601A (ja)
WO (1) WO2018142764A1 (ja)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210176430A1 (en) * 2019-12-06 2021-06-10 Canon Kabushiki Kaisha Image capturing apparatus using learned model, information processing apparatus, methods of controlling respective apparatuses, learned model selection system, and storage medium
US11227170B2 (en) * 2017-07-20 2022-01-18 Panasonic Intellectual Property Management Co., Ltd. Collation device and collation method
US20220246303A1 (en) * 2019-10-17 2022-08-04 Olympus Corporation Learning support system and learning support method
US20220319160A1 (en) * 2019-06-27 2022-10-06 Koninklijke Philips N.V. Multi-task deep learning method for a neural network for automatic pathology detection
US12131525B2 (en) * 2019-06-27 2024-10-29 Koninklijke Philips N.V. Multi-task deep learning method for a neural network for automatic pathology detection

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6398894B2 (ja) * 2015-06-30 2018-10-03 オムロン株式会社 データフロー制御装置およびデータフロー制御方法
US11410024B2 (en) 2017-04-28 2022-08-09 Intel Corporation Tool for facilitating efficiency in machine learning
US10417527B2 (en) * 2017-09-06 2019-09-17 Irdeto B.V. Identifying an object within content
US20190236360A1 (en) * 2018-01-30 2019-08-01 Mashgin Inc. Feedback loop for image-based recognition
US11526759B2 (en) * 2018-11-05 2022-12-13 International Business Machines Corporation Large model support in deep learning
US11004449B2 (en) * 2018-11-29 2021-05-11 International Business Machines Corporation Vocal utterance based item inventory actions
CN113614754A (zh) * 2019-03-27 2021-11-05 松下知识产权经营株式会社 信息处理系统、计算机系统、信息处理方法和程序
JP7403995B2 (ja) * 2019-08-22 2023-12-25 キヤノン株式会社 情報処理装置、制御方法およびプログラム
US11537689B2 (en) 2019-10-10 2022-12-27 Baidu Usa Llc Method and system for signing an artificial intelligence watermark using a kernel
US11704390B2 (en) 2019-10-10 2023-07-18 Baidu Usa Llc Method and system for signing an artificial intelligence watermark using a query
US11457002B2 (en) 2019-10-10 2022-09-27 Baidu Usa Llc Method and system for encrypting data using a command
US11637697B2 (en) 2019-10-10 2023-04-25 Baidu Usa Llc Method and system for signing output using a kernel
US11775692B2 (en) 2019-10-10 2023-10-03 Baidu Usa Llc Method and system for encrypting data using a kernel
US11436305B2 (en) * 2019-10-10 2022-09-06 Baidu Usa Llc Method and system for signing an artificial intelligence watermark using implicit data
US20220374950A1 (en) * 2019-10-15 2022-11-24 Nec Corporation Consideration calculation device, control method, and non-transitory storage medium
CN114586033A (zh) * 2019-10-31 2022-06-03 美国西门子医学诊断股份有限公司 用于在自动化诊断分析系统中对患者进行表征期间保护患者信息的方法和装置
EP3840415B1 (en) * 2019-12-19 2023-08-02 GN Hearing A/S Hearing device with third party model control and related methods
JP7469912B2 (ja) * 2020-03-03 2024-04-17 キヤノン株式会社 情報処理装置、推論装置、それらの制御方法、プログラム、記憶媒体
JP7456844B2 (ja) * 2020-05-13 2024-03-27 東芝ライフスタイル株式会社 情報処理システム
US20230237774A1 (en) * 2020-07-07 2023-07-27 Sony Group Corporation Data collection system, sensor device, data collection device, and data collection method
WO2022101996A1 (ja) * 2020-11-10 2022-05-19 日本電信電話株式会社 モデル利用制御システム、モデル利用制御方法、モデル利用制御装置、および、モデル利用制御プログラム
JP6982672B1 (ja) * 2020-11-21 2021-12-17 株式会社オプティム プログラム、方法、情報処理装置及びシステム
JP7176019B2 (ja) * 2021-01-29 2022-11-21 株式会社 みずほ銀行 認証システム、認証方法及び認証プログラム
CN114971443A (zh) * 2021-02-20 2022-08-30 菜鸟智能物流控股有限公司 一种物流对象的处理方法及成像设备
KR20240044929A (ko) * 2022-09-29 2024-04-05 네이버랩스 주식회사 식별자 인식 기반 위치 결정 방법 및 장치
KR102663409B1 (ko) * 2023-05-30 2024-05-03 김수진 기계학습용 데이터 수집 장치 및 방법

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004318391A (ja) 2003-04-15 2004-11-11 Mitsubishi Electric Corp 情報提供装置及び情報提供システム及び分散データベースシステム
JP2006244301A (ja) 2005-03-04 2006-09-14 Nec Corp 画像証明システム及び証明書発行システム並びにこれらを用いた証明書発行支援システム
US20100281267A1 (en) 2009-04-30 2010-11-04 Sakumoto Koichi Image Processing Apparatus, Electronic Signature Generation System, Electronic Signature Key Generation Method, Image Processing Method, and Program
US20120087556A1 (en) * 2010-10-12 2012-04-12 Shengyang Dai Digital image analysis using multi-step analysis
JP5445722B1 (ja) 2012-09-12 2014-03-19 オムロン株式会社 データフロー制御指令発生装置およびセンサ管理装置
US9147128B1 (en) * 2013-11-12 2015-09-29 214 Technologies Inc. Machine learning enhanced facial recognition
JP2015220574A (ja) 2014-05-16 2015-12-07 ソニー株式会社 情報処理システム、記憶媒体、およびコンテンツ取得方法
JP2016053869A (ja) 2014-09-04 2016-04-14 富士ゼロックス株式会社 情報処理装置及び情報処理プログラム
JP2016126597A (ja) 2015-01-06 2016-07-11 マツダ株式会社 車両用歩行者画像取得装置
WO2016205432A1 (en) 2015-06-16 2016-12-22 Microsoft Technology Licensing, Llc Automatic recognition of entities in media-captured events
US20170054895A1 (en) * 2015-08-19 2017-02-23 Google Inc. Smart image sensor having integrated memory and processor
US20170076142A1 (en) * 2015-09-15 2017-03-16 Google Inc. Feature detection and masking in images based on color distributions
US9672370B1 (en) * 2014-10-24 2017-06-06 Google Inc. Sharing digital media within a space
US10062133B1 (en) * 2016-01-26 2018-08-28 Google Llc Image retrieval for computing devices

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120323794A1 (en) * 2011-06-15 2012-12-20 Microsoft Corporation Monetization strategies in privacy-conscious personalization
KR20150021280A (ko) * 2013-08-20 2015-03-02 한국전자통신연구원 영상 처리 장치 및 그 동작방법
US9519408B2 (en) * 2013-12-31 2016-12-13 Google Inc. Systems and methods for guided user actions
JP6428152B2 (ja) * 2014-10-24 2018-11-28 富士通株式会社 肖像権保護プログラム、情報通信装置及び肖像権保護方法
JP5848481B1 (ja) * 2015-05-12 2016-01-27 グリー株式会社 情報管理システム、情報管理方法及び情報管理プログラム

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004318391A (ja) 2003-04-15 2004-11-11 Mitsubishi Electric Corp 情報提供装置及び情報提供システム及び分散データベースシステム
JP2006244301A (ja) 2005-03-04 2006-09-14 Nec Corp 画像証明システム及び証明書発行システム並びにこれらを用いた証明書発行支援システム
US20100281267A1 (en) 2009-04-30 2010-11-04 Sakumoto Koichi Image Processing Apparatus, Electronic Signature Generation System, Electronic Signature Key Generation Method, Image Processing Method, and Program
JP2010278482A (ja) 2009-04-30 2010-12-09 Sony Corp 情報処理装置、電子署名生成システム、電子署名用の鍵生成方法、情報処理方法、及びプログラム
US20120087556A1 (en) * 2010-10-12 2012-04-12 Shengyang Dai Digital image analysis using multi-step analysis
US20160210862A1 (en) 2012-09-12 2016-07-21 Omron Corporation Data flow control order generating apparatus and sensor managing apparatus
JP5445722B1 (ja) 2012-09-12 2014-03-19 オムロン株式会社 データフロー制御指令発生装置およびセンサ管理装置
US20140372561A1 (en) 2012-09-12 2014-12-18 Omron Corporation Data flow control order generating apparatus and sensor managing apparatus
US9147128B1 (en) * 2013-11-12 2015-09-29 214 Technologies Inc. Machine learning enhanced facial recognition
JP2015220574A (ja) 2014-05-16 2015-12-07 ソニー株式会社 情報処理システム、記憶媒体、およびコンテンツ取得方法
US20170039495A1 (en) 2014-05-16 2017-02-09 Sony Corporation Information processing system, storage medium, and content acquisition method
JP2016053869A (ja) 2014-09-04 2016-04-14 富士ゼロックス株式会社 情報処理装置及び情報処理プログラム
US9672370B1 (en) * 2014-10-24 2017-06-06 Google Inc. Sharing digital media within a space
JP2016126597A (ja) 2015-01-06 2016-07-11 マツダ株式会社 車両用歩行者画像取得装置
WO2016205432A1 (en) 2015-06-16 2016-12-22 Microsoft Technology Licensing, Llc Automatic recognition of entities in media-captured events
US9704020B2 (en) * 2015-06-16 2017-07-11 Microsoft Technology Licensing, Llc Automatic recognition of entities in media-captured events
US20170054895A1 (en) * 2015-08-19 2017-02-23 Google Inc. Smart image sensor having integrated memory and processor
US20170076142A1 (en) * 2015-09-15 2017-03-16 Google Inc. Feature detection and masking in images based on color distributions
US10062133B1 (en) * 2016-01-26 2018-08-28 Google Llc Image retrieval for computing devices

Non-Patent Citations (10)

* Cited by examiner, † Cited by third party
Title
"‘Metadata Incorporated’, a venture established by leaders in AI research, to sell artificial intelligence that has been taught," Dream Gate Bureau, https://web.archive.org/web/20161120124912/http://www.dreamgate.gr.jp/news/4183, p. 1, 1.5, Nov. 20, 2016, together with an English language translation thereof.
Communication pursuant to Article 94(3) EPC dated Aug. 13, 2020 for the related European Patent Application No. 17895308.9.
International Search Report issued in International Patent Application No. PCT/JP2017/044295, dated Febmary 27, 2018.
MARTIN ATZMUELLER, ALVIN CHIN, FREDERIK JANSSEN, IMMANUEL SCHWEIZER, CHRISTOPH TRATTNER (EDS.): "Big data analytics in the social and ubiquitous context : 5th International Workshop on Modeling Social Media, MSM 2014, 5th International Workshop on Mining Ubiquitous and Social Environments, MUSE 2014 and first International Workshop on Machine Learning for Urban Sensor Data, SenseML 2014 : revis", vol. 5535 Chap.14, 22 June 2009, SPRINGER, Berlin, Heidelberg, ISBN: 978-3-642-17318-9, article MORI JUNICHIRO; MATSUO YUTAKA; KOSHIBA HITOSHI; AIHARA KENRO; TAKEDA HIDEAKI: "Predicting Customer Models Using Behavior-Based Features in Shops", pages: 126 - 137, XP047437808, 032548, DOI: 10.1007/978-3-642-02247-0_14
Mori Junichiro et al: "Predicting Customer Models Using Behavior-Based Features in Shops", Jun. 22, 2009 (Jun. 22, 2009), International Conference on Computer Analysis of Images and Patterns. CAIP 2017: Computer Analysis of Images and Patterns; [Lecture Notes in Computer Science; Lect. Notes Computer], Springer, Berlin, Heidelberg, pp. 126-137, XP047437808.
Sei Yuichi et al: "Privacy-Preserving Publication of Deep Neural Networks", 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/Smartcity/DSS), IEEE, Dec. 12, 2016 (Dec. 12, 2016), pp. 1418-1425, XP033052562.
SEI YUICHI; OKUMURA HIROSHI; OHSUGA AKIHIKO: "Privacy-Preserving Publication of Deep Neural Networks", 2016 IEEE 18TH INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS; IEEE 14TH INTERNATIONAL CONFERENCE ON SMART CITY; IEEE 2ND INTERNATIONAL CONFERENCE ON DATA SCIENCE AND SYSTEMS (HPCC/SMARTCITY/DSS), IEEE, 12 December 2016 (2016-12-12), pages 1418 - 1425, XP033052562, DOI: 10.1109/HPCC-SmartCity-DSS.2016.0202
The Extended European Search Report dated Oct. 7, 2019 for the related European Patent Application No. 17895308.9.
U.S. Appl. No. 16/097,998 to Yuichi Matsumoto et al., filed Oct. 31, 2018.
U.S. Appl. No. 16/098,023 to Yuichi Matsumoto et al., filed Oct. 31, 2018.

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11227170B2 (en) * 2017-07-20 2022-01-18 Panasonic Intellectual Property Management Co., Ltd. Collation device and collation method
US20220319160A1 (en) * 2019-06-27 2022-10-06 Koninklijke Philips N.V. Multi-task deep learning method for a neural network for automatic pathology detection
US12131525B2 (en) * 2019-06-27 2024-10-29 Koninklijke Philips N.V. Multi-task deep learning method for a neural network for automatic pathology detection
US20220246303A1 (en) * 2019-10-17 2022-08-04 Olympus Corporation Learning support system and learning support method
US20210176430A1 (en) * 2019-12-06 2021-06-10 Canon Kabushiki Kaisha Image capturing apparatus using learned model, information processing apparatus, methods of controlling respective apparatuses, learned model selection system, and storage medium
US11546553B2 (en) * 2019-12-06 2023-01-03 Canon Kabushiki Kaisha Image capturing apparatus using learned model, information processing apparatus, methods of controlling respective apparatuses, learned model selection system, and storage medium

Also Published As

Publication number Publication date
WO2018142764A1 (ja) 2018-08-09
JP7129598B2 (ja) 2022-09-02
US20190095757A1 (en) 2019-03-28
CN109074601A (zh) 2018-12-21
EP3579160A4 (en) 2019-12-11
JPWO2018142764A1 (ja) 2019-11-21
EP3579160A1 (en) 2019-12-11

Similar Documents

Publication Publication Date Title
US10810461B2 (en) Learned model generating method, learned model generating device, and learned model use device
US20190147361A1 (en) Learned model provision method and learned model provision device
US10803407B2 (en) Method for selecting learned model corresponding to sensing data and provisioning selected learned model, and learned model provision device
Yepes-Estrada et al. The global earthquake model physical vulnerability database
US20190272575A1 (en) Methods and systems for measuring articles of apparel
He et al. Diversified third-party library prediction for mobile app development
WO2020020137A1 (zh) 商品推荐方法、装置、系统及计算机可读存储介质
Anthony A study of consumer acceptance of mobile payment services in Hong Kong
KR20210066495A (ko) 렌탈 서비스 제공 시스템
US20200099621A1 (en) Sensing device management apparatus
JP7221614B2 (ja) 物件情報処理装置及び物件情報処理方法、物件情報処理プログラム並びに物件情報処理システム
JP2015069404A (ja) 調査システム、調査方法、サーバ、ユーザ端末、プログラム、記録媒体
JP2023119055A (ja) 情報処理装置、情報処理方法及びプログラム
US20230317215A1 (en) Machine learning driven automated design of clinical studies and assessment of pharmaceuticals and medical devices
KR20180087633A (ko) 관리서버, 이를 포함하는 피부관리 시스템 및 방법
Lengua et al. Design of a optimization algorithm for binary classification
KR102283607B1 (ko) 임대물 내의 시설물의 파손 정보를 결정하기 위한 컴퓨팅 장치 및 방법
US20230030754A1 (en) Information processing server, information processing system, determination device, and method
CN118679488A (zh) 一种按主题提供搜索结果的电子设备的控制方法
JP6724325B2 (ja) 情報処理装置
Cano Lengua et al. Design of a optimization algorithm for binary classification
JP2023039447A (ja) プログラム、システムおよび情報処理方法
KR20210133418A (ko) 상가 부동산 매물 정보 제공 방법 및 서버
CN114553684A (zh) 网点运维方法、装置、计算机设备、存储介质
CN117056561A (zh) 视频推荐方法、装置、设备、存储介质及程序产品

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUGIURA, MASATAKA;MATSUMOTO, YUICHI;REEL/FRAME:048720/0339

Effective date: 20180824

Owner name: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUGIURA, MASATAKA;MATSUMOTO, YUICHI;REEL/FRAME:048720/0339

Effective date: 20180824

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4