WO2023286105A1

WO2023286105A1 - Information processing device, information processing system, information processing program, and information processing method

Info

Publication number: WO2023286105A1
Application number: PCT/JP2021/026060
Authority: WO
Inventors: 井上哲也; 小栗佳祐
Original assignee: 株式会社ウフル
Priority date: 2021-07-12
Filing date: 2021-07-12
Publication date: 2023-01-19
Also published as: JPWO2023286105A1

Abstract

[Problem] To generate data to be used for machine learning. [Solution] This information processing device comprises: an acquisition unit that acquires a result of detection by a sensor that detects a subject, and acquires, from a predetermined terminal associated with the subject, state information representing the state of the subject and obtained from information different from the result of detection by the sensor; and a generation unit that generates data to be used for machine learning by using the result of detection and state information acquired by the acquisition unit.

Description

Information processing device, information processing system, information processing program, and information processing method

The present invention relates to an information processing device, an information processing system, an information processing program, and an information processing method.

Artificial intelligence (hereinafter referred to as AI) that uses machine learning such as deep learning is expected to be used in various fields. For machine learning, for example, teacher data, which is a set of known inputs and outputs, is used. Patent Document 1 (Japanese Patent No. 6492880) below proposes machine learning using feature information extracted from an image and user information input by a user visually recognizing the image. Inference results by AI depend on teacher data used for machine learning. Therefore, in the field of machine learning, a technique for generating teacher data for machine learning according to the use of AI is desired.

Patent No. 6492880

According to the first aspect of the present invention, the detection result of the sensor that detects the target is acquired, and the state information obtained from the information different from the detection result of the sensor and representing the state of the target is associated with the target. An information processing apparatus is provided that includes an acquisition unit that acquires from a predetermined terminal, and a generation unit that generates data used for machine learning using the detection result and state information acquired by the acquisition unit.

According to the second aspect of the present invention, the information processing apparatus according to the first aspect processes the detection result obtained by the obtaining unit using a model generated by machine learning by the learning unit to estimate the state of the object. An information processing system is provided, comprising: an estimator;

According to the third aspect of the present invention, the computer acquires the detection result of the sensor that detects the target, and the state information obtained from the information different from the detection result of the sensor and representing the state of the target is transmitted to the target. and generating data used for machine learning using the obtained detection results and state information.

According to a fourth aspect of the present invention, an information processing method using one or more computers, wherein the communication unit of the computer receives the detection result of a sensor that detects an object, and receiving state information obtained from different information and representing the state of the object from a predetermined terminal associated with the object; and generating data utilized for machine learning.

1 is a diagram showing an information processing system according to a first embodiment; FIG. FIG. 4 is a diagram showing processing related to a predetermined terminal; FIG. 4 is a diagram showing association between sensors and terminals; It is a figure which shows the process which derives the detection result of the sensor corresponding to state information. It is a figure which shows the process which performs machine learning. It is a figure which shows the information processing method which concerns on 1st Embodiment. It is a figure which shows the information processing system which concerns on 2nd Embodiment. It is a figure which shows the information processing system which concerns on 3rd Embodiment. It is a figure which shows the information processing system which concerns on 4th Embodiment. It is a figure which shows the computer which concerns on embodiment.

[First embodiment]
A first embodiment will be described. FIG. 1 is a diagram showing an information processing system according to the first embodiment. The information processing system 1 associates the first information and the second information regarding the target 2 to generate data used for machine learning. This data includes teacher data for machine learning or data that is the source of the teacher data. In the following description, data used for machine learning is collectively referred to as teacher data as appropriate.

In the present embodiment, the first information corresponds to the result of detection of the target 2 by the sensor 10, and the second information corresponds to the result of the person 3 judging the state of the target 2. In this embodiment, the subject 2 is a person. In the following description, the target 2 will be referred to as the target person 2 as appropriate. The subject 2 is, for example, an employee working in an office. The person 3 is, for example, an evaluator who evaluates the working condition of the subject person 2 . Person 3 will be referred to as evaluator 3 as appropriate in the following description.

The information processing system 1 uses machine learning to generate an AI model that estimates the second information from the first information. In the following description, the above model will be referred to as an inference model as appropriate. The information processing system 1 associates, for example, the detection result (first information) of the detection of the subject 2 by the sensor 10 with the evaluation result (second information) of the state of the subject 2 evaluated by the evaluator 3. to perform machine learning. The information processing system 1 generates an AI inference model for estimating the state of the subject 2 from the detection result of the sensor 10 by machine learning.

After generating the inference model, the information processing system 1 estimates the state of the subject 2 from the detection results of the sensor 10 using AI in which this inference model is implemented. For example, the information processing system 1 estimates the working condition as the condition of the subject 2 . The information processing system 1 provides the application 60 with the estimated work status of the subject person 2 . The application 60 includes, for example, an application that manages attendance of the subject 2 . The application 60, for example, accumulates the results of estimating the working condition of the subject 2 and creates a log. For example, the superior of the subject 2 (for example, the evaluator 3) can grasp the attendance of the subject 2 from the log generated by the application 60. FIG.

According to the information processing system 1 in the present embodiment, for example, based on the detection result of the sensor 10 that detects the data (physical quantity, etc.) of the subject 2, the AI in which the inference model is implemented estimates the state of the subject. and can be managed. For example, after the inference model is generated, the information processing system 1 reduces the time and effort of the evaluator 3 to evaluate and objectively evaluates the state of the subject 2 in managing the state of the subject 2. help to do

Next, each part of the information processing system 1 will be described. The information processing system 1 includes a sensor 10 , a terminal 20 , an information processing device 30 , an information processing device 40 , and an information processing device 50 .

The sensor 10 detects the target 2. Sensors 10 include, for example, sensors associated with subject 2 . The sensor 10 may be a wearable device such as a smartwatch worn by the subject 2 . The sensor 10 detects, for example, information representing the life activity of the subject 2 . The information representing vital activity includes, for example, at least one of the subject's 2 body temperature, the subject's 2 surface humidity, pulse, blood pressure, blood oxygen concentration, and respiratory rate. The sensor 10 may detect the movement (eg, acceleration) of the part (eg, hand) of the subject 2 to which the sensor 10 is attached. The sensor 10 may include smart glasses and may detect the line of sight of the subject 2 .

In this embodiment, the terminal 20 includes the terminal of the evaluator 3 associated with the subject 2. The status information includes, for example, information input to the terminal 20 by the evaluator 3 (working status of the subject 2, etc.). The terminal 20 may be a portable device (eg, smart phone, tablet) possessed by the evaluator 3, or may be a stationary device (eg, personal computer).

FIG. 2 is a diagram showing an example of processing related to a terminal. Here, the name of the target person 2 is assumed to be "A". The terminal 20 is a smartphone, for example, and includes a display unit 21 . The display unit 21 is overlapped with, for example, a transmissive touch pad, and the display unit 21 and the touch pad constitute a touch panel. The terminal 20 displays information D1 to D4 on the display unit 21, for example.

The information D1 includes information indicating that the input of the condition of the subject 2 is accepted. For example, the information D1 includes the text "What is your status?" as information notifying that the status input is being accepted. The information D1 may include information specifying the target 2 . For example, the information D1 includes text of the name of the target person 2 (eg, “Mr. A”) as information specifying the target 2 . The information specifying the target 2 may include information different from the target person's name, and may include, for example, identification information (eg, employee number) of the target person 2 . The information specifying the target 2 may be displayed separately from the information notifying that the status of the target is being received. The information D1 may not include information specifying the target 2 . For example, when the number of subjects 2 corresponding to the terminal 20 is 1, the evaluator 3 can specify the subject whose state is to be input even if the information specifying the subject 2 is not displayed.

Information D2-D4 includes information corresponding to a plurality of candidates representing the state of target 2. In FIG. 2, information D2 to information D4 are items (eg, icons) in the displayed image. In the following description, information D2, information D3, and information D4 will be referred to as icon D2, icon D3, and icon D4, respectively. Icon D2 includes the text "at work". Icon D3 includes the text "resting". Icon D4 contains the text "Other". The texts "at work", "resting", and "others" are information representing candidate states of the subject 2, respectively. Information representing candidates for the state of the object 2 may be represented in a form different from text (for example, a pictogram).

When the icon D2 is selected, the terminal 20 detects that information associated with the icon D2 (eg, "at work") has been input as information representing the state of the target 2. For example, the evaluator 3 visually recognizes the subject 2 and determines that the subject 2 is at work. In this case, when the evaluator 3 touches the icon D2, information corresponding to "at work" is input to the terminal 20 as the status information.

The "Other" icon D4 has a state of the target 2 different from the state candidates represented by the other icons (icon D2, icon D3), for example, when the state of the target 2 is "absent". is selected when it is determined that The “other” icon D4 may be selected when the evaluator 3 cannot determine the condition of the subject 2, such as when the evaluator 3 is in a position where the subject 2 cannot be recognized.

The evaluator 3 may directly recognize the real object of the subject 2 and input the state of the subject 2 into the terminal 20 . Direct recognition includes, for example, evaluator 3 obtaining information about subject 2 without going through a device. For example, the evaluator 3 may judge the state of the subject 2 by directly seeing the subject 2 or directly hearing the sound emitted by the subject 2 , and input the state of the subject 2 to the terminal 20 . The evaluator 3 may ask the subject 2 what state he or she is in using voice, text, or gestures, and input the state of the subject 2 into the terminal 20 based on the response from the subject 2. .

The evaluator 3 may indirectly recognize the subject 2 and input the state of the subject 2 to the terminal 20 . Indirectly perceiving includes, for example, evaluator 3 obtaining information about subject 2 via a device. The evaluator 3 inputs the state of the target 2 to the terminal 20 using the result of detecting the target 2 with a sensor (suitably referred to as a second sensor) different from the sensor 10 (suitably referred to as a first sensor). You may The second sensor may differ from the first sensor in the type of information it detects. For example, the first sensor may be a sensor that detects the subject's 2 vital activity, and the second sensor may be a camera that detects the subject's 2 appearance. In this case, the evaluator 3 may input the state of the subject 2 to the terminal 20 by viewing an image (eg, moving image) of the subject 2 captured by a camera.

The positional relationship of the second sensor with respect to the target 2 may be different from that of the first sensor. For example, the second sensor may have a different distance from the target 2 than the first sensor. For example, the first sensor may be a sensor attached to the target 2, such as a wearable device, and the second sensor may be a sensor remote from the target 2, such as a surveillance camera. Also, the second sensor may be a sensor that detects the target 2 in a direction different from that of the first sensor. For example, both the first sensor and the second sensor include a camera, the first sensor detects (eg, photographs) the object 2 from a first direction (eg, lateral), and the second sensor , the second sensor may detect (eg, photograph) the target 2 from a second direction (eg, front) different from the first direction.

Returning to the description of FIG. 1, the information processing device 30 includes a communication unit 31, a processing unit 32, and a storage unit 33. The communication unit 31 controls communication performed by the information processing device 30 with other devices. The processing unit 32 processes information. The processing executed by the processing unit 32 includes, for example, at least one of arithmetic processing, image processing, analysis processing, determination processing, inference processing, and control processing. The storage unit 33 stores at least one of information acquired by the information processing device 30 from another device, information used for processing executed by the processing unit 32, and information generated by processing executed by the processing unit 32. memorize The storage unit 33 stores, for example, candidate information D6 as information used for processing executed by the processing unit 32 . The candidate information D6 will be described later.

The processing unit 32 includes an acquisition unit 34, an identification unit 35, and a generation unit 36. The acquisition unit 34 acquires the detection result of the sensor 10 that detects the subject 2 . The acquisition unit 34 acquires the status information of the subject 2 from the terminal 20 associated with the subject 2 . The state information is information obtained from information different from the detection result of the sensor 10 and includes information representing the state of the target 2 . For example, in FIG. 2, status information includes information (eg, at work, on break, or other) entered by subject 2's superior (evaluator 3).

In this embodiment, the storage unit 33 stores candidate information D6. The candidate information D6 includes information that associates the candidate of the terminal 20 used for inputting the status information of the subject 2 with the subject 2 . The specifying unit 35 specifies the terminal 20 from candidates associated with the subject 2 in the candidate information stored in the storage unit 33 .

FIG. 3 is a diagram showing an example of association between sensors and terminals. In this embodiment, the subject 2 includes multiple subjects (2A, 2B). In the following description, when a plurality of subjects are not distinguished, they are referred to as subject 2, and when a plurality of subjects are distinguished, they are referred to as subject 2A and subject 2B. In this embodiment, the sensor 10 includes multiple sensors (10A, 10B). In the following description, when a plurality of sensors are not distinguished, they are referred to as sensor 10, and when they are distinguished, they are referred to as

sensors

10A and 10B. The sensor 10A corresponds to the subject 2A and detects the subject 2A. The sensor 10B corresponds to the subject 2B and detects the subject 2B.

In this embodiment, evaluators 3 include multiple evaluators (3A, 3B). In the following description, when a plurality of evaluators are not distinguished, they are referred to as evaluator 3, and when a plurality of evaluators are distinguished, they are referred to as

evaluators

3A and 3B. The evaluator 3A corresponds to each of the subject 2A and the subject 2B. The evaluator 3A is, for example, the superior of the subject 2A and the subject 2B. Evaluator 3B corresponds to subject 2B. The evaluator 3B is, for example, the superior of the subject 2B. For example, the subject 2B may be an employee belonging to two departments, and the

evaluators

3A and 3B may be managers of the respective departments to which the subject 2B belongs.

In this embodiment, the terminal 20 includes multiple terminals (20A, 20B). In the following description, when a plurality of terminals are not distinguished, they are referred to as a terminal 20, and when a plurality of terminals are distinguished, they are referred to as a terminal 20A and a terminal 20B. Terminal 20A is associated with evaluator 3A. The terminal 20A receives the input of the condition information of the subject 2 from the evaluator 3A. For example, the evaluator 3A judges the condition of the subject 2A corresponding to the evaluator 3A, and inputs the condition information representing the judgment result to the terminal 20A. For example, the evaluator 3A judges the condition of the subject 2B corresponding to the evaluator 3A, and inputs the condition information indicating the judgment result to the terminal 20A. Terminal 20B is associated with evaluator 3B. The terminal 20B receives the input of the condition information of the subject 2 from the evaluator 3B. For example, the evaluator 3B determines the state of the subject 2B corresponding to the evaluator 3B, and inputs state information indicating the determination result to the terminal 20B.

In FIG. 3, reference D6 represents candidate information that associates the terminal 20 candidate with the target 2. In FIG. The candidate information D6 is, for example, table data. Candidate information D6 includes, for example, items of sensor ID, target ID, terminal ID, and evaluator ID. The sensor ID includes information (eg, identification information) that identifies each sensor. Sensor IDs are assigned so as not to duplicate among a plurality of sensors. Here, it is assumed that the sensor ID of the sensor 10A is "S10A" and the sensor ID of the sensor 10B is "S10B". The target ID includes information (eg, identification information) specifying each target 2 . Target IDs are assigned so as not to duplicate among multiple targets 2 . Here, it is assumed that the target ID of the target person 2A is "U02A" and the target ID of the target 2B is "U02B". The terminal ID includes information identifying each terminal 20 . Terminal IDs are assigned so as not to duplicate among a plurality of terminals 20 . Here, it is assumed that the terminal ID of the terminal 20A is "T20A" and the terminal ID of the terminal 20B is "T20B". The evaluator ID includes information (eg, identification information) that identifies each evaluator 3 . Evaluator IDs are assigned so as not to overlap among multiple evaluators. Here, it is assumed that the evaluator ID of the evaluator 3A is "E01" and the evaluator ID of the evaluator 3B is "E02".

In the candidate information D6, the values of each item arranged on the same line are associated with each other. For example, the sensor ID "S10A" and the target ID "U02A" are arranged on the same line and have a correspondence relationship with each other. The sensor 10A whose sensor ID is "S10A" has a corresponding relationship with the target person 2A whose target ID is "U02A". The sensor 10A detects the target person 2A in correspondence. For example, the sensor 10A is installed in a place where the subject 2A is assumed to be in an office. The detection result output by the sensor 10A is treated as the result of detecting the subject 2A.

The sensor ID "S10A" and the terminal ID "T20A" are arranged on the same line and have a corresponding relationship. The sensor 10A whose sensor ID is "S10A" has a corresponding relationship with the terminal 20A whose terminal ID is "T20A". The detection result of the sensor 10A is associated with information output from the corresponding terminal 20A and used as teacher data or its original data. The sensor ID "S10A" and the evaluator ID "E03A" are arranged on the same line and have a correspondence relationship with each other. The sensor 10A whose sensor ID is "S10A" has a corresponding relationship with the evaluator 3A whose evaluator ID is "E03A". The detection result of the sensor 10A is associated with state information input by the evaluator 3A, which has a corresponding relationship, and is used as teacher data or its original data.

The target ID "U02A" and the terminal ID "T20A" are arranged on the same line and have a corresponding relationship. The target person 2A whose target ID is "U02A" has a corresponding relationship with the terminal 20A whose terminal ID is "T20A". The status information of the subject 2A is output from the corresponding terminal 20A. At least part of the information output from the terminal 20A is treated as the subject's 2A status information. The subject ID "U02A" and the evaluator ID "E03A" are arranged on the same line and have a correspondence relationship with each other. The subject 2A whose subject ID is "U02A" has a corresponding relationship with the evaluator 3A whose evaluator ID is "E03A". The status information of the subject 2A is input by the evaluator 3A who has a corresponding relationship.

The terminal ID "T20A" and the evaluator ID "E03A" are arranged on the same line and have a corresponding relationship. Terminal 20A whose terminal ID is "T20A" is in correspondence with evaluator 3A whose evaluator ID is "E03A". The status information input to the terminal 20A is treated as input by the evaluator 3A.

The sensor 10A, for example, information including a value obtained by detection (arbitrarily referred to as a sensor value) or a processing result of processing the sensor value, time information indicating the timing at which detection was performed (for example, a time stamp), It provides the information processing device 30 with the detection result including the sensor ID of its own device. The evaluator 3A judges the state of the subject 2A in correspondence. The evaluator 3A inputs information representing the condition of the subject 2A to the corresponding terminal 20A. The terminal 20A, for example, stores state information including information including an input value or a processing result of processing the input value, time information (for example, time stamp) indicating the timing of the input, and the terminal ID of its own device. provided to the processing device 30;

The evaluator 3A judges the state of the subject 2A who has a corresponding relationship with himself. The evaluator 3A inputs information representing the condition of the subject 2A to the terminal 20A, which has a corresponding relationship with him/herself. The terminal 20A provides the information processing apparatus 30 with state information including, for example, state information, time information (for example, time stamp) indicating input timing, and the terminal ID of the terminal itself. The state information output by the terminal 20A may include one or both of state information input to the terminal 20A and state information obtained by processing information (eg, input values) input to the terminal 20A.

Note that the sensor 10A may detect at least one of a target other than the target person 2A, a person other than the target, and an object other than the target. For example, the sensor 10A may include a camera installed in the space where the subject 2A works. The sensor 10A or a device external to the sensor 10 may analyze the image captured by the camera and detect the subject 2A appearing in the image. The sensor 10A or the above-mentioned external device may perform image analysis on an image showing a plurality of subjects 2 and identify the subject 2A appearing in the image. When the sensor 10A or a device external to the sensor 10 identifies the target person 2A from the detection result, the target ID does not have to be associated with the sensor ID. For example, the target ID may be associated with feature information representing the external features of the target person 2A. The sensor 10A or the external device may identify the target by comparing the detection result of the sensor 10A and the feature information, and associate the detection result of the specified target with the target ID associated with the feature information.

Note that the sensor 10A may detect a plurality of targets 2 in chronological order according to a predetermined schedule. In this case, the target 2 detected by the sensor 10A is specified using the timing at which the detection was performed and the above schedule. Further, the sensor 10A may detect a plurality of targets 2 by detecting the target 2 in a predetermined direction and switching the predetermined direction. In this case, the target 2 detected by the sensor 10A is specified using information that associates the direction of detection with the target 2 in advance, and the direction of detection that has been performed.

It should be noted that in the example of FIG. In such a case, the terminal 20A may output the information designating the subject 2 in the form of an image or voice when accepting the input of the status information. For example, the terminal 20A may display information specifying the target person 2 (for example, "Mr. A") as the information specifying the target person 2 as shown in FIG. Further, when receiving input of state information, the terminal 20A may receive input of information specifying the subject 2 corresponding to the input state information. For example, the evaluator 3A may input information specifying the subject 2B (eg, name of the subject 2B, subject ID) and state information about the subject 2B into the terminal 20A. In this case, the terminal 20A may provide the information processing device 30 with information including information specifying the subject 2B and state information.

The evaluator 3A may be a different person from the subject 2A, or may be the same person as the subject 2A. For example, the subject 2A may be the same person as the evaluator 3A, and the terminal 20A may be a terminal possessed by the subject 2A. In this case, the subject 2A may input his/her status information to the terminal 20A. If the subject 2A is the same person as the evaluator 3A, the evaluator ID may not be associated with the subject ID. For example, the state information output from the terminal 20A may be associated with the detection result of the sensor 10A associated with the terminal 20A and used as teacher data.

Returning to the description of FIG. 1, the specifying unit 35 specifies the correspondence relationship between the information output from the sensor 10 and the information output from the terminal 20. For example, the specifying unit 35 specifies the state information acquired from the terminal 20 corresponding to the detection result acquired from the sensor 10 by referring to the candidate information D6. Here, with reference to FIGS. 1 and 3, it is assumed that the sensor ID included in the detection result obtained from the sensor 10 is "S10A." The identifying unit 35 identifies the sensor ID (here, “S10A”) from the information acquired from the sensor 10 . The identifying unit 35 identifies the terminal ID (here, "T20A") corresponding to "S10A" in the candidate information D6. The identifying unit 35 identifies the state information acquired from the terminal 20A whose terminal ID is "T20A" as state information corresponding to the detection result of the sensor 10A whose sensor ID is "S10A".

Note that the identifying unit 35 may identify the detection result obtained from the sensor 10 corresponding to the state information obtained from the terminal 20 . Here, it is assumed that the evaluator 3 inputs information specifying the subject 2 (eg, name, subject ID) when inputting the status information of the subject 2 . The terminal 20 is the terminal 20A in FIG. 3, and when outputting the state information, the terminal ID of its own device (here, "T20A" and the information specifying the target 2 input by the evaluator 3 ( For example, the target ID “U02B”) is output from the identification unit 35 based on the information acquired from the terminal 20A. The identifying unit 35 identifies the sensor ID (here, "S10B") associated with "T20A" and "S02B" in the candidate information D6.The identifying unit 35 identifies the sensor ID "S10B". is the state information about "U02B" acquired from the terminal 20A whose terminal ID is "T20A".

It should be noted that the information processing device 30 may be the above-described external device and identify the target 2 from the detection result output from the sensor 10 . For example, the sensor 10A may include a camera that captures the subject 2A, and the information processing device 30 (eg, the identification unit 35) may identify the subject 2A by analyzing the image captured by the camera of the sensor 10A. The identifying unit 35 identifies the terminal ID (here, “T20A”) associated in the candidate information D6 with the target ID (here, “U02A”) of the subject 2A identified from the detection result of the sensor 10A. may The identifying unit 35 may associate the detection result of the sensor 10A with the terminal ID (here, "T20A") identified using the detection result of the sensor 10A. When the external device specifies the target 2 from the detection result of the sensor 10, the sensor ID may not be associated with the target ID in the candidate information D6, and may not be included in the candidate information D6.

The generation unit 36 in FIG. 1 generates data by associating the detection result of the sensor 10 with the state information. The detection result may be a sensor value of the sensor 10 or a value obtained by processing the sensor value of the sensor 10 (for example, a value derived as a detection result). For example, the generation unit 36 uses the detection result acquired by the acquisition unit 34 to derive the sensor detection result at the time when the state information was obtained, and associates the derived sensor detection result with the state information to generate data. Generate. The state information may be information output by the terminal 20, or may be information obtained by processing this information (for example, information derived as state information). The generating unit 36 generates data by associating, for example, the detection result derived from the sensor value of the sensor 10 with the state information specified by the specifying unit 35 as the state information corresponding to this detection result.

FIG. 4 is a diagram showing an example of the process of deriving the sensor detection results corresponding to the state information. The generation unit 36, for example, derives the timing at which the state information is acquired, and derives (eg, estimates) the sensor value at this timing. For example, the generating unit 36 uses the time indicated by the time stamp output by the terminal 20 together with the state information as the timing at which the state information was acquired. For example, the generation unit 36 derives the timing at which the state information was acquired by the process of specifying the value of the time stamp corresponding to the state information.

The sensor 10 performs detection operations at, for example, a predetermined sampling frequency. The sensor 10 outputs a sensor value, for example, at predetermined time intervals. In FIG. 4, symbol t is the time derived by the generator 36 as the time when the state information was obtained. The time t is information obtained from a time stamp output by the terminal 20, for example. The generation unit 36 derives the sensor value as the detection result corresponding to the state information by the process of specifying the sensor value closest to the time t from the time-series data of the sensor value.

Note that the process of deriving the detection result corresponding to the state information may be different from the above example. For example, the generation unit 36 may derive (eg, calculate) the detection result by statistically processing a plurality of sensor values in a predetermined period T including time t. For example, the generation unit 36 may calculate an average value of multiple sensor values in a predetermined period T as the detection result. This average value may be an arithmetic mean value or a geometric mean value of the sensor values in the predetermined period T. FIG. For example, the average value may be a geometric mean value using a weight according to the time between the time when the sensor value is obtained and the time t when the state information is obtained. Further, the generation unit 36 may calculate the detection result by approximation or interpolation using a plurality of sensor values in the predetermined period T. The generation unit 36 may generate teacher data that associates one detection result (eg, sensor value) obtained from the sensor 10 with one piece of state information. The generation unit 36 may generate teacher data that associates one piece of information (eg, a waveform of sensor values with respect to time) that is a set of a plurality of detection results obtained from the sensor 10 and one piece of state information. .

It should be noted that the terminal 20 does not have to output the time stamp corresponding to the timing at which the state information was acquired. In this case, the generator 36 may derive the time at which the information processing device 30 acquires the state information as the timing at which the state information was acquired. For example, the generation unit 36 may derive the time when the communication unit 31 received the state information as the timing at which the state information was acquired. The information processing device 30 (eg, the processing unit 32) may transmit information requesting transmission of state information to the terminal 20 (arbitrarily referred to as a state information request). The generation unit 36 may derive the timing at which the state information was acquired using the time when the communication unit 31 transmitted the state information request to the terminal 20 . For example, the generator 36 may derive the time at which the state information request was transmitted as the timing at which the state information was acquired. As the timing at which the state information is acquired, the generating unit 36 selects the time between the first time when the communication unit 31 transmitted the state information request and the second time when the communication unit 31 received the state information (for example, , the average value of the first time and the second time) may be derived. The generation unit 36 may derive the timing at which the state information is acquired using one or both of the first time and the second time and an estimated value of the time required for communication.

Note that the sensor 10 may detect the target 2 periodically, or may detect the target 2 irregularly. For example, the sensor 10 may detect the target 2 upon receiving information requesting detection of the target 2 (hereinafter referred to as a detection request). The sensor 10 may output a sensor value or information obtained by processing the sensor value as a response to the detection request. The device that outputs the detection request may be the information processing device 30, a device different from the information processing device 30 in the information processing system 1 (eg, the information processing device 50 for dictation), or an external device of the information processing system 1. good.

Returning to the description of FIG. 1, the information processing device 30 provides the data generated by the generation unit 36 to the information processing device that executes machine learning. In this embodiment, the information processing device 40 includes a learning unit 41 that executes machine learning, and the information processing device 30 provides the information processing device 40 with teacher data for machine learning or data that is the source of the teacher data. .

FIG. 5 is a diagram showing an example of processing for executing machine learning. In this example, model M is an inference model generated by deep learning. For example, the model M receives the detection result of the sensor 10 and outputs the estimation result of the state of the object 2 . The model M includes an input layer M1, an intermediate layer M2 and an output layer M3. The input layer M1 is a layer to which original data for inference is input. The output layer M3 is a layer that outputs data indicating an inference result. The intermediate layer M2 is a layer arranged between the input layer M1 and the output layer M3. The number of intermediate layers M2 is arbitrary, and may be one layer or multiple layers.

The learning unit 41 inputs the detection result of the sensor 10 in the teacher data to the input layer M1. A value input to the input layer M1 propagates to the output layer M3 via the intermediate layer M2. The model M includes parameters (eg, coupling coefficients, biases) for propagating values from the input layer M1 to the output layer M3. The learning unit 41 optimizes the parameters so that the output data output from the output layer M3 approaches the data representing the state information when the input data in the teacher data is input to the input layer M1.

The machine learning performed by the learning unit 41 is arbitrary and does not have to be deep learning. The machine learning method may be a neural network that does not include the intermediate layer M2, or may be a method other than the neural network. The information processing device 30 may provide the learning unit 41 with teacher data obtained from the detection results and state information of a plurality of targets 2 as teacher data for one inference model. In this case, the obtained inference model can be used, for example, as an inference model commonly used for a plurality of targets 2 (arbitrarily referred to as a shared model).　　

The information processing device 30 may provide the learning unit 41 with teacher data obtained from the detection results and state information of a specific target 2 (for example, the target 2A) as teacher data for one inference model. . In this case, the obtained inference model can be used, for example, as an inference model (arbitrarily referred to as an individual model) for estimating the state of a specific subject 2 (eg, subject 2A). The number of targets 2 included in the specific target 2 may be one, or two or more. A plurality of targets 2 may be a predetermined set, and a particular target 2 may be a part (eg, a subset) of the predetermined set. For example, the predetermined set may be a set of employees belonging to a specific company, and the subset may be a set of employees belonging to a predetermined department within the specific company.

Note that the learning unit 41 generates a shared model when the amount of teacher data accumulated for the specific target 2 is less than a predetermined amount, and generates an individual model when the amount of teacher data accumulated for the specific target 2 is equal to or greater than the predetermined amount. model may be generated. For example, the learning unit 41 generates a common model using the detection results obtained by the sensor 10 in a first period, and uses the detection results obtained by the sensor 10 in a second period longer than the first period. Individual models may be generated.

Returning to the description of FIG. 1 , the learning unit 41 provides the generated inference model to the information processing device that estimates the state of the target 2 using the detection result of the sensor 10 . For example, the information processing device 40 provides the inference model generated by the learning unit 41 to the information processing device 50 . The information processing device 50 includes a processing unit realized by a general-purpose processor or the like. The processing unit of the information processing device 50 configures the estimating unit 51 by executing the computation defined by the inference model. The estimating unit 51 is, for example, an AI in which an inference model generated by machine learning by the learning unit 41 is implemented.

In the present embodiment, the information processing device 30 provides the information processing device 50 with the detection result obtained by the obtaining unit 34 from the sensor 10 . The estimation unit 51 of the information processing device 50 estimates the state of the target 2 by performing an operation defined by the inference model on the detection result acquired by the acquisition unit 34 . The information processing device 50 provides, for example, the application 60 with the state of the target 2 estimated by the estimation unit 51 (arbitrarily referred to as an estimation result of the estimation unit 51).

The application 60 is an application that processes the estimation result of the estimation unit 51 . The application 60 uses the estimation result of the estimation unit 51, for example, to execute a process of creating a log indicating the work status of the subject 2. FIG. The processing executed by the application 60 is arbitrary, and may include processing for analyzing the estimation result of the estimation unit 51 . For example, the application 60 may estimate the time during which the subject 2 was working intensively, or calculate an index indicating the work status of the subject 2 (eg, efficiency, degree of concentration, amount of work). good too. The application 60 may perform processing for outputting (eg, displaying) one or both of the estimation result of the estimation unit 51 and the processing result obtained by processing the estimation result of the estimation unit 51 .

The application 60 may be installed in at least one of the information processing devices included in the information processing system 1 . For example, the application 60 may be installed in the information processing device 30 . The application 60 may be installed in at least one information processing device external to the information processing system 1 . The number of applications to which the estimation result of the estimation unit 51 is provided may be one or plural. The application 60 may include a web application, an application on the cloud, or an on-premises application.

When the estimation results of the estimation unit 51 are provided to multiple applications, the uses of the estimation results may be the same or different for the multiple applications. For example, a first application among the plurality of applications may manage the work status of the subject 2 and a second application among the plurality of applications may manage the health status of the subject 2 .

Next, an information processing method according to the embodiment will be described based on the information processing system 1 described above. FIG. 6 is a diagram showing an information processing method according to the first embodiment. FIG. 1 to FIG. 5 are referred to appropriately for each part of the information processing system 1 .

The sensor 10 shown in FIG. 1 detects the target 2. The sensor 10 transmits the detection result of detecting the target 2 . In step S1 in FIG. 6 , the acquisition unit 34 acquires the detection result from the sensor 10 . For example, the acquisition unit 34 controls the communication unit 31 to receive the detection result transmitted by the sensor 10 . The evaluator 3 judges the condition of the subject 2 and inputs information representing the judgment result into the terminal 20 as condition information. In step S2 , the acquisition unit 34 acquires state information from the terminal 20 . For example, the acquisition unit 34 controls the communication unit 31 to receive the detection result transmitted by the terminal 20 .

In step S3, the generating unit 36 generates data used for machine learning. Step S3 includes, for example, steps S4 to S6. In step S4, the generator 36 derives the timing at which the state information was acquired. In step S5, the generator 36 derives the detection result at the timing derived in step S4. In step S6, the generation unit 36 associates the state information acquired in step S2 with the detection result derived in step S5. For example, the generation unit 36 sets the detection result and the state information as input teacher data for the model M (see FIG. 5) and the state information as the output teacher data for the model M (see FIG. 5). The data obtained by

In step S7, the information processing device 30 provides data to the learning unit 41. For example, the communication unit 31 of the information processing device 30 transmits data including teacher data generated by the generation unit 36 to the information processing device 40 including the learning unit 41 . In step S8, the learning unit 41 uses the data provided in step S7 to generate a model through machine learning. In step S9 , the learning unit 41 provides the model generated in step S9 to the estimation unit 51 . For example, the information processing device 40 including the learning unit 41 transmits data representing the inference model generated by the learning unit 41 to the information processing device 40 including the estimation unit 51 .

In step S10, the estimation unit 51 processes the detection result of the sensor 10 using the model provided in step S9 to estimate the state of the object. For example, the communication unit 31 of the information processing device 30 transmits the sensor detection result to the information processing device 50, and the estimation unit 51 estimates the target state using the sensor detection result received by the information processing device 50. do. The estimating unit 51 inputs, for example, the model M (see FIG. 5) in which the result of machine learning is reflected to the input layer M1, and derives the data output from the output layer M3 as the estimation result. In step S11, the estimation unit 51 provides the application 60 with information indicating the target state estimated in step S10. For example, the information processing device 50 transmits data indicating the estimation result of the estimation unit 51 to the information processing device on which the application 60 is installed.

In the information processing method shown in FIG. 6, the process of step S2 may be performed before at least part of the process of step S1, or may be performed in parallel with at least part of the process of step S1. may In step S3, the generation unit 36 may derive the timing at which the detection result of the sensor 10 is acquired, and derive the state information at the derived timing. The information processing system 1 may repeatedly execute the processing from step S1 to step S3 (referred to as teacher generation processing as appropriate) and accumulate the teacher data generated in step S3. The information processing system 1 may execute the processes after step S7 when the data amount of the accumulated teacher data reaches or exceeds a predetermined amount.

Note that the process of step S8 (arbitrarily referred to as model generation process) may be repeatedly executed. For example, a first teacher generation process may generate a first model, and a second teacher generation process executed after the first teacher generation process may generate a second model. The second model may be generated using teacher data having a larger amount of data than the first model. For example, the first model may be generated using first teacher data, and the second model may be generated using second teacher data accumulated after the generation of the first teacher data. A second model may be generated using the first teacher data and the second teacher data. The second teacher data may be generated by adding (for example, feedback) the error of the estimation result using the first model.

It should be noted that the second training data may differ from the first training data in a set of objects corresponding to the original data when generated. For example, the first teacher data is teacher data based on the detection results and state information about the plurality of targets 2, and the second teacher data is a part of the plurality of targets 2 (e.g., a target person 2A) may be teacher data based only on detection results and state information. The second model may differ from the first model in terms of the state estimation target. For example, the first model is used for estimating the status of employees belonging to a given company, and the second model is used for estimating the status of employees belonging to a given department within a given company. may The second model may not be generated. The process of step S10 and the process of step S11 may be repeatedly performed using the first model.

The information processing system 1 in the above-described embodiment associates the detection result of the sensor 10 with information different from the detection result of the sensor 10 to generate an inference model. The state of the target 2 can be estimated by the estimation unit 51 . Such an information processing system 1 contributes to grasping the state of the subject 2 even if, for example, the evaluation by the evaluator 3 is simplified or omitted after the inference model is generated. The information processing system 1 can facilitate, for example, associating different information about the target 2, and contributes to facilitating multifaceted analysis of the target 2. FIG. The information processing system 1 can, for example, express the state of the target 2 as data according to a certain rule based on an inference model, and contributes to objective evaluation of the state of the target 2 .

In the above embodiment, the first application example has been described in which the subject 2 is an employee working in an office and the evaluator 3 is the employee's boss. Application examples of the information processing system 1 will be described below, but the scope of application of the information processing system 1 is not limited to the examples described above and the examples described later.

In the second application example, subject 2 is a worker at a construction site, and evaluator 3 is a site supervisor. The sensor 10 detects the temperature of the target 2 (eg, body temperature), the humidity in the vicinity of the target 2 (eg, sweating state), the movement of the target 2 (eg, hand movement, posture), and the vital activity of the target 2 (eg, , pulse, blood pressure). The evaluator 3 evaluates the health condition of the subject 2 based on, for example, the complexion and the like, and inputs the evaluation result into the terminal 20 as condition information. The information processing system 1 uses the detection result of the sensor 10 and state information to generate an inference model for estimating the health condition of the worker, for example. For example, the estimation unit 51 acquires the detection result of the sensor 10 in real time, and estimates the worker's health condition (eg, physical condition, degree of fatigue) in real time using an inference model.

The application 60 may use the estimation result of the estimation unit 51 to determine whether the worker's health condition is suitable for continuing work. For example, the application 60 estimates the health condition of the worker after a predetermined period of time has elapsed using the history of estimation results regarding the health condition of the worker. For example, the application 60 estimates the health condition of the worker by calculating an index indicating the health condition of the worker after a predetermined period of time has passed. The application 60 compares the health condition indicator with a threshold value to determine whether the worker's health condition is suitable for continuing work. For example, the worse the health condition, the lower the index, and the application 60 determines that the health condition of the worker is not suitable for continuing work when the calculated index is less than the threshold.

When the application 60 determines that the health condition of the worker is not suitable for continuing the work, the application 60 may output the determination result. For example, the application 60 is provided in the information processing device 30, and the information processing device 30 sends a warning about the worker's health condition to the terminal 20 (eg, the site supervisor's terminal) associated with the target 2 who is the worker, or It may be provided to other devices. The site supervisor may make the worker stop the work or take a rest, referring to the above warning. Thus, the information processing system 1 can be used for health management of workers. When the subject 2 is human, the position of the subject 2 and the relationship between the subject 2 and the evaluator 3 are not limited to the above examples. For example, the subject 2 may be a care recipient and the evaluator 3 may be the subject 2's caregiver. The subject 2 may be a protected person and the evaluator 3 may be the subject 2's guardian. The subject 2 may be a patient and the evaluator 3 may be a medical practitioner in charge of the subject 2 .

It should be noted that the target 2 may be an animal other than humans. In a third application, the subject 2 is a domesticated animal and the evaluator 3 is the keeper of this animal. The animal may be an animal raised in a zoo or the like, or may be a pet raised by an individual. The sensor 10 detects, for example, information about vital activity of the animal, which is the subject 2, and the evaluator 3 inputs the health condition of the animal, which is the subject 2, into the terminal 20 as condition information. The information processing system 1 generates an inference model by the same processing as in the second application example, and uses this inference model to estimate the state of health of the animal. Such an information processing system 1 can be used for health management of the subject 2, as in the second application example.

It should be noted that the subject 2 may be an animal raised in the livestock industry. The information processing system 1 may estimate the growth state as the state of the animal that is the target 2 . The estimation result of the growth state may be used, for example, as a reference for adjusting breeding conditions, or as a reference for deciding shipping of livestock products (eg, shipping amount, shipping time). Object 2 may be an organism other than an animal. When the target 2 is an organism other than an animal (eg, a plant), the information processing system 1 determines the state of the target 2 (eg, health condition, growth) as in the case where the target 2 is an animal as described above. Contribute to managing (e.g., understanding).

In addition, the target 2 may be an inanimate object (eg, machine, article). In the fourth application example, the target 2 is a machine, and the evaluator 3 is a manager (eg, operator, maintenance person, inspection person) of the target 2 which is the machine. The sensor 10 detects, for example, information (eg, temperature, vibration) indicating the operating state of the machine that is the object 2 . The evaluator 3 inspects the object 2, which is, for example, a machine, and inputs information indicating the operating state to the terminal 20 as state information. The operating state may include information as to whether the operation is normal, information indicating the degree of certainty that a failure has occurred, and information indicating whether inspection or maintenance is necessary. The information processing system 1 uses the detection result of the sensor 10 and state information to generate an inference model for estimating the operating state of the machine, for example. The estimating unit 51, for example, acquires the detection result of the sensor 10 and estimates the operating state of the machine using an inference model.

The application 60 uses the estimation result of the estimation unit 51 to output (eg, notify) information about the machine when the estimated operating state satisfies a predetermined condition. For example, the sensor 10 detects vibrations of the target 2, which is a machine, and the estimation unit 51 estimates the probability that the target 2 is out of order (arbitrarily referred to as failure probability). The application 60 is provided in the information processing device 30 and outputs a warning prompting inspection of the target 2 when the failure probability estimated by the estimation unit 51 is equal to or greater than a predetermined value. The information processing device 30 provides the warning output by the application 60 to the terminal 20 associated with the target 2 (for example, the terminal of the person in charge of inspection).

[Second embodiment]
Next, a second embodiment will be described. FIG. 7 is a diagram showing an information processing system according to the second embodiment. In this embodiment, the same reference numerals are given to the same configurations as those of the above-described embodiment, and the description thereof is omitted or simplified. In this embodiment, the information processing device 30 includes a control section 37 . The control unit 37 requests the predetermined terminal specified by the specifying unit 35 to provide the state information. The acquisition unit 34 acquires state information from the terminal 20 as a response to the request.

In this embodiment, the control unit 37 requests the terminal 20 to provide state information when the detection result of the sensor 10 satisfies a predetermined condition. The control unit 37 determines whether the detection result obtained from the sensor 10 by the obtaining unit 34 satisfies a predetermined condition. For example, the sensor 10 is a wearable device (eg, smart watch) worn by the subject 2 and detects information indicating the vital activity of the subject 2 . For example, it is assumed that the sensor 10 is worn on the hand of the subject 2 and detects acceleration. The detection results of the sensor 10 show different values or waveforms, for example, when the subject 2 is working (for example, operating a computer) and when the subject 2 is resting. Also, the sensor 10 detects at least one of the subject's 2 heart rate, blood pressure, body temperature, and perspiration. In this case, the detection result of the sensor 10 shows different values or waveforms depending on, for example, the subject's 2 stress, tension, and concentration level. For example, the control unit 37 performs frequency analysis on the waveform of the sensor value at predetermined time intervals to calculate the characteristic frequency of the waveform. For example, when the difference between the characteristic frequency in the first period and the characteristic frequency in the second period following the first period exceeds a threshold, the control unit 37 determines that the predetermined condition is satisfied.

Note that the control unit 37 may perform the above determination by comparing one value (eg, sensor value, moving average value of sensor values with respect to time) indicating the detection result of the sensor 10 with a threshold value. The predetermined condition may be set to the same condition for a plurality of targets 2 or may be set for each target 2 . The predetermined conditions for at least one target 2 among the plurality of targets 2 may differ from the predetermined conditions for other targets. A plurality of targets 2 are divided into a plurality of groups, and the predetermined condition may be set for each group. For example, a plurality of target persons 2 may be divided into a plurality of groups (eg, departments), and predetermined conditions may be set according to the work content of each group.

As described above, the control unit 37 monitors the detection result of the sensor 10, for example. When the control unit 37 determines that the detection result of the sensor 10 satisfies a predetermined condition, it generates a state information request.
The state information request is information (eg, command) that causes the terminal 20 to provide (eg, transmit) state information. The information processing device 30 , for example, designates the target 2 and requests provision of state information about the target 2 . For example, the status information request includes information identifying the target 2 for which status information is requested (eg, the target ID shown in FIG. 3). For example, the specifying unit 35 refers to the candidate information D6 stored in the storage unit 33, and specifies the target ID corresponding to the sensor ID of the sensor 10 that provided the detection result used in the determination. The control unit 37 generates a state information request including, for example, the target ID specified by the specifying unit 35 and a message requesting state information.

The information processing device 30 transmits a state information request to the terminal 20 to which the state information is requested. For example, the identifying unit 35 identifies the terminal 20 corresponding to the sensor 10 that provided the detection result used in the above determination. For example, the specifying unit 35 refers to the candidate information D6 stored in the storage unit 33, and specifies the terminal ID corresponding to the sensor ID (see FIG. 3) of the sensor 10 that provided the detection result used in the determination. do. The control unit 37 causes the communication unit 31 to transmit the state information request to the terminal 20 specified by the specifying unit 35 . The communication unit 31 causes the storage unit 33 to store information (eg, time stamp) indicating the timing of transmitting the state information request.

When the terminal 20 receives the state information request, it outputs a notification (eg, image, voice) prompting the user to enter the state information. For example, the terminal 20 displays an image for accepting input of state information on the display unit 21 as shown in FIG. This image includes, for example, information identifying the subject 2 (eg, the name of the subject 2). Status information is input to the terminal 20 by the evaluator 3 , and the terminal 20 provides the input status information to the information processing device 30 . The terminal 20, for example, includes information indicating the timing of transmitting the state information (eg, time stamp) in the state information or transmits it together with the state information.

The information processing device 30 receives the state information transmitted from the terminal 20 . For example, the generation unit 36 of the information processing device 30 uses one or both of the first timing at which the state information request is transmitted and the second timing at which the state information is transmitted so that the state information is transmitted by the terminal 20. Deriving (eg, estimating) the obtained timing. For example, the generation unit 36 calculates the time between the time indicating the first timing and the time indicating the second timing as the timing at which the terminal 20 acquires the state information. The generator 36 derives the detection result of the sensor 10 at the derived timing and generates teacher data.

The information processing device 30 provides teacher data to the information processing device 40 as described in the first embodiment. The learning unit 41 of the information processing device 40 executes machine learning using teacher data to generate a model. The information processing device 40 provides the model to the information processing device 50 . The estimation unit 51 of the information processing device 50 processes the detection result of the sensor 10 using a model to estimate the state of the target 2 . The information processing device 50 provides the estimation result of the state of the target 2 to the application 60 . For example, the information processing system 1 according to the present embodiment can selectively cause the evaluator 3 to evaluate the subject 2 in a situation where it is estimated that the state of the subject 2 has changed. Such an information processing system 1 contributes to, for example, reducing the load required for the process of acquiring state information when the state of the target 2 is substantially the same.

[Third embodiment]
Next, a third embodiment will be described. FIG. 8 is a diagram showing an information processing system according to the third embodiment. In this embodiment, the same reference numerals are given to the same configurations as those of the above-described embodiment, and the description thereof will be omitted or simplified. In the present embodiment, the identifying unit 35 of the information processing system 1 identifies a predetermined terminal from among the terminals 20 existing within a predetermined range with respect to the detection target position of the sensor 10 . The generation unit 36 generates teacher data using the state information provided from the predetermined terminal identified by the identification unit 35 and the detection result of the sensor 10 .

The storage unit 33 of the information processing device 30 stores, for example, the location information D7, and the specifying unit 35 uses the location information D7 to specify the location of the predetermined terminal. The position information D7 includes, for example, information on the position of the detection target of each sensor 10 (arbitrarily referred to as sensor position information). The sensor position information includes, for example, information indicating the position where each sensor 10 is installed and information indicating the detectable range of each sensor 10 . Sensor location information may include information indicating the location of the target 2 associated with the sensor 10 . For example, the sensor position information may be the position information of the seat of the subject 2 in the office. When the sensor 10 is a camera, the sensor position information may be the central position of the field of view (angle of view) when the sensor 10 takes an image. The sensor position information may be given as a fixed value when the movement of the sensor 10 is not assumed (for example, when the sensor 10 is a fixed point camera).

The sensor position information may be given as a variable value when the sensor 10 is assumed to move (for example, when the sensor 10 is a wearable device). For example, a device (eg, wearable device, mobile device) including the sensor 10 detects its own position and provides the detection result to the information processing device 30 . Any method may be used to detect the position of the sensor 10. For example, a method using GPS, a method using signals (e.g., beacon signals) received from a plurality of other devices, or other methods may be used. good. A device that detects the position of the sensor 10 may be a device including the sensor 10 or a device different from this device (for example, a device external to the sensor 10). When the sensor 10 is assumed to move, the information processing device 30 acquires the position information of the sensor 10 from a device including the sensor 10 or a device different from this device, and stores the information in the storage unit 33 using the acquired information. may update the sensor position information stored by .

The position information D7 includes, for example, the position information of the terminal 20 (arbitrarily referred to as terminal position information). The terminal location information may be given as a fixed value when the terminal 20 is not expected to move (for example, when the terminal 20 is a stationary computer). The terminal location information may include information indicating the location where the terminal 20 is installed. The terminal location information may include information indicating the location of the evaluator 3 associated with the terminal 20 . For example, the terminal 20 may be installed at the seat of the evaluator 3 in the office, and the terminal position information may include the position information of the seat of the evaluator 3 .

The terminal location information may be given as a variable value when the terminal 20 is expected to move (for example, when the terminal 20 is a smartphone). For example, the terminal 20 (eg, smart phone) detects its own position and provides the detection result to the information processing device 30 . Any method may be used to detect the position of the terminal 20. For example, a method using GPS, a method using signals (e.g., beacon signals) received from a plurality of other devices, or other methods may be used. good. A device that detects the position of the terminal 20 may be the terminal 20 or a device different from the terminal 20 (for example, a device external to the terminal 20). When the terminal 20 is expected to move, the information processing device 30 acquires the position information of the terminal 20 from the terminal 20 or a device different from the terminal 20, and the storage unit 33 stores the acquired information. Terminal location information may be updated.

The specifying unit 35 refers to the position information D7 stored in the storage unit 33 and specifies (eg, determines) the terminal 20 to which the state information of the target 2 is input. For example, the specifying unit 35 sets the predetermined range R using the sensor position information included in the position information D7. The predetermined range R is, for example, a range in which the distance from the detection target position of the sensor 10 is equal to or less than a predetermined value. In FIG. 8 , the position of the detection target of the sensor 10 is the position of the object 2 . The predetermined range R is, for example, an area inside a circle (eg, geofence) centered on the position of the target 2 . The radius of this circle corresponds to the predetermined value. The predetermined range R is set, for example, to a range in which the evaluator 3 can directly recognize the target 2 . For example, when it is assumed that the evaluator 3 visually recognizes the subject 2, the radius of the predetermined range R is set in advance to a distance (eg, several meters) at which a person can recognize the person with normal eyesight. be done.

The specifying unit 35 determines whether or not the terminal 20 exists inside the predetermined range R using the terminal location information included in the location information D7. The specifying unit 35 sets, among the terminals whose terminal location information is included in the location information D7, terminals existing inside a predetermined range R (

terminals

20A and 20B in FIG. 8) as predetermined terminal candidates. The specifying unit 35 excludes a terminal (the terminal 25 in FIG. 8) existing outside the predetermined range R from among the terminals whose terminal position information is included in the position information D7 from the predetermined terminal candidates.

The generating unit 36 generates teacher data by associating the state information provided by the terminal set as a predetermined terminal candidate by the specifying unit 35 with the detection result of the sensor 10 that detects the target 2 . For example, the information processing device 30 transmits a state information request to at least one of the predetermined terminal candidates. For example, the control unit 37 shown in FIG. 7 selects at least one terminal 20 from the predetermined terminal candidates as the predetermined terminal. The control unit 37 may randomly select a predetermined terminal from the predetermined terminal candidates. The control unit 37 may select a predetermined terminal from the predetermined terminal candidates according to a predetermined rule. For example, multiple terminals are registered in advance in the information processing device 30 . A priority order may be set in advance for a plurality of registered terminals, and the control unit 37 may select a predetermined terminal according to the priority order. For example, the predetermined terminal candidates include the terminal 20A of the boss of the subject 2 and the terminal 20B of the colleague of the subject 2, and the control unit 37 prioritizes the terminal 20A of the boss over the terminal 20B of the colleague. You may choose to

For example, when the information processing device 30 acquires the detection result of the sensor 10, it provides a state information request including information specifying the target 2 corresponding to this sensor 10 to a predetermined terminal. In this case, the terminal 20 does not have to be pre-associated with the target 2, and the generator 36 associates the detection result of the sensor 10 with the state information acquired as a response to the state information request to generate teacher data. can. The information specifying the target 2 may be associated in advance with the information specifying the sensor 10 or may be derived using the detection result of the sensor 10 . For example, the sensor 10 includes a camera, and the information processing device 30 extracts the features of the target 2 appearing in the image captured by the sensor 10, compares the information of the extracted features with pre-registered information, and extracts the feature of the target. 2 may be specified. In this case, the information specifying the target 2 may not be associated in advance with the information specifying the sensor 10 .

[Fourth embodiment]
A fourth embodiment will be described. FIG. 9 is a diagram showing an information processing system according to the fourth embodiment. In this embodiment, the same reference numerals are given to the same configurations as those of the above-described embodiment, and the description thereof will be omitted or simplified. In this embodiment, the information processing system 1 generates teacher data using the detection result of the sensor 11 and the detection result of the sensor 12 different from the sensor 11 . The sensor 10 is, for example, a sensor similar to the sensor 10 shown in FIG.

The sensor 12 differs from the sensor 11 in the type of information it detects, for example. For example, the sensor 11 may be a sensor that detects vital activity of the subject 2 and the sensor 12 may be a camera that detects the appearance of the subject 2 . Sensor 11 may be a sensor that detects sounds produced by object 2 and sensor 12 may be a camera that detects the appearance of object 2 . The sensor 12 may differ from the sensor 11 in positional relationship with respect to the target 2 . For example, sensor 12 may have a different distance from object 2 than sensor 11 . For example, the sensor 11 may be a sensor attached to the target 2 like a wearable device, and the sensor 12 may be a sensor remote from the target 2 like a surveillance camera. The sensor 12 may be a sensor that detects the target 2 in a direction different from that of the sensor 11 . For example, the

sensors

11 and 12 both include cameras, the sensor 11 detects (eg, photographs) the target 2 from a first direction (eg, lateral), and the sensor 12 detects the target 2 from the first direction. Detection (eg, photographing) may be performed from a second direction (eg, front) different from the direction.

The sensor 12 provides the detection result to the terminal 26. The terminal 26 may be the same terminal as the terminal 20 described in FIG. 1, or may be a terminal different from the terminal 20. FIG. The terminal 26 has an estimator 27 . The estimation unit 27 estimates the state of the subject 2 using the detection result of the sensor 12 . For example, the estimation unit 27 processes the detection result of the sensor 12 using a model generated by machine learning to estimate the state of the object. The terminal 20 uses the estimation result of the estimation unit 27 to generate state information. The terminal 20 provides the generated state information to the information processing device 30 .

The information processing device 30 generates training data using the detection result of the sensor 11 and the state information provided from the terminal 20. The information processing device 30 provides the generated training data to the information processing device 40. The learning unit 41 of the information processing device 40 executes machine learning using the teacher data provided from the information processing device 30 to generate an inference model. The information processing device 40 provides the generated inference model to the information processing device 50 . The estimation unit 52 of the information processing device 50 may be the same as or different from the estimation unit 51 described with reference to FIG. The estimation unit 52 estimates the state of the target 2 using the detection result of the sensor 11 and the inference model generated by the learning unit 41 . The information processing device 50 provides the estimation result of the estimation unit 52 to the application 60 .

The information processing system 1 according to this embodiment is used, for example, as follows. As described with reference to FIG. 1, the information processing system 1 generates teacher data using the detection result of the sensor 10 and the state information input to the terminal 20 by the evaluator 3, and uses the generated teacher data. Generate a first inference model. The sensor 12 in FIG. 9 includes a sensor that detects the same type of information as the sensor 10, and the estimation unit 27 of the terminal 26 performs estimation processing using the first inference model. For example, the sensor 10 in FIG. 1 and the sensor 12 in FIG. 9 both include sensors that detect information indicating vital activity of the subject 2 (eg, heart rate). A first inference model is a model for estimating the state of the subject 2 from information indicating the vital activity of the subject 2 . The estimation unit 27 of the terminal 26 processes the information indicating the vital activity of the subject 2 detected by the sensor 12 and estimates state information (eg, health condition of the subject 2). Sensor 11 includes, for example, a camera. For example, the information processing apparatus 30 uses the data of the image of the subject 2 photographed by the sensor 11 as the detection result, and the data indicating the health condition of the subject 2 estimated by the estimation unit 27 as the state information, as the second teacher data. to generate The learning unit 41 generates an inference model for estimating the health condition of the target 2 from the image of the target 2 using the second learning data. By repeating such processing, the information processing system 1 can generate an inference model that associates a plurality of types of information.

FIG. 10 is a diagram showing an example of a computer according to the embodiment. At least part of the computer 100 is, for example, at least an information processing device according to the embodiment (eg, information processing device 30, information processing device 40, information processing device 50) and a terminal (eg, terminal 20, terminal 26). used for one. Computer 100 includes CPU 101 , ROM 102 , RAM 103 , bus 104 , input section 105 , output section 106 , storage section 107 , communication section 108 and drive 109 . The CPU 101, ROM 102 and RAM 103 are interconnected via a bus 104, respectively. An input unit 105 , an output unit 106 , a storage unit 107 , a communication unit 108 and a drive 109 are connected to the bus 104 .

The CPU 101 includes a Central Processing Unit. ROM 102 includes Read Only Memory. RAM 103 includes Random Access Memory. The CPU 101 executes various processes according to programs recorded in the ROM 102 or programs loaded from the storage unit 107 to the RAM 103 . The RAM 103 also stores data necessary for the CPU 101 to execute various kinds of processing.

The input unit 105 includes, for example, at least one of a keyboard, mouse, trackball, and touchpad. The input unit 105 receives input of various information by being operated by the user. The input unit 105 may include a microphone or the like that receives voice input. The output unit 106 includes, for example, a display device (eg, display) that outputs images. The output unit 106 may include a speaker that outputs audio. The input unit 105 and the output unit 106 may include a touch panel in which a transmissive touch pad is superimposed on the display unit.

The storage unit 107 includes, for example, at least one of a hard disk, solid state drive, and nonvolatile memory. The storage unit 107 stores at least part of various data externally input to the computer 100 , various data used in processing by the computer 100 , and various data generated by the computer 100 . A communication unit 108 controls communication with another device via a network.

A removable medium 110 is appropriately attached to the drive 109 . Removable media 110 include, for example, storage media such as magnetic disks, optical disks, magneto-optical disks, and semiconductor memories. Drive 109 reads information from removable media 110 . Information read from the removable medium 110 is stored in the storage unit 107 . For example, drive 109 reads a program stored on removable media 110 and this program is installed on computer 100 .

Note that the computer 100 does not have to include one or both of the input unit 105 and the output unit 106 . For example, the output unit 106 may be a device externally attached to the computer 100, and the computer 100 may include an interface to which the output unit 106 can be connected.

In the above-described embodiments, the information processing device and the terminal read the program stored in the storage unit and the storage device, and execute various processes according to this program. This program, for example, causes the computer to receive the detection result of the sensor that detects the target and the state information representing the state of the target, which is information obtained from information different from the detection result of the sensor, from a predetermined terminal associated with the target. Acquiring and using the acquired detection results and state information to generate data used for machine learning are performed.

The above program may be recorded on a computer-readable storage medium and provided. The above program may be a differential program (also referred to as a differential file as appropriate) that executes various processes in combination with a program (eg, operating system) recorded in a computer system.

In the above-described embodiment, the information processing system 1 associates the first information and the second information about the target to generate data used for machine learning. The second information includes information different from the first information. For example, the first information includes information obtained by detecting the target 2 with a sensor, and the second information includes information representing the state of the target judged by a person. The second information may include information obtained by processing the first information by the device. For example, the first information may correspond to the detection result of a sensor that has detected an object, and the second information may include the result of arithmetic processing, statistical processing, or inference processing using the first information. For example, the second information includes information representing the state of the object obtained by other systems. For example, the first information may include information obtained by detecting the target 2 with a sensor, and the second information may include information on the target 2 provided by a system that monitors or provides information.

The first information may be information of a different type from the second information. For example, the first information may be image-based information and the second information may be material composition, temperature, or force-based information. The first information may be visually based information and the second information may be auditory, olfactory, gustatory, or tactile based. The first information may include information acquired by a sensor attached to the target, and the second information may include information acquired by a sensor installed away from the target.

It should be noted that the technical scope of the present invention is not limited to the aspects described in the above embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. Also, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by laws and regulations, the disclosure of all the documents cited in the above-described embodiments and the like is used as part of the description of the text.

1 Information processing system 2 Target 3 Evaluator 10 Sensor 20 Terminal 27 Estimation unit 31 Communication unit 32 Processing unit 33 Storage unit 34 Acquisition unit 35 Identification unit 36 Generation unit 37 Control unit 41 Learning unit 51 Estimation unit 52 Estimation unit 60 Application D6 Candidate information M Model

Claims

An acquisition unit that acquires a detection result of a sensor that detects an object, and acquires state information representing the state of the object, which is obtained from information different from the detection result of the sensor, from a predetermined terminal that is associated with the object. When,
and a generating unit that generates data used for machine learning, using the detection result and the state information acquired by the acquiring unit.
the predetermined terminal includes an evaluator's terminal associated with the subject;
The state information includes information input into the predetermined terminal by the evaluator,
The information processing device according to claim 1 .
The predetermined terminal displays a plurality of candidates representing the state of the target on a display unit, and receives an input designating the state of the target from the plurality of candidates as the state information.
The information processing apparatus according to claim 2.
The predetermined terminal generates the state information by estimating the state of the target using a detection result of a sensor different from the sensor, and provides the generated state information to the acquisition unit.
The information processing device according to claim 1 .
a storage unit that stores candidate information that associates the predetermined terminal candidate with the target;
The information processing according to any one of claims 1 to 4, further comprising a specifying unit that specifies the predetermined terminal from candidates associated with the target in the candidate information stored in the storage unit. Device.
The information processing apparatus according to any one of claims 1 to 4, further comprising a specifying unit that specifies the predetermined terminal from terminals existing within a predetermined range with respect to the position of the detection target of the sensor.
A control unit that requests the predetermined terminal specified by the specifying unit to provide the state information,
The acquisition unit acquires the state information from the predetermined terminal as a response to the request.
The information processing apparatus according to claim 5 or 6.
The control unit requests the predetermined terminal to provide the state information when the detection result satisfies a predetermined condition.
The information processing apparatus according to claim 7.
The generation unit uses the detection result acquired by the acquisition unit to derive the detection result of the sensor at the time when the state information was obtained, and associates the derived detection result of the sensor with the state information. to generate said data;
The information processing apparatus according to any one of claims 1 to 8.
an information processing apparatus according to any one of claims 1 to 9;
An information processing system comprising: an estimation unit that processes the detection result acquired by the acquisition unit using a model generated by the machine learning by the learning unit to estimate the state of the target.
to the computer,
Acquiring a detection result of a sensor that detects an object, and acquiring, from a predetermined terminal associated with the object, state information representing the state of the object, which is information obtained from information different from the detection result of the sensor. ,
An information processing program for generating data used for machine learning using the obtained detection result and the state information.
An information processing method using one or more computers,
The communication unit of the computer receives a detection result of a sensor that detects a target, and transmits state information representing the state of the target, which is information obtained from information different from the detection result of the sensor, to a predetermined state associated with the target. receiving from a terminal of
An information processing method, comprising a processing unit of the computer generating data used for machine learning using the detection result and the state information received by the communication unit.