JP7353328B2 - Terminal device, information processing method, and information processing program - Google Patents

Description

The present invention relates to a terminal device, an information processing method, and an information processing program.

A technique for sending training data to a server device for causing the server device to perform machine learning has been disclosed.

Patent No. 6880290

As in the prior art described above, machine learning is generally performed by a server device. However, if the data includes personal information, it may not be desirable to send it to an external server device such as a company. Furthermore, there are various restrictions on sending and receiving data including personal information, so it may not be easy to send and receive data freely.

Therefore, in recent years, on-device machine learning, such as federated learning, in which machine learning is performed while data is distributed to individual terminal devices without being aggregated on a server device, has been attracting attention. . However, when performing machine learning on individual terminal devices using sensor data that is affected by the shape and performance of the terminal device, it is necessary to There is a problem that differences occur in the measurement results depending on the method.

The present application has been made in view of the above, and aims to share sensor data between individual terminal devices that perform machine learning and learn using sensor data of other terminal devices.

The terminal device according to the present application is a terminal device owned by a user, and includes an acquisition unit that collects self-sensor data using a sensor of the terminal device, and an acquisition unit that collects self-sensor data from other terminal devices owned by other users. a receiving unit that receives other person's sensor data collected by a sensor of a terminal device; a comparing unit that compares the self sensor data and the other person's sensor data; and a comparison unit that compares the self sensor data and the other person's sensor data according to the comparison result. Based on the shared model provided by the recruitment department that adopts sensor data measured by a higher-performance sensor and more accurate than the other sensor data, and the information provision device that uses federated learning, A unique inference model is constructed by performing on-device machine learning using the sensor data adopted by the recruitment department as learning data among the self-sensor data and the other person's sensor data, and the inference model is combined with the shared model. a learning unit that generates a difference parameter ; an inference unit that performs inference from input data using the inference model; an inference result obtained as a result of inference; the inference model obtained as a result of learning; and the shared model. and a transmitting section that transmits a difference parameter between the information providing apparatus and the information providing apparatus .

According to one aspect of the embodiment, sensor data can be shared between individual terminal devices that perform machine learning, and learning can be performed using sensor data of other terminal devices.

FIG. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of a terminal device according to an embodiment. FIG. 4 is a sequence diagram showing the processing procedure according to the embodiment. FIG. 5 is a diagram showing an example of the hardware configuration.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Below, a terminal device, an information processing method, and an embodiment for implementing an information processing program (hereinafter referred to as "embodiments") according to the present application will be described in detail with reference to the drawings. Note that this embodiment does not limit the terminal device, information processing method, and information processing program according to the present application. In addition, in the following embodiments, the same parts are given the same reference numerals, and redundant explanations will be omitted.

[1. Overview of information processing method]
First, with reference to FIG. 1, an overview of an information processing method performed by an information processing apparatus according to an embodiment will be described. FIG. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. Note that in FIG. 1, an example will be described in which sensor data is shared between individual terminal devices that perform machine learning, and learning is performed using sensor data of other terminal devices.

As shown in FIG. 1, the information processing system 1 includes a terminal device 10 and an information providing device 100. The terminal device 10 and the information providing device 100 are connected to each other via a network N (see FIG. 2) so that they can communicate with each other by wire or wirelessly.

The terminal device 10 is a smart device such as a smartphone or a tablet used by a user U (user), and can communicate with any server device via a wireless communication network such as 4G (Generation) or LTE (Long Term Evolution). This is a mobile terminal device that can perform Further, the terminal device 10 has a screen such as a liquid crystal display, which has a touch panel function, and displays data such as content through a tap operation, a slide operation, a scroll operation, etc. from the user U using a finger or a stylus. Accepts various operations on. Note that an operation performed on an area of the screen where content is displayed may be an operation on the content. Further, the terminal device 10 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

The information providing device 100 cooperates with the terminal device 10 of each user U, and provides API (Application Programming Interface) services for various applications (hereinafter referred to as applications), etc., to the terminal device 10 of each user U, and various It is an information processing device that provides data, and is realized by a server device, a cloud system, etc.

Further, the information providing device 100 may be an information processing device that provides some kind of web service online to the terminal device 10 of each user U. For example, the information providing device 100 provides web services such as Internet connection, search service, SNS (Social Networking Service), electronic commerce, electronic payment, online games, online banking, online trading, accommodation/ticket reservation, video/music distribution, Services such as news, maps, route searches, route guidance, route information, operation information, weather forecasts, etc. may be provided. In reality, the information providing apparatus 100 may cooperate with various servers that provide the above-mentioned Web services, mediate the Web services, or may be in charge of processing the Web services.

As shown in FIG. 1, in the present embodiment, the information providing apparatus 100 (server side) employs federated learning, and the information providing apparatus 100 (server side) uses the terminal device 10 (terminal side) of each user U. A shared model (core model) is provided (step S1).

Based on the shared model, the terminal device 10 performs on-device machine learning using sensor data (context information of user U) collected using various sensors installed in (or connected to) the terminal device 10 as learning data. A pattern is created by this, and inference is made from the input data using the pattern (step S2).

Subsequently, the terminal device 10 transmits the inference result obtained as a result of the inference and the difference parameter between the shared model obtained as a result of learning to the information providing device 100 (server side) (step S3). At this time, the terminal device 10 may transmit the inference result in the form of vector data converted into a multidimensional vector (vectorized). Vector data is irreversible information, and the original data cannot be restored from the vector value of vector data. For example, the terminal device 10 may perform so-called embedding and generate an embedding vector based on the inference result. Note that it is possible to encrypt and transmit vector data, which is transmission data, within the scope of common technical knowledge. Furthermore, a closed virtual direct connection line may be established between the terminal side and the server side by tunneling. For example, a VPN (Virtual Private Network) may be constructed between the terminal side and the server side.

The information providing device 100 performs aggregation processing of the inference results received from the terminal device 10 of each user U (step S4). The aggregation process includes calculation processes such as statistics and analysis.

Subsequently, the information providing device 100 modifies the shared model based on the difference parameters received from the terminal device 10 of each user U (step S5). Then, the information providing device 100 provides the corrected shared model (or the difference parameter from the previous shared model) to the terminal device 10 of each user U (returns to step S1). That is, the information providing device 100 updates and upgrades the shared model provided to the terminal device 10 of each user U.

Note that if the information providing apparatus 100 does not need to update the shared model, the process of step S5 may not be performed. In this case, in step S3, the terminal device 10 only needs to transmit the inference result.

In this way, when the terminal device 10 (terminal side) rather than the information providing device 100 (server side) performs machine learning using sensor data as learning data, the model of the terminal device owned by each user, Differences in the type and performance of the installed sensors may cause differences in the measured sensor data. For example, even if users are in the same place or taking the same actions, the number and accuracy of sensor data measured will vary depending on the terminal device model, the type and performance of the installed sensor, etc. can vary greatly. As a result, learning results may also be affected. Therefore, it is desirable to equalize/homogenize sensor data as much as possible for users who are in the same place or take the same actions.

However, since sensor data is also information that can estimate an individual's location, behavior, and context, it may not be desirable to share it unconditionally among all users. Therefore, it is possible to share sensor data between users who have previously given permission/agreed to share sensor data with each other (for example, between family members, friends, etc.), or to share sensor data with other users who are currently in front of you (in the same location). It is desirable to share information and sensor data in situations where there is no problem in sharing it. Hereinafter, terminal devices around user U refer to terminal devices of other users who have permitted/agreed to share sensor data with user U's terminal device 10.

In this embodiment, sensor data is automatically/periodically shared between the terminal devices 10 of each user U (step S6). Thereby, data collected from terminal devices around the user U is regarded as "data acquired by the terminal device 10 of the user U" and learning is performed. At this time, among the terminal devices 10 of each user U, the terminal device 10 (10A) of the first user U (U1) sends a sensor to the terminal device 10 (10B) of the second user U (U2). Transmit data.

For example, the terminal device 10 (10B) of the second user U (U2) automatically/periodically requests sensor data by broadcast, and the terminal device 10 (10B) of the first user U (U1) automatically/periodically requests sensor data by broadcasting. 10A) sends sensor data in response. In this case, the terminal device 10 of the user U may inquire of surrounding terminal devices as to the presence or absence of predetermined sensor data, and if the surrounding terminal devices have the sensor data, may request the sensor data. . Further, the terminal device 10 of the user U declares the desired sensor data and the context of the user U, and receives the sensor data from the terminal device of another user whose context is similar to that of the user U. It's okay.

For example, if the terminal device 10 of user U does not have a temperature sensor, it broadcasts a request such as "Please give me temperature sensor data" to surrounding terminal devices. In response to the request, surrounding terminal devices transmit temperature sensor data to user U's terminal device 10. Note that the temperature sensor is just one example. Additionally, data from multiple sensors may be requested. Alternatively, data from all sensors may be unconditionally requested.

Further, if the terminal device 10 of the user U has sensor data when the user U is walking but does not have sensor data when the user U is running, Broadcast a request such as "Please give me the sensor data of the time." In response to the request, surrounding terminal devices transmit sensor data to the terminal device 10 of the user U when the user of the terminal device is running.

In addition, if there is not enough sensor data when the user U is at a predetermined location (store, point, etc.), the terminal device 10 of the user U will display a message "The user is at the location" to compensate for the lack of learning data. The system broadcasts information such as "Please give me data when you are at that location" and receives sensor data from surrounding terminal devices when the user of that terminal device is at that location.

Note that the terminal device 10 (10A) of the first user U (U1) automatically/periodically sends P2P (Peer to Peer) may be used to pass sensor data. Alternatively, sensor data may be passed between terminal devices of users who are running the same application (game, payment application, etc.) or taking the same action. Note that if both users are using the same app, sensor data may be passed between the users' terminal devices via the app.

Further, the terminal device 10 (10A) of the first user U (U1) automatically/periodically sends the terminal device 10 (10A) to the terminal device 10 (10B) using Bluetooth (registered trademark) or wireless communication. Sensor data may be transmitted by short-range wireless communication such as a LAN (Local Area Network). Note that sensor data may be transmitted by so-called "passing communication". If the data is collected from the terminal devices of users who are close enough to communicate via short-range communication, it is assumed that the sensor data at that time is common.

Alternatively, the terminal device 10 (10A) of the first user U (U1) automatically/periodically connects the home LAN to the terminal device 10 (10B) of the second user U (U2). Sensor data may also be transmitted via the For example, the terminal device 10 (10A) of the first user U (U1) connects the terminal device 10 (10B) of the second user U (U2) to a hub such as a smart home or a home router (use The sensor data may be transmitted via a local relay device managed by person U), a personally owned PC or a home server (home server), or the like. At this time, user U transfers sensor data from the terminal device 10 (10A) to a hub such as a smart home, a home router (a local relay device managed by user U), a personal PC, a home server, etc. You may back up the . Note that the backup of sensor data from the terminal device 10 (10A) may be performed in real time when the sensor data is acquired (real-time support). Then, the terminal device 10 (10B) may acquire the backed up sensor data. If the data is collected from terminal devices of family members or the like within the same household, it is assumed that the sensor data within the household is common. Please note that family is just one example. For example, they may be colleagues at work or members of a group located in the same campus, or employees and customers located in the same facility.

Alternatively, the user U can compress the sensor data into a file in a format that can only be decompressed by the user on the terminal device 10 (10A), and then download and decompress the compressed file on the terminal device 10 (10B). good. For example, the terminal device 10 (10A) may encrypt sensor data with a private key, and provide the encrypted sensor data and the public key only to the terminal device 10 (10B). Note that, in reality, the terminal device 10 (10A) of the first user U (U1) connects to the terminal device 10 (10B) of the second user U (U2) via a closed network. ) may be used to pass sensor data.

In this way, the method of transmitting/sharing sensor data between devices owned by each user U may be P2P, home LAN (home hub format), or file compression type.

Here, when transmitting/sharing sensor data between devices owned by each user U, the terminal device 10 (10A) of the first user U (U1) and the terminal device 10 (10A) of the second user U (U2) Authentication may be performed with the terminal device 10 (10B). The authentication may be FIDO (Fast Identity Online) authentication. Then, only when the authentication is successful, the terminal device 10 (10A) of the first user U (U1) transmits the sensor data to the terminal device 10 (10B) of the second user U (U2). . Note that it is possible to encrypt and transmit the sensor data to be transmitted within the scope of common technical knowledge.

In fact, the terminal device 10 (10B) of the second user U (U2) also transmits its own sensor data to the terminal device 10 (10A) of the first user U (U1). I can do it. That is, the transmission of sensor data between terminals can be performed bidirectionally rather than unidirectionally. In this case, the terminal device 10 (10A) of the first user U (U1) also performs the same process as the terminal device 10 (10B) of the second user U (U2).

Furthermore, not only sensor data but also learned models may be shared between the terminal devices 10 of each user U. For example, the terminal device 10 (10A) of the first user U (U1) learns not only the sensor data but also the sensor data to the terminal device 10 (10B) of the second user U (U2). The learned model may be transmitted as data.

In this embodiment, among the terminal devices 10 of each user U, the terminal device 10 (10B) of the second user U (U2) is the terminal device 10 (10A) of the first user U (U1). The sensor data of the terminal device 10 (10B) of the second user U (U2) are compared (step S7).

Here, it is desirable that the sensor data to be compared be sensor data measured at the same (or close) time and location (that is, sensor data having common position information and location history). Therefore, the terminal device 10 (10B) of the second user U (U2) uses the sensor data of the terminal device 10 (10A) of the first user U (U1) and the second The sensor data of the terminal device 10 (10B) of the user U (U2) is compared. In this case, position information and sensor data are linked. Note that position information and sensor data may be linked using time information as a key. Furthermore, in reality, the sensor data may include measured position information and measurement data from other sensors.

However, when using real-time sensor data, the terminal device 10 of the user U may compare real-time sensor data of itself and others at the time of collection, regardless of location information or time information.

Subsequently, the second user U (U2)'s terminal device 10 (10B) reads the sensor data of the first user U (U1)'s terminal device 10 (10A) from the sensor data of the terminal device 10 (10B). data that is not included in the sensor data of the terminal device (measurement data of a sensor not installed in the terminal device) or data that is more accurate than the sensor data of the terminal device 10 (10B) (measurement data of a sensor with higher performance). ) is adopted (step S8).

At this time, the second user U (U2)'s terminal device 10 (10B) receives the sensor data of the first user U (U1)'s terminal device 10 (10A) and the terminal device 10 (10B). Based on the difference with the sensor data, data that is not included in the sensor data of the terminal device 10 (10B) or data that is more accurate than the sensor data of the terminal device 10 (10B) may be determined. For example, the terminal device 10 (10B) collects data that is missing from the sensor data of the terminal device 10 (10B) among the sensor data of the terminal device 10 (10A) of the first user U (U1), Data (detailed data) having a larger number of digits after the decimal point than the sensor data of the terminal device 10 (10B) may be adopted.

Alternatively, the terminal device 10 (10B) of the second user U (U2) transmits information regarding the terminal device 10 (10A) along with the sensor data of the terminal device 10 (10A) of the first user U (U1). Based on the information about the terminal device 10 (10A) and the information about the terminal device 10 (10B), data not included in the sensor data of the terminal device 10 (10B) and information about the terminal device 10 (10B) are acquired. Data with higher precision than the sensor data in 10B) may be determined. The information regarding the terminal device 10 is, for example, model information of the terminal device 10, information regarding a sensor installed in the terminal device 10, and the like.

Subsequently, the terminal device 10 (10B) of the second user U (U2) receives the sensor data of the terminal device 10 (10A) of the first user U (U1) and the second user U (U2). ), on-device machine learning is performed using the adopted sensor data as learning data, and inference is made based on the learning results (step S9).

Thereby, the terminal device 10 of the user U can share sensor data among the individual terminal devices that perform machine learning and learn using the sensor data of other terminal devices. In particular, learning can be performed using sensor data from other terminal devices that have sensors that are not present in the own terminal device. Furthermore, by comparing the sensor data of one's own and other terminal devices, it is possible to learn using better sensor data.

In addition, in the above description, the sensor data of one's own and other devices are compared, and data not included in the sensor data of the terminal device 10 among the sensor data collected from the other terminal device 10 and the sensor data of the terminal device 10 are compared. Although data with higher accuracy than the data is used, in reality, machine learning may be performed using all sensor data of own and others as learning data without comparison.

Furthermore, when the terminal device 10 of the user U cannot collect sensor data from other terminal devices 10, the terminal device 10 of the user U performs machine learning using only its own sensor data as learning data. Alternatively, the terminal device 10 of the user U does not perform machine learning if sensor data cannot be collected from other terminal devices 10, and performs the above procedure only if sensor data can be collected from other terminal devices 10. Machine learning may also be performed using

[2. Configuration example of information processing system]
Next, the configuration of the information processing system 1 including the information providing apparatus 100 according to the embodiment will be described using FIG. 2. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a terminal device 10 and an information providing device 100. In FIG. 2, as the terminal devices 10, the terminal device 10 (10A) of the first user U (U1) and the terminal device 10 (10B) of the second user U (U2) are shown. These various devices are connected via a network N so that they can communicate by wire or wirelessly. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Furthermore, the number of devices included in the information processing system 1 shown in FIG. 2 is not limited to what is illustrated. For example, in FIG. 2, for simplification of illustration, the terminal device 10 is the terminal device 10 (10A) of the first user U (U1), and the terminal device 10 (10A) of the second user U (U2). 10B) are shown one by one, but this is just an example and is not limiting, and there may be two or more of each. For example, the terminal device 10 (10A) of the first user U (U1) and the terminal device 10 (10B) of the second user U (U2) may exist for each user.

The terminal device 10 is an information processing device used by the user U. For example, the terminal device 10 may be a smart device such as a smartphone or a tablet terminal, a feature phone, a PC (Personal Computer), a PDA (Personal Digital Assistant), a game console or AV device with a communication function, a car navigation system, a smart watch, or the like. These include wearable devices such as head-mounted displays, smart glasses, smart speakers, cameras, etc. However, in reality, it is not limited to these examples.

The terminal device 10 also supports wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), Bluetooth (registered trademark), and wireless LAN (Local It is possible to connect to a network N via short-distance wireless communication such as a local area network) and communicate with the information providing device 100.

The information providing device 100 is, for example, a PC, a server device, a mainframe, a workstation, or the like. Note that the information providing device 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be explained using FIG. 3. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. As shown in FIG. 3, the terminal device 10 includes a communication section 11, a display section 12, an input section 13, a positioning section 14, a sensor section 20, a control section 30 (controller), and a storage section 40. Be prepared.

(Communication Department 11)
The communication unit 11 is connected to a network N (see FIG. 2) by wire or wirelessly, and transmits and receives information to and from the information providing device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, or the like.

(Display section 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic electro-luminescent display (EL display). Further, the display unit 12 is a touch panel type display, but is not limited to this.

(Input section 13)
The input unit 13 is an input device that receives various operations from the user U. For example, the input unit 13 includes buttons for inputting characters, numbers, and the like. Note that the input unit 13 may be an input/output port (I/O port), a USB (Universal Serial Bus) port, or the like. Further, when the display section 12 is a touch panel display, a part of the display section 12 functions as the input section 13. Further, the input unit 13 may be a microphone or the like that receives voice input from the user U. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives a signal (radio wave) sent from a GPS (Global Positioning System) satellite, and based on the received signal, determines position information (for example, latitude and longitude). That is, the positioning unit 14 positions the terminal device 10 . Note that GPS is just one example of GNSS (Global Navigation Satellite System).

Further, the positioning unit 14 can measure the position using various methods other than GPS. For example, the positioning unit 14 may use various communication functions of the terminal device 10 to measure the position as an auxiliary positioning means for position correction and the like, as described below.

(Wi-Fi positioning)
For example, the positioning unit 14 positions the terminal device 10 using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 performs Wi-Fi communication, etc., and determines the position of the terminal device 10 by measuring the distance to nearby base stations and access points.

(Beacon positioning)
Further, the positioning unit 14 may use the Bluetooth (registered trademark) function of the terminal device 10 to measure the position. For example, the positioning unit 14 measures the position of the terminal device 10 by connecting to a beacon transmitter connected by a Bluetooth (registered trademark) function.

(geomagnetic positioning)
Furthermore, the positioning unit 14 positions the terminal device 10 based on the geomagnetic pattern of the structure measured in advance and the geomagnetic sensor included in the terminal device 10 .

(RFID positioning)
Further, for example, if the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used at station ticket gates, stores, etc., or has a function to read an RFID tag. In this case, the location where the terminal device 10 used the terminal device 10 is recorded together with the information that the payment was made. The positioning unit 14 may measure the position of the terminal device 10 by acquiring such information. Further, the position may be determined by an optical sensor, an infrared sensor, or the like provided in the terminal device 10.

The positioning unit 14 may position the terminal device 10 using one or a combination of the above-mentioned positioning means, if necessary.

(sensor section 20)
The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. Note that the connection may be a wired connection or a wireless connection. For example, the sensors may be a detection device other than the terminal device 10, such as a wearable device or a wireless device. In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, an atmospheric pressure sensor 23, an air temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( camera) 28.

Note that each of the sensors 21 to 28 described above is merely an example and is not limited to the above. That is, the sensor section 20 may be configured to include a portion of each of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of each of the sensors 21 to 28. .

The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects the physical movement of the terminal device 10, such as the moving direction, speed, and acceleration of the terminal device 10. The gyro sensor 22 detects physical movements of the terminal device 10 such as tilt in three axes directions based on the angular velocity of the terminal device 10 and the like. The atmospheric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

Since the terminal device 10 is equipped with the above-mentioned acceleration sensor 21, gyro sensor 22, atmospheric pressure sensor 23, etc., it is possible to implement technologies such as pedestrian autonomous navigation (PDR) using these sensors 21 to 23, etc. It becomes possible to measure the position of the terminal device 10 using the . This makes it possible to obtain indoor position information that is difficult to obtain using positioning systems such as GPS.

For example, a pedometer using the acceleration sensor 21 can calculate the number of steps, walking speed, and distance walked. Furthermore, by using the gyro sensor 22, it is possible to know the direction of travel of the user U, the direction of the line of sight, and the inclination of the user's body. Further, from the atmospheric pressure detected by the atmospheric pressure sensor 23, it is also possible to know the altitude and the number of floors where the terminal device 10 of the user U is located.

The temperature sensor 24 detects, for example, the temperature around the terminal device 10. The sound sensor 25 detects, for example, sounds around the terminal device 10. The optical sensor 26 detects the illuminance around the terminal device 10 . The magnetic sensor 27 detects, for example, the earth's magnetism around the terminal device 10. The image sensor 28 captures an image of the surroundings of the terminal device 10.

The above-mentioned atmospheric pressure sensor 23, temperature sensor 24, sound sensor 25, optical sensor 26, and image sensor 28 each detect atmospheric pressure, temperature, sound, and illuminance, and capture images of the surroundings, so that the terminal device 10 It is possible to detect the surrounding environment and situation. Furthermore, it is possible to improve the accuracy of the location information of the terminal device 10 based on the environment and situation around the terminal device 10.

(Control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input/output port, etc., and various circuits. Further, the control unit 30 may be configured with hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control section 30 includes a transmitting section 31 , a receiving section 32 , a processing section 33 , a learning section 34 , an acquiring section 35 , a comparing section 36 , and a recruiting section 37 .

(Transmission unit 31)
The transmitting unit 31 transmits the inference result obtained as a result of inference based on on-device machine learning and the difference parameter from the shared model obtained as a result of learning to the information providing apparatus 100 (server) via the communication unit 11. side).

Further, the transmitting unit 31 transmits the sensor data collected by itself to another terminal device 10 owned by another user U via the communication unit 11 or by short-range wireless communication. For example, if the terminal device 10 is the terminal device 10 (10A) of the first user U (U1), the transmitting section 31 transmits the information to the own terminal device via the communication section 11 or by short-range wireless communication. 10 (10A) transmits the collected sensor data to the terminal device 10 (10B) of the second user U (U2).

That is, the transmitting unit 31 transmits sensor data collected by the sensor installed in the terminal device 10 to another terminal device 10 owned by another user U via the communication unit 11. For example, the transmitter 31 transmits sensor data collected by a sensor installed in the terminal device 10 to another terminal device 10 having a different number of sensors or performance than the terminal device 10 via the communication unit 11. Send. Further, the transmitting unit 31 may transmit the learned model using the sensor data as learning data to another terminal device 10 owned by another user U via the communication unit 11.

In addition, the transmitting unit 31 transmits the information to another terminal device 10 owned by another user U through the communication unit 11 using P2P, a home LAN, or a closed network without leaking the information to the outside. Sensor data collected by a sensor mounted on the terminal device 10 is transmitted.

(Receiving unit 32)
The receiving unit 32 receives the shared model provided from the information providing device 100 via the communication unit 11 . The receiving unit 32 also receives the revised shared model (or the difference parameter from the previous shared model) from the information providing device 100 via the communication unit 11 .

Further, the receiving unit 32 receives sensor data from another terminal device 10 owned by another user U via the communication unit 11 or by short-range wireless communication. For example, when the terminal device 10 is the terminal device 10 (10B) of the second user U (U2), the receiving section 32 receives the first usage information via the communication section 11 or by short-range wireless communication. The sensor data of the terminal device 10 (10A) of the person U (U1) is received.

That is, the receiving unit 32 receives sensor data collected by a sensor installed in another terminal device 10 from another terminal device 10 owned by another user U via the communication unit 11. The receiving unit 32 also receives sensor data collected by sensors installed in the other terminal device 10 from another terminal device 10 having a different number of sensors or performance from the terminal device 10 through the communication unit 11. receive. Further, the receiving unit 32 may receive a model learned using sensor data as learning data from another terminal device 10 owned by another user U via the communication unit 11.

In addition, the receiving unit 32 can receive information from other terminal devices 10 owned by other users U through the communication unit 11 using P2P, home LAN, or closed network without leaking to the outside. The sensor data collected by the sensor installed in the terminal device 10 is received.

In this way, the receiving unit 32 receives other person's sensor data collected by the sensor of the other terminal device 10 from the other terminal device 10 owned by the other user U. At this time, the receiving unit 32 may receive other person's sensor data collected by the sensor of the other terminal device 10 from the other terminal device 10 via short-range wireless communication or a LAN in the same campus. Further, the receiving unit 32 may receive other person's sensor data collected in real time by the sensor of the other terminal device 10 from the other terminal device 10 .

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10, including the display unit 12 and the like. For example, the processing unit 33 can output various types of information transmitted by the transmitting unit 31 and various types of information received by the receiving unit 32 from the information providing device 100 to the display unit 12 for display.

Furthermore, the processing unit 33 updates and upgrades the current shared model based on the corrected shared model (or the difference parameter from the previous shared model).

(Learning Department 34)
The learning unit 34 performs on-device machine learning based on the shared model and using sensor data of the terminal device 10 as learning data. Thereby, the learning unit 34 constructs a unique inference model based on the shared model, and generates a difference parameter between the inference model and the shared model. That is, the learning unit 34 performs machine learning using sensor data as learning data.

In this embodiment, the learning unit 34 performs machine learning using other person's sensor data as learning data. For example, the learning unit 34 performs machine learning using self-sensor data and other-person sensor data as learning data. At this time, the learning unit 34 performs machine learning using the adopted sensor data among the own sensor data and the other person's sensor data as learning data.

(Acquisition unit 35)
The acquisition unit 35 collects sensor data using a sensor installed in the terminal device 10 owned by the user U. That is, the acquisition unit 35 collects self-sensor data using the sensor of the terminal device 10 . For example, the acquisition unit 35 receives various information input by the user U using the input unit 13, various sensor data measured by the sensors 21 to 28 mounted on or connected to the terminal device 10, and information obtained by the positioning unit 14. The position information of the terminal device 10 that has been positioned is collected as sensor data.

(Comparison section 36)
The comparison unit 36 compares self-sensor data and other-person sensor data. At this time, the comparison unit 36 may compare the own sensor data and the other person's sensor data using the position information as a reference.

(Recruitment Department 37)
The hiring unit 37 adopts one of the self-sensor data and the other person's sensor data (preferred sensor data) according to the comparison result. At this time, the hiring unit 37 adopts other person's sensor data that includes data that is not included in the self-sensor data. For example, the recruitment unit 37 employs other person sensor data collected by a sensor installed in another terminal device 10 but not installed in the terminal device 10 concerned. Further, the hiring unit 37 adopts other person's sensor data that is more accurate than the self-sensor data.

(Storage unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk. Ru. The storage unit 40 stores various programs, various data, and the like.

In this embodiment, the storage unit 40 stores a model 41, self-sensor data 42, other-person sensor data 43, and employment rules 44.

The model 41 is a shared model provided by the information providing device 100, an inference model obtained as a result of on-device machine learning using sensor data as learning data, or the like.

Further, the self-sensor data 42 is sensor data collected by the terminal device 10. For example, the self-sensor data 42 includes various information input by the user U using the input unit 13, various sensor data measured by the sensors 21 to 28 mounted on or connected to the terminal device 10, and the positioning unit 14. This is the position information of the terminal device 10 determined by the above. Further, the self-sensor data 42 may include attribute information of the user U and various kinds of history information (log data) indicating the user U's actions. Further, the self-sensor data 42 may include various information regarding the terminal device 10 of the user U.

Further, the other person's sensor data 43 is sensor data collected from another terminal device 10 owned by another user U. That is, the other person's sensor data 43 is sensor data collected by another terminal device 10 owned by another user U.

Further, the adoption rule 44 indicates a rule when either one of the self-sensor data 42 and the other-person sensor data 43 is adopted. For example, out of the other person's sensor data 43, data that is not included in the self-sensor data 42 (measurement data of a sensor not installed in the terminal device) or data that is more accurate than the self-sensor data 42 (higher performance rules such as adopting sensor measurement data). Note that the adoption rule 44 may be an adoption model obtained by machine learning using other person's sensor data 43 (and self-sensor data 42) and adopted sensor data as learning data. In this case, the sensor data obtained as output may be adopted by inputting the other person's sensor data 43 (and the self-sensor data 42) as input data to the adopted model.

Note that the storage unit 40 is not limited to the above, and may store various information depending on the purpose. For example, the storage unit 40 stores attributes such as demographic, psychographic, geographical, and behavioral attributes of the user U. Information may also be stored. For example, the storage unit 40 stores name, family structure, place of birth (locality), occupation, position, income, qualifications, residence type (single-family house, condominium, etc.), presence or absence of a car, commuting/commuting time, commuting/commuting route, etc. Even if you memorize information such as commuter pass sections (stations, routes, etc.), frequently used stations (other than the stations closest to your home or work), lessons (location, time zone, etc.), hobbies, interests, lifestyle, etc. good. The storage unit 40 also stores a location history that is a history of the location and date and time of the user U, a search history that is a history of search queries input by the user U, a viewing history that is a history of contents viewed by the user U, Purchase history (payment history), which is the history of user U's product purchases and payment processing, listing history and sales history, which is the history of user U's listings on the marketplace, and posting history, which is the history of user U's posts. etc. may be stored. Further, the storage unit 40 may store the usage history of various applications installed on the terminal device 10 by the user U, the payment history of the user U's payments (electronic payments) using the terminal device 10, etc. . This information can be used as learning data together with sensor data.

[4. Processing procedure]
Next, a processing procedure by the terminal device 10 according to the embodiment will be described using FIG. 4. FIG. 4 is a sequence diagram showing the processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 30 of the terminal device 10. Here, the process expressed as terminal device 10 is common to the terminal device 10 (10A) of the first user U (U1) and the terminal device 10 (10B) of the second user U (U2). Indicates processing.

As shown in FIG. 4, the acquisition unit 35 of the terminal device 10 collects sensor data (self-sensor data) using the sensor installed in the terminal device 10 (step S101). For example, the acquisition unit 35 receives various information input by the user U using the input unit 13, various sensor data measured by the sensors 21 to 28 mounted on or connected to the terminal device 10, and the positioning unit 14. The position information of the terminal device 10 determined by the above is collected as sensor data.

Subsequently, the transmitting unit 31 of the terminal device 10 transmits the sensor data to another terminal device 10 owned by another user U via the communication unit 11 (step S102). For example, the transmitting unit 31 of the terminal device 10 (10A) of the first user U (U1) transmits sensor data to the terminal device 10 (10B) of the second user U (U2) via the communication unit 11. Send to. At this time, the receiving unit 32 of the terminal device 10B receives the sensor data (other person's sensor data) of the terminal device 10A via the communication unit 11. The same is true vice versa. That is, sensor data is shared between the terminal device 10A and the terminal device 10B. Note that the processing unit 33 of the terminal device 10 may encrypt the sensor data. In this case, the transmitter 31 may transmit a decryption key (such as a public key) to the other terminal device 10 along with the encrypted sensor data. Furthermore, the method of transmitting sensor data between devices may be P2P, home LAN (home hub format), or file compression type.

Subsequently, the comparison unit 36 of the terminal device 10 compares the self-sensor data and the other person's sensor data (step S103). At this time, the comparison unit 36 may compare the own sensor data and the other person's sensor data based on the position information. Note that when receiving sensor data from another person via short-range wireless communication or LAN within the same campus, or when comparing sensor data collected in real time, the comparing unit The sensor data and the other person's sensor data may be directly compared.

Subsequently, the adoption unit 37 of the terminal device 10 adopts one of the self-sensor data and the other person's sensor data (preferred sensor data) according to the comparison result (step S104). For example, the hiring unit 37 adopts other person's sensor data that includes data that is not included in the self-sensor data. Specifically, the recruitment unit 37 employs other person sensor data collected by a sensor installed in another terminal device 10 but not installed in the terminal device 10 concerned. Further, the hiring unit 37 adopts other person's sensor data that is more accurate than the self-sensor data. In the present embodiment, the recruitment unit 37 selects data that is not included in the sensor data of the terminal device 10 from among the sensor data collected from other terminal devices 10 or data that is more accurate than the sensor data of the terminal device 10. Embrace data.

Subsequently, the learning unit 34 of the terminal device 10A performs on-device machine learning based on the shared model and using the adopted sensor data as learning data (step S105). Note that the receiving unit 32 of the terminal device 10A may receive the shared model provided from the information providing device 100 (server side) via the communication unit 11. Further, the transmitting unit 31 of the terminal device 10A may transmit the difference parameters with respect to the shared model as learning results and the inference results to the information providing device 100 (server side) via the communication unit 11. In the present embodiment, the learning unit 34 performs on-device machine learning using the sensor data adopted according to the comparison result as learning data, but in reality, the learning unit 34 performs on-device machine learning using the sensor data adopted according to the comparison result. All of the data may be used as learning data for learning.

[5. Modified example]
The terminal device 10 and the information providing device 100 described above may be implemented in various different forms other than the above embodiments. Therefore, a modification of the embodiment will be described below.

In the above embodiment, part or all of the processing executed by the information providing device 100 may actually be executed by the terminal device 10. For example, the process may be completed stand-alone (by the terminal device 10 alone). In this case, it is assumed that the terminal device 10 has the functions of the information providing device 100 in the above embodiment. Furthermore, in the embodiment described above, since the terminal device 10 cooperates with the information providing device 100, it appears to the user U that the terminal device 10 also executes the processing of the information providing device 100. That is, from another perspective, it can be said that the terminal device 10 includes the information providing device 100.

Furthermore, although the above embodiment describes a case in which sensor data is shared between individual terminal devices that perform machine learning, in reality, the terminal device 10 of user U is owned by another user. Sensor data may be acquired from an external device other than the other terminal device. For example, the terminal device 10 of the user U may acquire sensor data from a fixed terminal device installed on the street or in a store, or may acquire sensor data from a mobile object such as a vehicle moving around the city. . Examples of fixed terminal devices installed on streets or in stores include terminals installed in stores, digital signage (electronic signboards), base stations, access points, and the like. In this case, these external devices correspond to other terminal devices owned by other users.

Furthermore, in the above embodiment, the terminal device 10 may be a smartphone or the like that has a highly functional camera function using on-device AI (Artificial Intelligence). Further, the terminal device 10 may be a robot, a self-driving car, a drone, etc. equipped with on-device AI.

[6. effect〕
As described above, the terminal device 10 according to the present application has a reception unit 32 that receives other person sensor data collected by the sensor of the other terminal device 10 from another terminal device 10 owned by another user U. and a learning unit 34 that performs machine learning using other person's sensor data as learning data.

The terminal device 10 according to the present application further includes an acquisition unit 35 that collects self-sensor data using a sensor of the terminal device 10 owned by the user U. The learning unit 34 then performs machine learning using the self-sensor data and the other-person sensor data as learning data.

The terminal device 10 according to the present application also includes a comparison unit 36 that compares self-sensor data and other-person sensor data, and sensor data of one of the self-sensor data and other-person sensor data according to the comparison result. The system further includes a recruitment section 37 that employs. The learning unit 34 then performs machine learning using the adopted sensor data as learning data.

Furthermore, the comparison unit 36 compares the own sensor data and the other person's sensor data using the position information as a reference.

Further, the hiring unit 37 adopts other person's sensor data that includes data that is not included in the self-sensor data.

Further, the recruitment unit 37 employs other person sensor data collected by a sensor installed in another terminal device 10 but not installed in the terminal device 10 concerned.

Further, the hiring unit 37 adopts other person's sensor data that is more accurate than the self-sensor data.

Further, the receiving unit 32 receives other person's sensor data collected by the sensor of the other terminal device 10 from the other terminal device 10 via short-range wireless communication or a LAN in the same campus.

The receiving unit 32 also receives other person sensor data collected in real time by the sensors of the other terminal devices 10 from the other terminal devices 10 .

By using any one or a combination of the above-described processes, the terminal device according to the present application can share sensor data among individual terminal devices that perform machine learning, and can learn using the sensor data of other terminal devices.

[7. Hardware configuration]
Further, the terminal device 10 and the information providing device 100 according to the embodiments described above are realized by, for example, a computer 1000 having a configuration as shown in FIG. The following will explain the terminal device 10 as an example. FIG. 5 is a diagram showing an example of the hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are connected to a bus. 1090.

The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and performs various processes. The arithmetic device 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the calculation device 1030 for various calculations. Further, the secondary storage device 1050 is a storage device in which data used by the calculation device 1030 for various calculations and various databases are registered, and includes a ROM (Read Only Memory), an HDD (Hard Disk Drive), and an SSD (Solid Disk Drive). This is realized using flash memory, etc. The secondary storage device 1050 may be a built-in storage or an external storage. Further, the secondary storage device 1050 may be a removable storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) memory card. Further, the secondary storage device 1050 may be a cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a display, a projector, and a printer. (Digital Visual Interface) and HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input I/F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, keypad, button, scanner, etc., and is realized by, for example, a USB or the like.

Further, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, output device 1010 and input device 1020 may be wireless devices.

Moreover, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

Note that the input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

Network I/F 1080 receives data from other devices via network N and sends it to computing device 1030, and also sends data generated by computing device 1030 to other devices via network N.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output I/F 1060 and input I/F 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the terminal device 10, the arithmetic unit 1030 of the computer 1000 realizes the functions of the control unit 30 by executing a program loaded onto the primary storage device 1040. Further, the arithmetic device 1030 of the computer 1000 may load a program obtained from another device via the network I/F 1080 onto the primary storage device 1040, and execute the loaded program. Furthermore, the arithmetic unit 1030 of the computer 1000 may cooperate with other devices via the network I/F 1080, and may call and use program functions, data, etc. from other programs of other devices.

[8. others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are in a so-called equivalent range. Furthermore, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the constituent elements can be made without departing from the gist of the embodiments described above.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

For example, the information providing apparatus 100 described above may be realized by a plurality of server computers, and depending on the function, it may be realized by calling an external platform etc. using an API (Application Programming Interface), network computing, etc. can be changed flexibly.

Furthermore, the above-described embodiments and modifications can be combined as appropriate within a range that does not conflict with the processing contents.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the control section can be read as a control means or a control circuit.

1 Information processing system 10 Terminal device 11 Communication unit 14 Positioning unit 20 Sensor unit 30 Control unit 31 Transmission unit 32 Receiving unit 33 Processing unit 34 Learning unit 35 Acquisition unit 36 Comparison unit 37 Recruitment unit 40 Storage unit 100 Information providing device

Claims (11)

A terminal device owned by a user,
an acquisition unit that collects self-sensor data using a sensor of the terminal device;
a receiving unit that receives other person's sensor data collected by the sensor of the other terminal device from another terminal device owned by another user;
a comparison unit that compares the self-sensor data and the other-person sensor data;
a hiring unit that adopts sensor data that is measured by a higher-performance sensor and has higher accuracy than the other one of the self-sensor data and the other-person sensor data according to the comparison result;
Perform on-device machine learning based on a shared model provided by an information providing device that employs federated learning, using sensor data adopted by the hiring unit among the self-sensor data and other-person sensor data as learning data. a learning unit that constructs a unique inference model by doing so, and generates a difference parameter between the inference model and the shared model ;
an inference unit that performs inference from input data using the inference model;
a transmitting unit that transmits an inference result obtained as a result of inference and a difference parameter between the inference model and the shared model obtained as a result of learning to the information providing device;
A terminal device comprising: When real-time sensor data is not used, the comparison unit compares the self-sensor data and the other person measured at the same or close time and position based on the position information and time information linked to the sensor data. Compare with sensor data ,
When real-time sensor data is used, the real-time self-sensor data and the other-person sensor data at the time of collection are compared, regardless of location information or time information . The terminal device described. The recruitment department collects the other person's sensor data received from the other terminal device which has a different model or the type and performance of the installed sensor from the terminal device and which is not included in the self-sensor data. , employing the other person's sensor data having data missing from the self-sensor data or data having more digits after the decimal point than the self-sensor data . The terminal device described. The receiving unit receives the sensor data of the other person collected by the sensor of the other terminal device from the other terminal device that is close enough to be able to communicate with the terminal device by short-range wireless communication. Receive the other person's sensor data from the other terminal device whose sensor data at the time of collection is presumed to be common to the terminal device.
The terminal device according to any one of claims 1 to 3 , characterized in that: The receiving unit is configured to receive the other person's sensor data collected by the sensor of the other terminal device from the other terminal device via a LAN in the same campus , so that the reception unit is configured to receive the other person's sensor data collected by the sensor of the other terminal device via a LAN in the same campus. receiving the other person's sensor data from the other terminal device owned by the other user;
The terminal device according to any one of claims 1 to 4 , characterized in that: The receiving unit inquires of the other terminal devices around the terminal device about the presence or absence of predetermined sensor data collected by a sensor not installed in the terminal device, and the surrounding terminal devices check the predetermined sensor data. If the data is available, request the predetermined sensor data and receive the predetermined sensor data as a response.
The terminal device according to any one of claims 1 to 5. The receiving unit asserts desired sensor data and a context of the user, and receives the other person's sensor data from the other terminal device owned by the other user whose context is similar to that of the user. do
The terminal device according to any one of claims 1 to 6. When transmitting sensor data between the terminal device and the other terminal device is not unidirectional but bidirectional, and the sensor data is shared between the terminal devices, the terminal device and the other terminal device Authentication section that performs authentication between terminals
The terminal device according to any one of claims 1 to 7, further comprising the following. The transmitting unit transmits the inference result obtained as a result of the inference to the information providing device in the form of vector data obtained by converting the inference result into a multidimensional vector,
The vector data is irreversible information, and the original data cannot be restored from the vector value of the vector data.
The terminal device according to any one of claims 1 to 8. An information processing method executed by a terminal device owned by a user ,
an acquisition step of collecting self-sensor data using a sensor of the terminal device;
a receiving step of receiving other person's sensor data collected by the sensor of the other terminal device from another terminal device owned by another user;
a comparison step of comparing the self-sensor data and the other-person sensor data;
a hiring step of adopting sensor data that is measured by a higher-performance sensor and has higher precision than the other one of the self-sensor data and the other-person sensor data according to the comparison result;
Perform on-device machine learning based on a shared model provided by an information providing device that employs federated learning, using the sensor data adopted in the adoption process among the self-sensor data and the other- person sensor data as learning data. a learning step of constructing a unique inference model and generating a difference parameter between the inference model and the shared model ;
an inference step of inferring from input data using the inference model;
a transmission step of transmitting an inference result obtained as a result of inference and a difference parameter between the inference model and the shared model obtained as a result of learning to the information providing device;
An information processing method characterized by comprising: An information processing program to be executed by a terminal device owned by a user,
an acquisition procedure for collecting self-sensor data using a sensor of the terminal device;
A reception procedure for receiving other person's sensor data collected by a sensor of the other terminal device from another terminal device owned by another user;
a comparison procedure of comparing the self-sensor data and the other-person sensor data;
a hiring procedure of adopting sensor data that is measured by a higher performance sensor and has higher accuracy than the other one of the self-sensor data and the other-person sensor data according to the comparison result;
Based on a shared model provided by an information providing device that employs federated learning, on-device machine learning is performed using the sensor data adopted by the adoption procedure among the self-sensor data and the other-person sensor data as learning data. a learning procedure for constructing a unique inference model and generating a difference parameter between the inference model and the shared model ;
an inference step of inferring from input data using the inference model;
a transmission procedure of transmitting an inference result obtained as a result of inference and a difference parameter between the inference model and the shared model obtained as a result of learning to the information providing device;
An information processing program that causes a terminal device to execute

Images