WO2023083026A1

WO2023083026A1 - Data acquisition method and system, and related device

Info

Publication number: WO2023083026A1
Application number: PCT/CN2022/128453
Authority: WO
Inventors: 王成录; 高翔宇; 宋超领; 吴金娴; 周昕宇; 王翃宇; 黄维; 王松涛
Original assignee: 华为技术有限公司
Priority date: 2021-11-12
Filing date: 2022-10-29
Publication date: 2023-05-19
Also published as: CN116127082A

Abstract

A data acquisition method and system, and a related device. The data acquisition method can be applied to a data acquisition system, the data acquisition system comprising a plurality of apparatuses and a federation management center, and the method comprising: the federation management center can receive a data acquisition task sent by a first apparatus of the plurality of apparatuses; in accordance with the data acquisition task and in accordance with sensing capability information of the plurality of apparatuses, the federation management center determines one or more apparatuses for providing sensing data required by the data acquisition task; the federation management center instructs the one or more apparatuses to provide the required sensing data. In the data acquisition method provided by the embodiments of the present application, a plurality of (two or more) electronic apparatuses can establish a data acquisition system, and one electronic apparatus in the data acquisition system can obtain raw data acquired by a sensor of another electronic apparatus, or can obtain semantic information obtained from the raw data acquired by the sensor.

Description

A data acquisition method, system and related device

This application claims the priority of the Chinese patent application with the application number 202111340157.5 and the application name "A Data Acquisition Method and Related Device" filed with the China Patent Office on November 12, 2021, and claims that it was filed on December 30, 2021 The priority of the Chinese patent application with the application number 202111669822.5 and the application title "A Data Acquisition Method, System and Related Device" submitted to the China Patent Office, the entire content of which is incorporated by reference in this application.

technical field

The present application relates to the fields of electronic equipment and data processing, and in particular to a data collection method, system and related devices.

Background technique

Artificial intelligence technology (artificial intelligence, AI) and artificial intelligence Internet of Things technology (artificial intelligence & Internet of things, AIOT) and its application have made breakthroughs in many fields and are rapidly changing the economy and lifestyle. On the basis of AI technology and AIOT technology, electronic devices are becoming more and more intelligent. The electronic device can adjust the working state of the electronic device according to various data (such as current user status (exercise, stillness), user age, etc.), so as to bring better experience to the user. For example, a smart light equipped with an ambient light collection sensor can adjust the brightness of the smart light according to ambient light data.

However, many electronic devices do not have their own sensors capable of collecting the required data. In some scenarios, there are other electronic devices around the electronic device that can collect the required data. Electronic devices can collect required data with the help of other electronic devices. However, different electronic devices collect data in different formats, the electronic devices have different data processing capabilities, and different electronic devices have different data interfaces. It is difficult for electronic equipment to obtain consistent data such as the required data format and accuracy through other electronic equipment.

Therefore, how an electronic device integrates multiple electronic devices that can provide different sensor data into a system, so that an electronic device in the system can obtain data meeting requirements through other electronic devices in the system, is an urgent problem to be solved.

Contents of the invention

This application provides a data collection method, system and related devices. Through the data collection method provided by the embodiment of this application, the data demand device can only send a data collection request, and the central management device can use the semantic information in the data collection request Find data acquisition devices capable of this semantic information.

In a first aspect, the present application provides a data collection method, which is applied to a data collection system. The data collection system may include multiple devices and a federal management center, and the multiple devices include the first device. The data collection method may include: the federation management center receives the data collection task sent by the first device; the federation management center determines one of the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices or multiple devices; the federation management center instructs one or more devices to provide the required sensory data.

The federation management center can be deployed in one device, or distributed in one or more devices in the data collection system, and the devices can include electronic devices and servers.

Through the data collection method provided by the embodiment of the present application, the data demand device only needs to send the data collection task to the central management device, and does not need to know which device can provide the data required for the data collection request, and the central management device can based on the data collection task The data required in Identify the data acquisition devices that can provide that data. The data demand equipment does not need to have a unified data transmission interface with the data acquisition equipment, and can also obtain the data collected by the data acquisition equipment.

In combination with the first aspect, in a possible implementation, the federation management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices; Including: the federal management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices; As well as the association relationship between the sensing data of multiple devices and/or the device states of multiple devices, it is determined that one or more devices that provide the sensing data required by the data collection task perform the data collection task.

Wherein, the device status includes one or more of the battery status of the multiple devices, the wearing status, the distance from the first device, and the accuracy of sensor data provided by the multiple devices.

In this way, the federal management center can determine the data collection equipment that can provide the sensory data required for the data collection task.

In combination with the first aspect, in a possible implementation, the federation management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensor data of multiple devices; including: The federation management center analyzes the data collection task, and determines that the sensing data required for the data collection task includes the first sensing data and the second sensing data; the federation management center uses the first sensing data and the second sensing data; Determine the second device that provides the first sensing data and the third device that provides the second sensing data from multiple devices; the federal management center sends the first collection subtask to the second device, and sends the second collection subtask to the third device Subtasks; the first collection subtask is used to instruct the second device to provide the first sensing data, and the second collection subtask is used to instruct the third device to provide the second sensing data.

In this way, the federation management center can decompose the data collection task into different sub-tasks, and find the equipments that respectively complete the sub-tasks.

In combination with the first aspect, in a possible implementation manner, after the federation management center sends the first collection subtask to the second device, and sends the second collection subtask to the third device, the data collection method may further include: second The device accepts or rejects the first collection subtask; the third device accepts or rejects the second collection subtask.

In this way, one or more devices in the data collection system can decide whether to accept the data collection task.

With reference to the first aspect, in a possible implementation manner, the second device accepting or rejecting the first collection subtask includes: the second device determines to accept or reject the first collection subtask based on remaining resources in the second device. Task.

With reference to the first aspect, in a possible implementation manner, the second device determines to accept or reject the first collection subtask based on remaining resources in the second device, including: determining at the second device that the second device When the remaining resources in the resource are greater than the resources to be used required by the first acquisition subtask, the second device sends a first message to the federation management center, and the first message is used to instruct the second device to accept the first acquisition subtask; the second The remaining resources in the device include remaining computing resources and storage resources in the second device; when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask, the second device Sending a second message to the federation management center, where the second message is used to instruct the second device to refuse to accept the first acquisition subtask.

In this way, one or more devices in the data collection system can determine whether to accept the data collection task according to the remaining resources in the devices and the resources to be used required to perform the data collection task.

With reference to the first aspect, in a possible implementation manner, the federation management center instructs one or more devices to provide required sensing data, including: one or more devices establish a data transmission channel with the first device; a One or more devices send the required sensing data to the first device through the data transmission channel.

In this way, the data collection device, that is, one or more devices, can directly send the collected sensing data to the data demand device, that is, the first device.

In combination with the first aspect, in a possible implementation, the federation management center instructs one or more devices to provide the required sensing data, including: the federation management center sends the required sensing data to one or more devices , sent to the first device.

In this way, the data collection device, that is, one or more devices, can send the collected sensing data to the data demand device, that is, the first device through the federation management center.

In combination with the first aspect, in a possible implementation, the sensory data required for the data collection task includes the first semantic information; the federal management center sends the required sensory data sent by one or more devices to the first A device, including: the federation management center converts raw signals collected by sensors or second semantic information sent by one or more devices into first semantic information, and sends the first semantic information to the first device.

In this way, the federal management center can convert the original signal or low-level semantic information collected by the sensor sent by the data collection device into the semantic information or high-level semantic information required by the data demand device.

With reference to the first aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask, the second device sends a The second message, the second message is used to indicate that the second device refuses to accept the first collection subtask; including: the federation management center determines the provider of the first sensing data according to the first collection subtask and the sensing capability information of multiple devices Fourth device.

In this way, when a device in the data collection system rejects the task, the federation management center can find another device to perform the data collection task.

With reference to the first aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are greater than the resources to be used required by the first acquisition subtask, the second device sends the After the federation management center sends the first message, the first message is used to instruct the second device to accept the first collection subtask, including: the second device collects the first sensing data based on the first collection subtask; the federation management center It is determined that the second device is in an abnormal state, and the federal management center determines the fourth device that provides the first sensor data; the abnormal state includes that the battery power of the second device is lower than the power threshold, the connection between the second device and the federal management center is disconnected, and the fourth device is connected to the federal management center. One or more of the second device not being worn, and the second device disconnected from the power supply.

In this way, when the data collection device is abnormal in the execution of the data collection task, the federation management center can also find another device to continue to complete the data collection task.

With reference to the first aspect, in a possible implementation manner, the data collection task carries an arbitration identifier, and the arbitration identifier is used to indicate that the number of times the fifth device in the one or more devices refuses to accept the data collection task exceeds a preset threshold; federated management The center instructs one or more devices to provide the required sensing data; including: the federal management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device.

In this way, the data collection device can be prevented from maliciously refusing to accept the data collection task.

With reference to the first aspect, in a possible implementation, the federation management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center sends a third message to the fifth device , the required third message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center, and the resources to be used estimated by the fifth device to perform data collection tasks; the federation management center receives the fifth device The remaining resources in the fifth device sent, and the estimated resources to be used by the fifth device to perform data collection tasks; if the remaining resources are greater than the required resources to be used, the federation management center instructs the fifth device to provide Data required for data collection tasks.

In this way, the federal management center can more accurately judge whether the fifth device can accept the data collection task.

With reference to the first aspect, in a possible implementation, the federation management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center sends a fourth message to the fifth device , the required fourth message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center; the federation management center receives the remaining resources in the fifth device sent by the fifth device; based on the data collection task, the federation management center, Evaluate the to-be-used resources required by the fifth device to perform the data collection task; if the remaining resources are greater than the required to-be-used resources, the federation management center instructs the fifth device to provide the data required by the data collection task.

In combination with the first aspect, in a possible implementation manner, before the federation management center receives the data collection task sent by the first device; the method may also include: the federation management center receives the sensing capability information of multiple devices; the federation management The center derives new sensing capability information based on the sensing capability information of multiple devices and the downloaded first knowledge map.

In this way, the federal management center can obtain all the sensing capability information in the data acquisition system.

In combination with the first aspect, in a possible implementation, the federation management center receives the sensing capability information of multiple devices; it may include: the federation management center receives a message that the sixth device among the multiple devices joins the data collection system and The device model of the sixth device; based on the device model of the sixth device, the federation management center obtains the sensing capability information of the sixth device from the first server; the first server stores different device models and the sensor information corresponding to the device model sensory information.

In this way, whenever a device is added to the data collection system, the federal management center can update the sensing capability information in the data collection system.

With reference to the first aspect, in a possible implementation manner, after the federal management center obtains the capability information of the sixth device from the first server based on the device model of the sixth device, the method may further include: Sixth, the sensing capability information of the device, and the second knowledge graph, deduce new sensing capability information; the second knowledge graph includes the first knowledge graph, and the sensing capability information contained in the second knowledge graph is more than that contained in the first knowledge graph It contains a lot of sensing capability information; based on the new sensing capability information, the sensing capability information stored in the federal management center is updated.

In this way, whenever a device is added to the data collection system, the federal management center can also derive new sensing capability information based on the sensing capability information of the newly added device.

With reference to the first aspect, in a possible implementation manner, after the federation management center derives new sensing capability information based on the sensing capability information of the sixth device and the second knowledge map, the method may further include: The federation management center determines that the sixth device exits the data collection system; the federation management center deletes the stored sensing capability information of the sixth device, and deletes new sensing capability information derived based on the sixth device's sensing capability information.

In combination with the first aspect, in a possible implementation manner, the federation management center determines that the sixth device exits the data collection system, including: the federation management center does not receive the fifth message sent by the sixth device within a preset period of time Next, the federal management center determines that the sixth device exits the data collection system; the fifth message is used to indicate that the sixth device is in the data collection system.

With reference to the first aspect, in a possible implementation manner, the federation management center determines that the sixth device exits the data collection system, including: the federation management center receives a sixth message sent by the sixth device, and the sixth message is used to indicate that the sixth device The device exits the data acquisition system.

With reference to the first aspect, in a possible implementation manner, one or more devices further include a seventh device, and the method further includes: the federal management center sends first indication information to the seventh device, and the first indication information is used to indicate In the case of determining that the third sensor data collected by the seventh device is in the first state, the seventh device sends the third sensor data to the federation management center; the federation management center receives the third sensory data sent by the seventh device; The federal management center sends the third sensing data to one or more devices subscribed to the third sensing data.

In combination with the first aspect, in a possible implementation manner, after the federation management center receives the data collection task sent by the first device, the method further includes: ; The federation management center determines that the federation management center stores the sensing data required for the data collection task; the federation management center sends the required sensing data to the first device.

With reference to the first aspect, in a possible implementation manner, the sensing capability information includes the sensing data in the device and the accuracy of the sensing data in the device; the first knowledge map is created by the federation management center based on the Sensitivity information is generated; the second knowledge map is stored in the server, and the first knowledge map includes the sensing data of the multiple devices and the association relationship between the sensing data of the multiple devices. When the fourth sensing data and the fifth sensing data can be calculated or derived to obtain the sixth sensing data, it can be said that the fourth sensing data has an associated relationship with the fifth sensing data and the sixth sensing data.

In the second aspect, a data collection method is provided, and the data collection method is applied to a data collection system. The data collection system may include multiple devices and a federal management center, and the multiple devices include the first device. The data collection method may include: the first device sends a first data collection task to the federal management center, and the first data collection task indicates that when the sensor data required by the first data collection task meets the first condition, obtain the first Sensing data required for the data collection task; based on the data collection task and the sensing capability information of multiple devices, the federal management center determines one or more devices that provide the sensing data required for the first data collection task; The federal management center sends first indication information to one or more devices, and the first indication information is used to indicate that when it is determined that the required sensor data collected by one or more devices meets the first condition, one or more devices Send the required sensor data to the state management center.

Wherein, the federation management center is deployed in one device, or distributed in multiple devices in the data collection system, and the devices include electronic devices and servers.

Among them, the sensing capability information includes the sensing data in the device and the accuracy of the sensing data in the device; the first condition includes that the required sensor data exceeds the first threshold, and the sensing data required for collection at the second time is different from the first threshold. The difference of the sensor data required for collection at a time is greater than a second threshold; the second time is later than the first time.

In this way, the data demand device can subscribe data to the data collection device, that is, only when the sensing data meets the first condition, the data collection system device will send the sensing data to the data demand device.

With reference to the second aspect, in a possible implementation, the data collection method may further include: the second device sends a second data collection task to the federation management center, the second collection task includes a collection cycle parameter, and the collection cycle parameter indicates The second device needs the cycle of the fourth sensing data; the federal management center determines one or more devices to provide the fourth sensing data based on the second data collection task and the sensing capability information of multiple devices; the federal management center instructs one The or multiple devices collect fourth sensory data based on the collection period parameter.

In this way, the data demand device can periodically obtain the sensing data collected by the data collection device.

With reference to the second aspect, in a possible implementation, the data collection method may further include: the federation management center receives the data collection task sent by the first device; the federation management center information to determine one or more devices that provide the sensory data required for the data collection task; the federal management center instructs one or more devices to provide the required sensory data.

In combination with the second aspect, in a possible implementation, the federation management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices; Including: the federal management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices; As well as the association relationship between the sensing data of multiple devices and/or the device states of multiple devices, it is determined that one or more devices that provide the sensing data required by the data collection task perform the data collection task.

In combination with the second aspect, in a possible implementation, the federation management center determines one or more devices that provide the sensing data required for the data collection task according to the data collection task and the sensor data of multiple devices; including: The federation management center analyzes the data collection task, and determines that the sensing data required for the data collection task includes the first sensing data and the second sensing data; the federation management center uses the first sensing data and the second sensing data; Determine the second device that provides the first sensing data and the third device that provides the second sensing data from multiple devices; the federal management center sends the first collection subtask to the second device, and sends the second collection subtask to the third device Subtasks; the first collection subtask is used to instruct the second device to provide the first sensing data, and the second collection subtask is used to instruct the third device to provide the second sensing data.

With reference to the second aspect, in a possible implementation manner, after the federation management center sends the first collection subtask to the second device, and sends the second collection subtask to the third device, the data collection method may further include: second The device accepts or rejects the first collection subtask; the third device accepts or rejects the second collection subtask.

With reference to the second aspect, in a possible implementation manner, the second device accepting or rejecting the first collection subtask includes: the second device determines to accept or reject the first collection subtask based on remaining resources in the second device. Task.

With reference to the second aspect, in a possible implementation manner, the second device determines to accept or reject the first acquisition subtask based on remaining resources in the second device, including: determining at the second device that the second device When the remaining resources in the resource are greater than the resources to be used required by the first acquisition subtask, the second device sends a first message to the federation management center, and the first message is used to instruct the second device to accept the first acquisition subtask; the second The remaining resources in the device include remaining computing resources and storage resources in the second device; when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask, the second device Sending a second message to the federation management center, where the second message is used to instruct the second device to refuse to accept the first collection subtask.

With reference to the second aspect, in a possible implementation manner, the federation management center instructs one or more devices to provide required sensing data, including: one or more devices establish a data transmission channel with the first device; a One or more devices send the required sensing data to the first device through the data transmission channel.

In combination with the second aspect, in a possible implementation manner, the federation management center instructs one or more devices to provide the required sensing data, including: the federation management center sends the required sensing data to one or more devices , sent to the first device.

In combination with the second aspect, in a possible implementation, the sensory data required for the data collection task includes the first semantic information; the federal management center sends the required sensory data sent by one or more devices to the first A device, including: the federation management center converts raw signals collected by sensors or second semantic information sent by one or more devices into first semantic information, and sends the first semantic information to the first device.

With reference to the second aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first collection subtask, the second device sends a The second message, the second message is used to indicate that the second device refuses to accept the first collection subtask; including: the federation management center determines the provider of the first sensing data according to the first collection subtask and the sensing capability information of multiple devices Fourth device.

With reference to the second aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are greater than the resources to be used required by the first collection subtask, the second device sends the After the federation management center sends the first message, the first message is used to instruct the second device to accept the first collection subtask, including: the second device collects the first sensing data based on the first collection subtask; the federation management center It is determined that the second device is in an abnormal state, and the federal management center determines the fourth device that provides the first sensor data; the abnormal state includes that the battery power of the second device is lower than the power threshold, the connection between the second device and the federal management center is disconnected, and the fourth device is connected to the federal management center. One or more of the second device not being worn, and the second device disconnected from the power supply.

With reference to the second aspect, in a possible implementation manner, the data collection task carries an arbitration identifier, and the arbitration identifier is used to indicate that the number of times the fifth device in the one or more devices refuses to accept the data collection task exceeds a preset threshold; federated management The center instructs one or more devices to provide the required sensing data; including: the federal management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device.

With reference to the second aspect, in a possible implementation, the federation management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center sends a third message to the fifth device , the required third message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center, and the resources to be used estimated by the fifth device to perform data collection tasks; the federation management center receives the fifth device The remaining resources in the fifth device sent, and the estimated resources to be used by the fifth device to perform data collection tasks; if the remaining resources are greater than the required resources to be used, the federation management center instructs the fifth device to provide Data required for data collection tasks.

With reference to the second aspect, in a possible implementation, the federation management center instructs the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center sends a fourth message to the fifth device , the required fourth message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center; the federation management center receives the remaining resources in the fifth device sent by the fifth device; based on the data collection task, the federation management center, Evaluate the to-be-used resources required by the fifth device to perform the data collection task; if the remaining resources are greater than the required to-be-used resources, the federation management center instructs the fifth device to provide the data required by the data collection task.

In combination with the second aspect, in a possible implementation manner, before the federation management center receives the data collection task sent by the first device; the method may also include: the federation management center receives the sensing capability information of multiple devices; the federation management The center derives new sensing capability information based on the sensing capability information of multiple devices and the downloaded first knowledge map.

With reference to the second aspect, in a possible implementation, the federation management center receives the sensing capability information of multiple devices; it may include: the federation management center receives a message that the sixth device among the multiple devices joins the data collection system and The device model of the sixth device; based on the device model of the sixth device, the federation management center obtains the sensing capability information of the sixth device from the first server; the first server stores different device models and the sensor information corresponding to the device model sensory information.

With reference to the second aspect, in a possible implementation manner, after the federal management center obtains the capability information of the sixth device from the first server based on the device model of the sixth device, the method may further include: Sixth, the sensing capability information of the device, and the second knowledge graph, deduce new sensing capability information; the second knowledge graph includes the first knowledge graph, and the sensing capability information contained in the second knowledge graph is more than that contained in the first knowledge graph It contains a lot of sensing capability information; based on the new sensing capability information, the sensing capability information stored in the federal management center is updated.

With reference to the second aspect, in a possible implementation manner, after the federation management center derives new sensing capability information based on the sensing capability information of the sixth device and the second knowledge map, the method may further include: The federation management center determines that the sixth device exits the data collection system; the federation management center deletes the stored sensing capability information of the sixth device, and deletes new sensing capability information derived based on the sixth device's sensing capability information.

In combination with the second aspect, in a possible implementation, the federation management center determines that the sixth device exits the data collection system, including: the federation management center does not receive the fifth message sent by the sixth device within a preset period of time Next, the federal management center determines that the sixth device exits the data collection system; the fifth message is used to indicate that the sixth device is in the data collection system.

With reference to the second aspect, in a possible implementation manner, the federation management center determines that the sixth device exits the data collection system, including: the federation management center receives a sixth message sent by the sixth device, and the sixth message is used to indicate that the sixth device The device exits the data acquisition system.

With reference to the second aspect, in a possible implementation, the one or more devices further include a seventh device, and the method further includes: the federation management center sends first indication information to the seventh device, and the first indication information is used to indicate In the case of determining that the third sensor data collected by the seventh device is in the first state, the seventh device sends the third sensor data to the federation management center; the federation management center receives the third sensory data sent by the seventh device; The federal management center sends the third sensing data to one or more devices subscribed to the third sensing data.

With reference to the second aspect, in a possible implementation manner, after the federation management center receives the data collection task sent by the first device, the method further includes: ; The federation management center determines that the federation management center stores the sensing data required for the data collection task; the federation management center sends the required sensing data to the first device.

With reference to the second aspect, in a possible implementation manner, the sensing capability information includes the sensing data in the device and the accuracy of the sensing data in the device; the first knowledge graph is created by the federation management center based on the Sensitivity information is generated; the second knowledge map is stored in the server, and the first knowledge map includes the sensing data of the multiple devices and the association relationship between the sensing data of the multiple devices. When the fourth sensing data and the fifth sensing data can be calculated or derived to obtain the sixth sensing data, it can be said that the fourth sensing data has an associated relationship with the fifth sensing data and the sixth sensing data.

In a third aspect, a data collection method is provided, the data collection method is applied to a data collection system, and the data collection system may include a first electronic device, a second electronic device, and a third electronic device. Wherein, the first electronic device establishes a communication connection with the second electronic device, and the second electronic device establishes a communication connection with the third electronic device; wherein, the data collection method may include: sending the first electronic device to the second electronic device Send the data acquisition task; the second electronic device analyzes the data acquisition task, determines the first sensor data required in the data acquisition task, and provides the third electronic device for the first sensor data; the first sensor data includes the hardware in the third electronic device The data collected by the sensor and the data provided by the software in the third electronic device; the second electronic device sends the data collection task to the third electronic device; the third electronic device collects the first sensor data based on the data collection task; the third electronic device The first sensor data is sent to the first electronic device.

The first electronic device may be a data demand device, the second electronic device may be a federal management center, and the third electronic device may be a data collection device.

In this way, through the data collection method provided by the embodiment of the present application, the data requesting device only needs to send a data collection task to the central management device, and does not need to know which device can provide the data required for the data collection request. The central management device can The data required in the collection task determines the data collection equipment that can provide the data. The data demand equipment does not need to have a unified data transmission interface with the data acquisition equipment, and can also obtain the data collected by the data acquisition equipment.

With reference to the third aspect, in a possible implementation manner, the second electronic device parses the data collection task, determines the first sensor data required in the data collection task, and provides the first sensor data to a third electronic device; The first sensor data includes the data collected by the hardware sensor in the third electronic device and the data generated by the software in the third electronic device; including: the second electronic device analyzes the data collection task, and determines the first data collection task required in the data collection task. Sensor data; the second electronic device queries the third electronic device that provides the first sensor data from the sensing capability information list; the sensing capability information list is used to store the identification of one or more sensor data included in the data acquisition system , and the identification of the electronic device that provides the sensor data; the identification of one or more types of sensor data includes the identification of the first sensor data, and the identification of the electronic device includes the identification of the third electronic device.

With reference to the third aspect, in a possible implementation manner, the second electronic device analyzes the data collection task, determines the first sensor data required in the data collection task, and provides the third electronic device with the first sensor data; the first Sensor data includes data collected by hardware sensors in the third electronic device and data generated by software in the third electronic device; including:

The second electronic device parses the data collection task, and determines the first semantic information required by the data collection task, and the second electronic device determines that the first semantic information is obtained based on the first sensor data; the second electronic device determines from the sensor The third electronic device that provides sensor data is found in the capability information list; the sensor capability information list is used to store the identification of one or more sensor data included in the data acquisition system and the identification of the electronic device that provides sensor data; The identification of one or more types of sensor data includes the identification of the first sensor data, and the identification of the electronic device includes the identification of the third electronic device.

With reference to the third aspect, in a possible implementation manner, the identification of the sensor data includes one or more of the name, ID, and type of the sensor data; the identification of the electronic device includes the name of the electronic device and the device ID.

In combination with the third aspect, in a possible implementation manner, the second electronic device sends the data collection task to the third electronic device; including: the second electronic device decomposes the data collection task into data collection subtasks, and the data collection subtask It is used to instruct the third electronic device to collect the first sensor data; the second electronic device sends the data collection subtask to the third electronic device.

With reference to the third aspect, in a possible implementation manner, before the third electronic device collects sensor data based on the data collection task, the data collection method may further include: before the third electronic device determines remaining resources in the third electronic device If it is greater than the resource to be used required by the data collection task, the third electronic device sends a first message to the second electronic device, and the first message is used to instruct the third electronic device to accept the data collection task; When the remaining resources in the third electronic device are less than the resources to be used required by the data collection task, the third electronic device sends a second message to the second electronic device, and the second message is used to instruct the third electronic device to refuse to accept the data collection task .

With reference to the third aspect, in a possible implementation, the third electronic device sends the sensor data to the first electronic device, including: the third electronic device sends the sensor data to the second electronic device; the second electronic device sends the sensor data to the second electronic device; The data is sent to the first electronic device; or, the third electronic device sends the sensor data to the first electronic device.

With reference to the third aspect, in a possible implementation manner, the third electronic device sends the sensor data to the first electronic device, including: the third electronic device establishes a data transmission channel with the first electronic device; the third electronic device passes the data The transmission channel sends the sensor data to the first electronic device.

With reference to the third aspect, in a possible implementation manner, the second electronic device decomposes the data collection task into data collection subtasks, and the data collection subtask is used to instruct the third electronic device to collect the first sensor data, including: the second The electronic device determines the first semantic information required for determining the data collection task, and the first semantic information is obtained based on the first sensor data; the second electronic device decomposes the data collection task into a data collection subtask and a semantic conversion subtask, and the data The collection subtask is used to instruct the third electronic device to collect the first sensor data; the semantic conversion subtask is used to instruct the third electronic device to convert the collected first sensor data into first semantic information.

With reference to the third aspect, in a possible implementation, the third electronic device sends the first sensor data to the first electronic device, including: the third electronic device converts the first sensor data into first semantic information; the third electronic device converts the first sensor data into first semantic information; The electronic device sends the first semantic information to the first electronic device.

With reference to the third aspect, in a possible implementation manner, sending the first sensor to the first electronic device by the third electronic device includes: sending the first sensor data to the second electronic device by the third electronic device; The device converts the first sensor data into first semantic information; the second electronic device sends the first semantic information to the first electronic device.

In combination with the third aspect, in a possible implementation manner, the data collection system further includes a fourth electronic device, the fourth electronic device, and the second electronic device analyzes the data collection task to determine the first sensor data required in the data collection task , and a third electronic device that provides first sensor data; the first sensor data includes data collected by hardware sensors in the third electronic device and data provided by software in the third electronic device; including: the second electronic device parses data collection tasks , to determine the first sensor data and second sensor data required in the data acquisition task, and the third electronic device providing the first sensor data and the fourth electronic device providing the second sensor data; the first sensor data includes the third electronic device The data collected by the hardware sensor in the device and the data provided by the software in the third electronic device; the second sensor data includes the data collected by the hardware sensor in the fourth electronic device and the data provided by the software in the third electronic device; the first sensor data The type of is different from the type of the second sensor data.

In combination with the third aspect, in a possible implementation manner, the data collection task may include the first collection parameters of the first sensor data required; the first collection parameters include collection duration, collection period, sensor data accuracy, collection algorithm One or more items in the sub-items; the collection duration is used to indicate the duration for the third electronic device to collect the first sensor data; the collection cycle is used to indicate the cycle for the third electronic device to collect the first sensor data; the sensor data accuracy is used to indicate the first Accuracy of sensor data; the acquisition operator is used to indicate the processing method of the first sensor data.

With reference to the third aspect, in a possible implementation manner, the data collection system further includes a fifth electronic device, and the third electronic device determines that the remaining resources in the third electronic device are less than the resources to be used required by the data collection task. In this case, after the third electronic device sends a second message to the second electronic device, after the second message is used to indicate that the third electronic device refuses to accept the data collection task, the data collection method may further include: the second electronic device provides the first sensor The fifth electronic device for data; the second electronic device sends the data collection task to the fifth electronic device.

With reference to the third aspect, in a possible implementation manner, the data collection system further includes a fifth electronic device, and the second electronic device analyzes the data collection task, determines the first sensor data required in the data collection task, and provides the fifth electronic device. A third electronic device with sensor data; the first sensor data includes the data collected by the hardware sensor in the third electronic device and the data provided by the software in the third electronic device, including: the second electronic device analyzes the data collection task and determines the data collection The first sensor data required in the task, and the third electronic device and the fifth electronic device that provide the first sensor data; The data provided; the first sensory data includes the data collected by the hardware sensor in the fifth electronic device and the data provided by the software in the fifth electronic device; when the data collection task indicates that multiple electronic devices perform the data collection task, The second electronic device determines that the third electronic device and the fifth electronic device perform the data collection task; when the data collection task indicates that one electronic device performs the data collection task, the second electronic device selects the third electronic device to perform the data collection task; Or the second electronic device selects the fifth electronic device to perform the data collection task.

With reference to the third aspect, in a possible implementation manner, the second electronic device selects the third electronic device to perform the data collection task, including: the accuracy of the first sensor data provided by the third electronic device is higher than that of the fifth electronic device. In the case of the accuracy of the first sensor data provided by the device, the second electronic device selects the third electronic device to perform the data collection task;

Or, when the computing resources and/or storage resources in the third electronic device are greater than the computing resources and/or storage resources in the fifth electronic device, the second electronic device selects the third electronic device to perform the data collection task;

Or, when the battery power in the third electronic device is greater than the battery power in the fifth electronic device, the second electronic device selects the third electronic device to perform the data collection task;

Or, when the first distance between the third electronic device and the first electronic device is smaller than the second distance between the fifth electronic device and the first electronic device, the second electronic device selects the third electronic device to perform the data collection task;

The second electronic device selects the fifth electronic device to perform data collection tasks, including:

When the accuracy of the first sensor data provided by the third electronic device is lower than the accuracy of the first sensor data provided by the fifth electronic device, the second electronic device selects the fifth electronic device to perform the data acquisition task;

Or, when the computing resources and/or storage resources in the third electronic device are smaller than the computing resources and/or storage resources in the fifth electronic device, the second electronic device selects the fifth electronic device to perform the data collection task;

Or, when the battery power in the third electronic device is less than the battery power in the fifth electronic device, the second electronic device selects the fifth electronic device to perform the data collection task;

Or, when the first distance between the third electronic device and the first electronic device is greater than the second distance between the fifth electronic device and the first electronic device, the second electronic device selects the fifth electronic device to perform the data collection task.

In combination with the third aspect, in a possible implementation manner, the data collection task carries an arbitration flag, and the arbitration flag indicates that the number of times the third electronic device refuses to accept the data collection task exceeds a preset threshold; the third electronic device, based on the data collection task, Before collecting sensor data, the data collection method further includes: the third electronic device sends a third message to the second electronic device, and the third message is used to notify the second electronic device, remaining resources in the third electronic device, and the data collection task Required resources to be used; when the second electronic device determines that the remaining resources in the third electronic device are greater than the resources to be used required by the data collection task, the second electronic device sends a fourth message to the third electronic device, and the second electronic device sends a fourth message to the third electronic device. Four messages are used to instruct the third electronic device to accept the data collection task.

With reference to the third aspect, in a possible implementation manner, the data collection method may further include: when the type of sensor data provided by the data collection system increases, the second electronic device updates the sensor capability information list.

With reference to the third aspect, in a possible implementation manner, the data collection method may further include: the second electronic device establishes a semantic reasoning model based on multiple sensor data in the data collection system; the semantic reasoning model is used to sensor data to obtain semantic information corresponding to one or more sensor data.

In a fourth aspect, a data collection method is provided, and the data collection method may include: the second electronic device receives the data collection task sent by the first electronic device; the second electronic device establishes a communication connection with the first electronic device; the second electronic device The device analyzes the data collection task, determines the first sensor data required in the data collection task, and the third electronic device that provides the first sensor data; the first sensor data includes the data collected by the hardware sensor in the third electronic device, and is provided by the third electronic device The data provided by the software in the electronic device; the second electronic device sends the data collection task to the third electronic device; the second electronic device receives the first sensor data collected by the third electronic device; the second electronic device sends the first sensor data to the first electronic device.

With reference to the fourth aspect, in a possible implementation manner, the second electronic device analyzes the data collection task, determines the first sensor data required in the data collection task, and provides the first sensor data to the third electronic device; the first The sensor data includes the data collected by the hardware sensor in the third electronic device and the data generated by the software in the third electronic device; including: the second electronic device analyzes the data collection task and determines the first sensor data required in the data collection task; The second electronic device queries the third electronic device that provides the first sensor data from the sensing capability information list; the sensing capability information list is used to store the identification of one or more sensor data included in the data acquisition system, and provide sensor The identification of the electronic device of the data; the identification of one or more types of sensor data includes the identification of the first sensor data, and the identification of the electronic device includes the identification of the third electronic device.

With reference to the fourth aspect, in a possible implementation manner, the second electronic device analyzes the data collection task, determines the first sensor data required in the data collection task, and provides the first sensor data to the third electronic device; the first The sensor data includes the data collected by the hardware sensor in the third electronic device and the data generated by the software in the third electronic device; including: the second electronic device parses the data collection task and determines the first semantic information required by the data collection task , the second electronic device determines that the first semantic information is obtained based on the first sensor data; the second electronic device determines that a third electronic device that provides sensor data is found from the sensor capability information list; the sensor capability information list is used to store the The identification of one or more sensor data included in the data acquisition system, and the identification of the electronic device that provides the sensor data; the identification of one or more sensor data includes the identification of the first sensor data, and the identification of the electronic device includes An identification of the third electronic device.

With reference to the fourth aspect, in a possible implementation manner, the identification of the sensor data includes one or more of the name, ID, and type of the sensor data; the identification of the electronic device includes the name of the electronic device and the device ID.

In combination with the fourth aspect, in a possible implementation manner, the second electronic device sends the data collection task to the third electronic device; including: the second electronic device decomposes the data collection task into data collection subtasks, and the data collection subtask It is used to instruct the third electronic device to collect the first sensor data; the second electronic device sends the data collection subtask to the third electronic device.

With reference to the fourth aspect, in a possible implementation manner, the second electronic device decomposes the data collection task into data collection subtasks, and the data collection subtask is used to instruct the third electronic device to collect the first sensor data; including: the second The electronic device determines the first semantic information required for the data collection task, and the first semantic information is obtained based on the sensor data; the second electronic device decomposes the data collection task into a data collection subtask and a semantic conversion subtask, and the data collection subtask It is used to instruct the third electronic device to collect sensor data; the semantic conversion subtask is used to instruct the third electronic device to convert the collected sensor data into first semantic information.

With reference to the fourth aspect, in a possible implementation manner, the second electronic device analyzes the data collection task, determines the first sensor data required in the data collection task, and provides the first sensor data to the third electronic device; the first The sensor data includes the data collected by the hardware sensor in the third electronic device and the data provided by the software in the third electronic device; including:

The second electronic device analyzes the data collection task, determines the first sensor data and the second sensor data required in the data collection task, and the third electronic device that provides the first sensor data and the fourth electronic device that provides the second sensor data; The first sensor data includes the data collected by the hardware sensor in the third electronic device and the data provided by the software in the third electronic device; the second sensor data includes the data collected by the hardware sensor in the fourth electronic device and is provided by the software in the third electronic device Data provided; the first sensor data is of a different type than the second sensor data.

With reference to the fourth aspect, in a possible implementation manner, the data collection task may include the first collection parameters of the first sensor data required; the first collection parameters include collection duration, collection cycle, sensor data accuracy, collection algorithm One or more items in the sub-items; the collection duration is used to indicate the duration for the third electronic device to collect the first sensor data; the collection cycle is used to indicate the cycle for the third electronic device to collect the first sensor data; the sensor data accuracy is used to indicate the first Accuracy of sensor data; the acquisition operator is used to indicate the processing method of the first sensor data.

In a fifth aspect, a data collection system is provided, and the data collection system includes a federation management center and multiple devices. Among them: the federation management center is used to receive the data collection task sent by the first device; the federation management center is used to determine one or more of the sensing data required for the data collection task according to the data collection task and the sensing capability information of multiple devices. Multiple devices; used by the federation management center to instruct one or more devices to provide the required sensory data.

Through the data collection system provided by the embodiment of the present application, the data requesting device only needs to send a data collection task to the central management device, and does not need to know which device can provide the data required for the data collection request, and the central management device can based on the data collection task The data required in Identify the data acquisition devices that can provide that data. The data demand equipment does not need to have a unified data transmission interface with the data acquisition equipment, and can also obtain the data collected by the data acquisition equipment.

In combination with the fifth aspect, in a possible implementation, the federation management center is used to analyze the data collection task, and determine that the sensory data required for the data collection task includes the first sensory data and the second sensory data; the federated management center The center is used to determine the second device providing the first sensing data and the third device providing the second sensing data from multiple devices according to the first sensing data and the second sensing data; the federal management center uses The first collection subtask is sent to the second device, and the second collection subtask is sent to the third device; the first collection subtask is used to instruct the second device to provide the first sensing data, and the second collection subtask is used to instruct the first collection subtask The third device provides second sensory data.

With reference to the fifth aspect, in a possible implementation manner, the second device is configured to accept or reject the first collection subtask; the third device is configured to accept or reject the second collection subtask.

With reference to the fifth aspect, in a possible implementation manner, the second device accepting or rejecting the first collection subtask includes: the second device is configured to: determine to accept or reject the first acquisition subtask based on remaining resources in the second device. A collection subtask.

With reference to the fifth aspect, in a possible implementation manner, the second device is configured to determine to accept or reject the first collection subtask based on remaining resources in the second device, including: When the remaining resources in the second device are greater than the resources to be used required by the first collection subtask, the second device is used to send a first message to the federation management center, and the first message is used to instruct the second device to accept the first collection subtask Task; the remaining resources in the second device include the remaining computing resources and storage resources in the second device; when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask , the second device is used to send a second message to the federation management center, and the second message is used to instruct the second device to refuse to accept the first acquisition subtask.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to instruct one or more devices to provide the required sensing data, including: one or more devices are used to establish data with the first device Transmission channel: one or more devices are used to send the required sensing data to the first device through the data transmission channel.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to send the required sensing data sent by one or more devices to the first device.

With reference to the fifth aspect, in a possible implementation manner, it includes: the federal management center is used to convert the original signal or the second semantic information collected by the sensor sent by one or more devices into the first semantic information, and convert the second semantic information into the first semantic information. A semantic message is sent to the first device.

With reference to the fifth aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask, the second device sends a The second message, the second message is used to indicate that the second device refuses to accept the first collection subtask; including: the federation management center is used to determine to provide the first sensing subtask according to the first collection subtask and the sensing capability information of multiple devices data to the fourth device.

With reference to the fifth aspect, in a possible implementation manner, when the second device determines that the remaining resources in the second device are greater than the resources to be used required by the first acquisition subtask, the second device sends the After the federation management center sends the first message, the first message is used to instruct the second device to accept the first collection subtask, including: the second device is used to collect the first sensory data based on the first collection subtask; the federation The management center determines that the second device is in an abnormal state, and the federal management center determines the fourth device that provides the first sensor data; the abnormal state includes that the battery power of the second device is lower than the power threshold, and the connection between the second device and the federal management center is disconnected , the second device is not in the wearing state, and the second device is disconnected from the power source.

With reference to the fifth aspect, in a possible implementation manner, the data collection task carries an arbitration identifier, and the arbitration identifier is used to indicate that the number of times the fifth device in the one or more devices refuses to accept the data collection task exceeds a preset threshold; federated management The center instructs one or more devices to provide the required sensing data; including: the federal management center is used to instruct the fifth device to provide the required sensing data based on the remaining resources of the fifth device.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to instruct the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center is used to send the fifth device Sending a third message, the required third message is used to instruct the fifth device to send to the federation management center the remaining resources in the fifth device and the resources to be used estimated by the fifth device to perform data collection tasks; the federation management center Receive the remaining resources in the fifth device sent by the fifth device, and the resources to be used that are estimated by the fifth device to perform data collection tasks; if the remaining resources are greater than the required resources to be used, the federation management center instructs The fifth device provides the data required for the data collection task.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to instruct the fifth device to provide the required sensing data based on the remaining resources of the fifth device, including: the federation management center is used to send the fifth device Sending a fourth message, the required fourth message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center; the federation management center is used to receive the remaining resources in the fifth device sent by the fifth device; federation management The center is used to evaluate the resources to be used by the fifth device to perform the data collection task based on the data collection task; when the remaining resources are greater than the required resources to be used, the federation management center is used to instruct the fifth device to provide data collection data required for the task.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to receive the sensing capability information of multiple devices; Atlas to derive new sensory capability information.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to receive the message that the sixth device among the multiple devices joins the data collection system and the device model of the sixth device; For the device model of the device, the sensing capability information of the sixth device is acquired from the first server; different device models and sensing capability information corresponding to the device model are stored in the first server.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is further configured to derive new sensing capability information based on the sensing capability information of the sixth device and the second knowledge graph; the second knowledge graph includes The first knowledge map, the sensing capability information contained in the second knowledge map is more than the sensing capability information contained in the first knowledge map; the federal management center is used to update the sensor capability information stored in the federal management center based on the new sensing capability information sensory information.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to determine that the sixth device exits the data collection system; the federation management center is used to delete the stored sensor capability information of the sixth device, and delete The new sensing capability information derived from the sensing capability information of the device.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to determine that the sixth device exits the data collection system, including: the federation management center is used to send the sixth device if it has not received the sixth device within a preset period of time. In the case of five messages, the federation management center determines that the sixth device exits the data collection system; the fifth message is used to indicate that the sixth device is in the data collection system.

With reference to the fifth aspect, in a possible implementation manner, the federation management center is used to determine that the sixth device exits the data collection system, including: the federation management center is used to receive the sixth message sent by the sixth device, and the sixth message uses In order to instruct the sixth device to exit the data collection system.

With reference to the fifth aspect, in a possible implementation manner, one or more devices further include a seventh device, and the federation management center is configured to send first indication information to the seventh device, and the first indication information is used to indicate When the third sensor data collected by the seventh device is in the first state, the seventh device is used to send the third sensor data to the federation management center; the federation management center is used to receive the third sensor data sent by the seventh device Data; the federation management center is used to send the third sensing data to one or more devices subscribed to the third sensing data.

With reference to the fifth aspect, in a possible implementation manner, after the federation management center is configured to receive the data collection task sent by the first device, the method further includes: the federation management center is used to store data based on the data collection task, and the federation management center The federation management center is used to determine that the federation management center has stored the sensing data required for the data collection task; the federation management center sends the required sensing data to the first device.

With reference to the fifth aspect, in a possible implementation, the sensing capability information includes the sensing data in the device and the accuracy of the sensing data in the device; Sensitivity information is generated; the second knowledge map is stored in the server, and the first knowledge map includes the sensing data of the multiple devices and the association relationship between the sensing data of the multiple devices. When the fourth sensing data and the fifth sensing data can be calculated or derived to obtain the sixth sensing data, it can be said that the fourth sensing data has an associated relationship with the fifth sensing data and the sixth sensing data.

In a sixth aspect, an electronic device is provided, including one or more processors, one or more memories, and a transceiver; wherein, the transceiver, one or more memories are coupled with one or more processors, and one or more The memory is used to store computer program codes, and the computer program codes include computer instructions. When one or more processors are executing computer instructions, the electronic equipment executes: receiving the data collection task sent by the first electronic equipment; There is a communication connection; analyze the data acquisition task, determine the first sensor data required in the data acquisition task, and provide the third electronic device for the first sensor data; the first sensor data includes the data collected by the hardware sensor in the third electronic device, The data provided by the software in the third electronic device; sending the data collection task to the third electronic device; receiving the first sensor data collected by the third electronic device; sending the first sensor data to the first electronic device.

With reference to the sixth aspect, in a possible implementation manner, the electronic device is further configured to perform: parsing the data collection task, determining the first sensor data required in the data collection task; querying from the sensing capability information list to provide The third electronic device for the first sensor data; the sensing capability information list is used to store the identification of one or more sensor data included in the data acquisition system, and the identification of the electronic device providing the sensor data; one or more sensors The identifier of the data includes the identifier of the first sensor data, and the identifier of the electronic device includes the identifier of the third electronic device.

With reference to the sixth aspect, in a possible implementation manner, the electronic device is further configured to perform: parsing the data collection task, determining first semantic information required by the data collection task, and determining the first semantic information based on the first The sensor data is obtained; it is determined that the third electronic device that provides the sensor data is queried from the sensing capability information list; the sensing capability information list is used to store the identification of one or more sensor data included in the data collection system, and The identification of the electronic device that provides the sensor data; the identification of one or more types of sensor data includes the identification of the first sensor data, and the identification of the electronic device includes the identification of the third electronic device.

With reference to the sixth aspect, in a possible implementation manner, the identification of the sensor data includes one or more of the name, ID, and type of the sensor data; the identification of the electronic device includes the name of the electronic device and the device ID.

With reference to the sixth aspect, in a possible implementation manner, the electronic device is further configured to: decompose the data collection task into data collection subtasks, and the data collection subtask is used to instruct the third electronic device to collect the first sensor data; Send the data collection subtask to the third electronic device.

With reference to the sixth aspect, in a possible implementation manner, the electronic device is further configured to: determine the first semantic information required for determining the data collection task, where the first semantic information is obtained based on sensor data; The task is decomposed into a data collection subtask and a semantic conversion subtask, the data collection subtask is used to instruct the third electronic device to collect sensor data; the semantic conversion subtask is used to instruct the third electronic device to convert the collected sensor data into first semantic information .

With reference to the sixth aspect, in a possible implementation manner, the electronic device is further configured to: analyze the data collection task, determine the first sensor data and the second sensor data required in the data collection task, and provide the first sensor data The third electronic device for the data and the fourth electronic device for providing the second sensor data; the first sensor data includes the data collected by the hardware sensor in the third electronic device, and the data provided by the software in the third electronic device; the second sensor data includes The data collected by the hardware sensor in the fourth electronic device and the data provided by the software in the third electronic device; the type of the first sensor data is different from the type of the second sensor data.

In combination with the sixth aspect, in a possible implementation manner, the data collection task may include the first collection parameters of the first sensor data required; the first collection parameters include collection duration, collection period, sensor data accuracy, collection algorithm One or more items in the sub-items; the collection duration is used to indicate the duration for the third electronic device to collect the first sensor data; the collection cycle is used to indicate the cycle for the third electronic device to collect the first sensor data; the sensor data accuracy is used to indicate the first Accuracy of sensor data; the acquisition operator is used to indicate the processing method of the first sensor data.

In a seventh aspect, the present application provides a data collection device, including one or more processors, one or more memories, and a transceiver. The transceiver, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the The data collection device executes the method in any possible implementation manner of the first aspect above.

In an eighth aspect, a data collection device is provided, and the data collection device includes a federation management center module. The federation management center can be used to receive data collection tasks sent by the sensor federation engine module in the data demand device. The federation management center can also send the data collection task to the sensor federation engine module or the data collection module of the data collection device. The federal management center can also receive the data 1 collected by the sensor sent by the data collection device or the semantic information obtained from the data 1 .

In combination with the eighth aspect, in a possible implementation, the federation management center may include a task generation module, a task allocation and scheduling module, a resource access authorization module, a semantic reasoning module, a task parsing module, a device joining/exiting management module, a sensor Capability management module, semantic update module, and semantic fusion and association relationship management module. in:

The task generation module is used to generate data collection tasks.

The task allocation and scheduling module is used to allocate data acquisition tasks to one or more data acquisition devices.

The resource access authorization module can be used to record authorized resources (for example, computing resources, storage resources, etc.) and sensor capabilities in each data acquisition device. The resource access authorization module can also be used to authorize the resource access of the data demand device, that is, authorize the data demand device to use which resources of the data collection device (for example, computing resources, storage resources, etc.).

The semantic inference module can be used to infer semantic information based on raw data collected by sensors, or infer new semantic information (also called high-order semantic information) based on multiple semantic information (also called low-order semantic information). It can be understood that, in the embodiment of the present application, high-level semantic information and low-level semantic information are relative concepts. In the embodiment of the present application, a plurality of semantic information that can be inferred or generated as new semantic information is referred to as low-level semantic information, and new semantic information generated from the plurality of semantic information is referred to as high-level semantic information. For example, semantic information A and semantic information B can be deduced or generated semantic information C. Then semantic information A and semantic information B can be called low-order semantic information, and semantic information C can be called high-order semantic information.

The task parsing module can be used to parse data collection tasks.

The equipment joining/exiting management module can be used to manage the joining and exiting of equipment in the data acquisition system.

The sensing capability management module can be used to manage the sensing capability information of each device in the data acquisition system. In this embodiment of the present application, the sensing capability information may include semantic information that can be provided by the device, the precision of the semantic information, raw data collected by the sensor, the precision of the raw data, and the like.

The semantic update module can be used to update semantic information and store the updated semantic information.

The semantic fusion and relationship management module can be used for the fusion of semantic information, knowledge fusion and management of the relationship between semantic information. In the embodiment of the present application, if semantic information A and semantic information B can derive semantic information C, then it can be said that semantic information A and semantic information B have an association relationship.

In a ninth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and when the instructions are run on a computer, the computer is made to execute any possible implementation of any one of the first to fourth aspects above. methods in methods.

In a tenth aspect, the present application provides a computer program product, which, when running on a computer, causes the computer to execute the method in any one possible implementation manner of the first aspect to the fourth aspect above.

Description of drawings

FIG. 1 is a schematic diagram of the architecture of a data acquisition system 10 provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a scene of a data acquisition system 10 provided in an embodiment of the present application;

Fig. 3 is the software architecture diagram of data demand equipment 20, federation management center 30, data acquisition equipment 40 in the data acquisition system 10 that the embodiment of the application provides;

Fig. 4 is a schematic diagram of the software architecture of the sensor federation engine provided by the embodiment of the present application;

Fig. 5 is a schematic diagram of the software architecture of the federal management center provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of a knowledge graph of semantic information provided by an embodiment of the present application;

FIG. 7A is a schematic diagram of a user interface 70A of a mobile phone 104 provided in an embodiment of the present application;

FIG. 7B is a schematic diagram of a user interface 70B of a mobile phone 104 provided in an embodiment of the present application;

FIG. 7C is a schematic diagram of a user interface 70C of a mobile phone 104 provided in an embodiment of the present application;

FIG. 7D is a schematic diagram of a user interface 70D of a mobile phone 104 provided in an embodiment of the present application;

FIG. 7E is a schematic diagram of a user interface 70E of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8A is a schematic diagram of a user interface 80A of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8B is a schematic diagram of a user interface 80B of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8C is a schematic diagram of a user interface 80C of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8D is a schematic diagram of a user interface 80D of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8E is a schematic diagram of a user interface 80E of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8F is a schematic diagram of a user interface 80F of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8G is a schematic diagram of a user interface 80G of a tablet 105 provided in an embodiment of the present application;

FIG. 8H is a schematic diagram of a user interface 80H of a mobile phone 104 provided in an embodiment of the present application;

FIG. 8I is a schematic diagram of a user interface 80I of a mobile phone 104 provided in an embodiment of the present application;

FIG. 9A is a schematic diagram of a user interface 90A of a tablet 105 provided in an embodiment of the present application;

FIG. 9B is a schematic diagram of a user interface 90B of a tablet 105 provided in an embodiment of the present application;

FIG. 9C is a schematic diagram of a user interface 90C of a tablet 105 provided in an embodiment of the present application;

FIG. 9D is a schematic diagram of a user interface 90D of a mobile phone 104 provided in an embodiment of the present application;

FIG. 9E is a schematic diagram of a user interface 90E of a tablet 105 provided in an embodiment of the present application;

FIG. 9F is a schematic diagram of a user interface 90F of a tablet 105 provided in an embodiment of the present application;

FIG. 9G is a schematic diagram of a user interface 90G of a tablet 105 provided in an embodiment of the present application;

FIG. 10 is a schematic flow chart of a data collection method provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a data acquisition task processing flow provided by an embodiment of the present application;

FIG. 12 is a schematic flow diagram of determining a target data collection device provided by an embodiment of the present application;

Fig. 13A is a schematic flow diagram of how the data collection device provided in the embodiment of the present application determines whether to accept the data collection task;

FIG. 13B is a schematic flowchart of how to determine whether the data collection device accepts the data collection task in the arbitration mode provided by the embodiment of the present application;

Fig. 13C is a schematic flow diagram of how the data collection device provided in the embodiment of the present application determines whether to accept the data collection task;

FIG. 14A is a schematic diagram of specific steps for establishing a data transmission channel between the data acquisition device 40 and the data demand device 20 provided by the embodiment of the present application;

FIG. 14B is a schematic diagram of specific steps for establishing a data transmission channel between the federal management center 30 and the data demand device 20 and the data collection device 40 provided by the embodiment of the present application;

FIG. 15A is a schematic diagram of specific steps for the data collection device 40 to send the collected sensor data to the data demand device 20 through the federation management center 30 provided by the embodiment of the present application;

Fig. 15B is a schematic diagram of specific steps of how the data collection device 40 provided by the embodiment of the application and how the federation management center 30 handles an abnormal event in the data collection device 40;

FIG. 15C is a schematic diagram of the processing flow of the collected sensor data by the data collection device 40 provided in the embodiment of the present application;

FIG. 16 is a schematic diagram of a specific application scenario of a data acquisition system provided by an embodiment of the present application;

FIG. 17 is a schematic diagram of a specific software model implementation process of the data collection scene provided by the embodiment of the present application;

FIG. 18 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also Plural expressions are included unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in this application refers to and includes any and all possible combinations of one or more of the listed items.

Hereinafter, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, unless otherwise specified, the "multiple" The meaning is two or more.

First, a data collection system provided by the embodiment of the present application is introduced.

FIG. 1 exemplarily shows a data acquisition system 10 provided by an embodiment of the present application. As shown in Figure 1, the data collection system includes one or more data demand devices, a federation management center and one or more data collection devices. In the data collection system 10, the federation management center may be in the data demand device or in the data collection device. Or the federation management center can also be located in an electronic device other than the data demand device and the data collection device, or a cloud server or server. Or the federation management center can be deployed in multiple electronic devices in a distributed manner, that is, multiple electronic devices can serve as the federation management center in the data collection system 10 together.

In the data acquisition system 10, the data demand equipment, the data acquisition equipment, and the electronic equipment or cloud server where the federation management center is located, and the server can establish communication through wireless local area networks (wireless local area networks, WLAN), or cellular networks, bluetooth, and distributed soft buses, etc. connect. The embodiment of the present application does not limit the data demand device, the data collection device, and the electronic device as the federation management center, or the cloud server, or the way in which the server establishes a communication connection in the data collection system 10 .

It is understandable that the distributed soft bus can be an upper-level software communication bus system that supports various wired and wireless communication protocols, and is a communication base for electronic devices such as mobile phones, tablets, smart wearables, smart screens, and cars. Interconnection between devices provides communication capabilities.

Specifically, the data demand device can establish a communication connection with the electronic device or cloud server or server where the federation management center is located. The data demand device and the electronic device or cloud server as the federal management center can establish communication through wireless local area network (wireless local area network, WLAN), cellular network, Bluetooth or distributed soft bus or universal serial bus (universal serial bus, USB) connection line, etc. connect. The embodiment of the present application does not limit the specific manner in which the data demand device establishes a communication connection with the electronic device, cloud server, or server where the federation management center is located.

The data collection device can also establish a communication connection with the electronic device or cloud server or server where the federal management center is located. The data acquisition equipment and the electronic equipment or cloud server where the federal management center is located, and the server can establish a communication connection through WLAN, cellular network, Bluetooth or distributed soft bus, USB cable, etc. The embodiment of the present application does not limit the specific manner in which the data collection device establishes a communication connection with the electronic device, cloud server, or server where the federation management center is located.

The data demand device can establish a communication connection with the data demand device through the device where the federation management center is located. Optionally, the data demand device may also directly establish a communication connection with the data collection device. A communication connection can be established between the data demand device and the data acquisition device through WLAN, cellular network, Bluetooth or distributed soft bus, USB cable, etc. This embodiment of the present application does not limit it.

Wherein, the data demand device may be used to send a data request task, and obtain sensor data collected by one or more data collection devices based on the data request task or semantic information obtained from the sensor data.

The federal management center can be used to determine whether there is a data collection device that meets the data collection request based on the data collection request sent by the data demand device.

The federal management center can also be used to process the data collected by the sensors in the data collection device to obtain semantic information corresponding to the data collected by the sensors.

The data collection device can be used to collect raw sensor data based on the data collection task, or process the collected raw sensor data into semantic information.

In a possible implementation manner, the data demand device may send a data collection request to the federation management center, and the federation management center may send the data collection request to one or more data collection devices. The data collection device can collect data based on the data collection request. The data collection device can send the collected data to the data demand device through the federation management center.

In another possible implementation manner, the data demand device may directly send the data collection request to the data collection device. The data collection device collects data based on the data collection request, and sends the collected data to the data demand device.

Sensors in data demanding devices and the capabilities of sensors are limited. The data-requiring device may perform certain tasks or realize certain functions, but the data-requiring device cannot provide sensor data that meets the requirements of these tasks or functions. For example, smart air conditioners, smart TVs, smart washing machines and other equipment. The federation management center module can be deployed in one or more electronic devices (such as smart phones, computers, tablets, etc.) or cloud servers or servers with strong data processing capabilities and computing capabilities. A data collection device may be a device with sensors that can collect data. A data acquisition device can also be a device consisting of only a single sensor. For example, smart cameras, temperature sensors, and more. It can be understood that the embodiment of the present application does not limit the data demand device, the device where the federation management center is located, and the data collection device.

It can be understood that, for a single electronic device in the data collection system, as the initiator of the data collection task changes, the role of the electronic device in the data collection system (ie: data demand device, federation management center, data acquisition equipment) may also change accordingly. In the data collection system 10, the federation management center can be deployed in multiple electronic devices in a distributed manner. How multiple electronic devices form a data acquisition system will be described in detail below, and details will not be repeated here.

It can be understood that, in some embodiments, the data demand device, the electronic device where the federation management center is located, and the data collection device can all be called electronic devices or terminals. The embodiment of the present application does not limit the names of the data demand device, the electronic device where the federation management center is located, and the data collection device.

In the embodiment of the present application, the above-mentioned data collection system 10 may be referred to as a sensor federation, and the sensor federation is used to fuse, schedule, and manage sensor data distributed in multiple devices. A sensor federation is composed of multiple electronic devices. In a sensor federation, each electronic device can provide data of one or more sensors in the electronic device (the specific number and specific sensors can be set by the electronic device). The sensor federation can fuse, schedule and manage the sensor data of multiple electronic devices in the sensor federation. The sensor data may be raw data collected by a sensor in the electronic device or data obtained by processing the raw data. The sensing data may be data acquired or calculated by software, or data of hardware sensors.

In this embodiment of the present application, the semantic information may be information that is identifiable and processable by an electronic device obtained by processing raw data collected by a sensor or data obtained after processing the raw data. For example, the original temperature waveform signal collected by the temperature sensor can be processed into semantic information that can be understood and applied by electronic equipment (for example, 26°C (Celsius)).

It can be understood that the raw data collected by sensors can be processed into different semantic information due to different constraints such as scenarios and application requirements. For example, the measurement result provided by the temperature sensor may be treated as "indoor temperature" or "outdoor temperature" according to the deployment location of the temperature sensor. For another example, the image signal provided by the camera can be processed into different semantic information such as "identity" or "age" according to different application requirements. It is understandable that sensor data of different models, precisions, and types may also be processed into the same semantic information. For example, the voice signal collected by the microphone can be processed as the semantic information of "identity", and the image signal collected by the camera can also be processed as the semantic information of "identity".

It can be understood that, in some embodiments, the semantic information not only includes information directly applicable to the electronic device, but also includes metadata used to describe the semantic information. For example, the metadata can be the type, attribute, precision, Constraints and other information. The metadata can assist electronic devices to process and apply sensory data more efficiently.

In the embodiment of the present application, the above-mentioned semantic information, sensor data, raw data collected by the sensor, raw signal, etc. are collectively referred to as sensory data.

It can be understood that the above-mentioned data collection system 10 shown in FIG. or multiple users' electronic devices to form a data acquisition system), or other scenarios. The frequency of increase or decrease in the number of electronic devices in the data collection system in different scenarios, as well as the frequency of replacement of electronic devices serving as the federal management center in the data collection system are different. For example, in the household scenario, the frequency of increase or decrease of the number of electronic devices in the data collection system is low (maybe no new electronic devices will be added for a long time), and the frequency of replacement of electronic devices serving as the federal management center is low . In office scenarios, the number of electronic devices in the data collection system increases or decreases frequently, and the frequency of replacement of electronic devices serving as the federal management center is also high. The embodiment of the present application does not limit the usage scenarios of the data acquisition system 10 .

The data collection system 10 shown in FIG. 1 will be described below in a specific scenario. Taking the personal home scene as an example, electronic devices such as air conditioners, tablets, mobile phones, watches, cameras, air quality sensors, etc. in a user's home can be combined into a data acquisition system.

Fig. 2 exemplarily shows a data collection system in a home scene. As shown in FIG. 2 , the data collection system in this home scene may include an air conditioner 101 , a camera 102 , a watch 103 , a mobile phone 104 , a tablet 105 , and an air quality sensor 106 . The air conditioner 101 can be used as a data demand device in the data collection system. The camera 102, watch 103, tablet 105, and air quality sensor 106 can all collect data. Camera 102, watch 103, tablet 105, and air quality sensor 106 can all be used as data collection devices in the data collection system. For example, the camera 102 may collect data such as user identity (the elderly, children, young and middle-aged people, etc.), user location, and the like. The watch 103 can collect data such as user status (sleep, work and study, exercise, etc.), and user body temperature. Tablet 105 can collect data such as user status (sleep, work, study, exercise, etc.), user identity (old people, children, young and middle-aged people, etc.), user health status. The air quality sensor 106 can collect data such as air pressure, humidity, and PM2.5 (fine particulate matter in the air) of indoor and outdoor air. Wherein, the mobile phone 104 can be used as the federal management center in the data collection system. The air conditioner 101 can obtain data collected by electronic devices such as a camera 102 , a watch 103 , a mobile phone 104 , a tablet 105 , and an air quality sensor 106 through a mobile phone 104 . The air conditioner 101 can adjust the working mode of the air conditioner based on the data collected by electronic devices such as the camera 102 , the watch 103 , the tablet 105 , and the air quality sensor 106 . In this way, the air conditioner 101 does not need a presence sensor, and can also acquire user information or environmental conditions that can intelligently adjust the working mode of the air conditioner.

Optionally, in a possible implementation manner, before one device in the data collection system obtains user information and sensing data of another device, it needs to obtain authorization from a user of another device.

Exemplarily, a frame diagram of the data demand device, the federation management center, and the data collection device may be as shown in FIG. 3 . Fig. 3 exemplarily shows a schematic diagram of the software framework of the data demand device 20, the federation management center 30 and the data collection device 40 in the data collection system 10.

As shown in FIG. 3 , the data demand device 20 may include an application layer 201 and a sensor federation engine module 202 . Wherein, the application layer 201 may include one or more application programs, for example, application program APP2011, application program APP2012 and other application programs. The embodiment of the present application does not limit the number of application programs included in the data demand device 20 and the specific types of application programs. An application program of the application layer 201 in the data demand device 20 may initiate a data collection request. Then, the sensor federation engine module 202 may generate a data collection task from the data collection request. The sensor federation engine module 202 may send the data collection task to the federation management center 30 .

As shown in FIG. 3 , the federation management center 30 may receive the data collection task sent by the sensor federation engine module 202 in the data demand device 20 . The federation management center 30 may also send the data collection task to the sensor federation engine module 401 or the data collection module 402 of the data collection device 40 . The federal management center 30 can also receive the data 1 collected by the sensor sent by the data collection device 40 or the semantic information obtained from the data 1 . Specifically, the federation management center 30 may receive the data 1 collected by the sensor or the semantic information obtained from the data 1 sent by the sensor federation engine module 401 or the data collection module 402 in the data collection device 40 . The federation management center 30 can send the received data 1 to the sensor federation engine module 202 in the data demand device 20 . Alternatively, the federation management center 30 may convert the received data 1 into semantic information, and then send the semantic information to the sensor federation engine module 202 .

As shown in FIG. 3 , the data collection device 40 may include a data collection module 402 . Optionally, the data collection device 40 may also include a sensor federation engine module 401 . It can be understood that, in some data collection devices 40, the sensor federation engine module 401 may not be included. The data collection module 402 can directly receive the data collection task sent by the federation management center 30 , or receive the data collection task sent by the federation management center 30 through the sensing federation engine module 401 . The data collection module 402 can send the collected data 1 to the federation management center 30 based on the data collection task, or send the collected data 1 to the sensor federation engine module 401, and the sensor federation engine module 401 converts the data 1 into The corresponding semantic information is then sent to the federal management center 30 .

Optionally, the data collection device 40 may directly send the collected data 1 or the semantic information obtained from the data 1 to the data demand device 20 . Specifically, the sensor federation engine module 401 in the data collection device 40 can directly send the data 1 collected by the data collection module 402 or the semantic information obtained from the data 1 to the data demand device 20 .

It can be understood that the data collection module 402 may be a specific hardware sensor in the data collection device 40, or some software modules or algorithms for collecting data in the data collection device 40, which is not limited in this embodiment of the present application.

As shown in FIG. 3 , the federation management center 30 may receive the data 1 sent by the data collection module 402 , and then convert the data into corresponding semantic information and send it to the sensor federation engine module 202 .

Optionally, in another possible implementation manner, the federation management center 30 may receive the semantic information sent by the sensing federation engine module 401 .

The federation management center 30 can send the semantic information to the sensing federation engine module 202 . The sensor federation engine module 202 can send the semantic information to the application program in the data demand device 20 that initiates the data collection request.

The specific functions of the sensor federation engine module 202 and the sensor federation engine module 401 will be introduced in detail in FIG. 4 , and will not be repeated here. The specific functions of the federation management center 30 will be introduced in detail in FIG. 5 and will not be repeated here.

It can be understood that the data demand device 20 and the data collection device 40 may have more or less modules, which is not limited in this embodiment of the present application.

Fig. 4 exemplarily shows a schematic diagram of functional modules of the sensor federation engine module. As shown in FIG. 4 , the sensor federation engine module may include a subscription and query module 501 , a model exploration module 502 , a data analysis module 503 , a management module 504 , a data storage module 505 , and a data access module 506 . in:

The subscription and query module 501 is configured to receive query requests, and send query information based on the query requests. The subscription and query module 501 can also be used for subscribing events, and sending notifications for subscribing events based on subscribing events.

The model exploration module 502 may include a task generation module 5021 , a task analysis module 5022 , a task management module 5023 and a task execution module 5024 . Among them: the task generation module 5021 can be used to generate data collection tasks. The data collection task may include information such as the type of data to be collected, accuracy, real-time data or historical data, and collection operators. The task parsing module 5022 can be used for parsing the received data collection tasks, and determining the resources needed for the data collection tasks (such as computing resources, storage resources, etc.) and the like. The task management module 5023 can be used to start a data collection task, suspend a data collection task, and cancel a data collection task. The task execution module 5024 can be used to execute data collection tasks.

The data analysis module 503 may include a portrait analysis module 5031 , a short-term memory module 5032 , and a perception analysis module 5033 . Among them, the portrait analysis module 5031 can form user portrait information (such as , user's age, user's preference, user's sleep condition, etc.). The portrait analysis module 5031 can also store user portrait information. The short-term memory module 5032 can be used to obtain semantic information based on historical data, and store the semantic information; and store the semantic information obtained by the perceptual analysis module. The perception analysis module 5033 can obtain semantic information based on the real-time data collected by the data access module 506 .

The management module 504 may include a device discovery module 5041 and a task negotiation and decision module 5042 . Among them, the device discovery module 5041 can be used to discover devices, join or exit the data collection system. The task negotiation and decision-making module 5042 is used to interact with other devices to collect task information, and determine whether to accept the data collection task sent by another device according to the working status of the device.

The data storage module 505 can be used to store data, and can also be used to receive and store data sent by other devices and data sent by the data access module 506 .

The data access module 506 can be used to receive data sent by other devices.

It can be understood that the sensor federation engine module shown in FIG. 4 is only an example, and the sensor federation engine module may include more or less modules, which is not limited in this embodiment of the present application. The modules shown in FIG. 4 may also have other names in other scenarios, and this embodiment of the present application does not limit the names of the modules in the sensor federation engine module.

In a possible implementation manner, the sensor federation engine module can be elastically and tailorably installed in an electronic device. The electronic device can download one or more modules in the sensor federation engine module based on the computing capability and storage capability of the electronic device. For example, all the modules in the sensing federation engine module shown in FIG. 4 above can be installed in the mobile phone. The device discovery module 5041 and the data access module 506 in the sensor federation engine module shown in FIG. 4 above can be installed in the air conditioner.

FIG. 5 exemplarily shows a schematic diagram of functional modules of the federation management center 30 . As shown in Figure 5, the federation management center 30 may include a task generation module 601, a task allocation and scheduling module 602, a resource access authorization module 603, a semantic reasoning module 604, a task analysis module 605, a device joining/exiting management module 606, a sensing capability A management module 607 , a semantic update module 608 , and a semantic fusion and association relationship management module 609 . in:

The task generation module 601 is used to generate data collection tasks. The specific data collection task may refer to the description above, and the description below of a specific example of the data collection task in a specific scenario, which will not be repeated here.

The task assignment scheduling module 602 is used to assign data collection tasks to one or more data collection devices.

The resource access authorization module 603 may be used to record authorized resources (for example, computing resources, storage resources, etc.) and sensor capabilities in each data collection device. The resource access authorization module 603 can also be used to authorize resource access to the data demand device, that is, authorize the data demand device to use which resources of the data collection device (for example, computing resources, storage resources, etc.).

The semantic reasoning module 604 can be used to infer semantic information based on raw data collected by sensors, or infer new semantic information (also called high-level semantic information) based on multiple semantic information (also called low-level semantic information). It can be understood that, in the embodiment of the present application, high-level semantic information and low-level semantic information are relative concepts. In the embodiment of the present application, a plurality of semantic information that can be inferred or generated as new semantic information is referred to as low-level semantic information, and new semantic information generated from the plurality of semantic information is referred to as high-level semantic information. For example, semantic information A and semantic information B can be deduced or generated semantic information C. Then semantic information A and semantic information B can be called low-order semantic information, and semantic information C can be called high-order semantic information.

The task parsing module 605 can be used for parsing data collection tasks.

The device joining/exiting management module 606 can be used to manage joining and leaving of devices in the data collection system.

The sensing capability management module 607 can be used to manage the sensing capability information of each device in the data collection system. In this embodiment of the present application, the sensing capability information may include semantic information that can be provided by the device, the precision of the semantic information, raw data collected by the sensor, the precision of the raw data, and the like.

The semantic update module 608 may be used to update the semantic information and store the updated semantic information.

The semantic fusion and relationship management module 609 can be used for the fusion of semantic information, knowledge fusion and management of the relationship between semantic information. In the embodiment of the present application, if semantic information A and semantic information B can derive semantic information C, then it can be said that semantic information A and semantic information B have an association relationship.

In some examples, knowledge fusion may refer to fusing multiple semantic knowledge graphs corresponding to certain semantic information into one semantic knowledge graph. Exemplarily, as shown in FIG. 6 , (a) in FIG. 6 is a semantic knowledge graph corresponding to semantic information A, in which semantic information A and semantic information B can obtain semantic information C. (b) in Fig. 6 is another semantic knowledge graph corresponding to semantic information A. In this semantic knowledge graph, semantic information E can be obtained from semantic information A and semantic information D. (c) in FIG. 6 is a semantic knowledge graph obtained by fusing the semantic knowledge graph in (a) in FIG. 6 and the semantic knowledge graph in (b) in FIG. 6 .

In some examples, semantic fusion may refer to fusing multiple semantic information (low-order semantic information) into new semantic information (high-order semantic information).

It can be understood that the functional modules of the federation management center 30 shown in FIG. 5 are only examples, and the federation management center may include more or fewer modules, which is not limited in this embodiment of the present application. The modules shown in FIG. 5 may also have other names in other scenarios, and the embodiment of the present application does not limit the names of the modules in the federation management center 30 .

In a possible implementation manner, the federation management center 30 may be deployed in electronic devices such as mobile phones, tablets, and computers, or in cloud servers or servers.

In a possible implementation manner, the federation management center 30 may be installed in electronic devices in a distributed manner. The electronic device can download one or more modules in the federation management center based on the computing capability and storage capability of the electronic device. For example, in some scenarios, all the modules in the federation management center 30 shown in FIG. 5 above can be installed in the mobile phone. In other scenarios, the task generation module 601, task allocation and scheduling module 602, resource access authorization module 603, task analysis module 605, device joining/exit management module 606, and transmission The sensory ability management module 607 can be installed in the mobile phone. The semantic reasoning module 604, the semantic update module 608, and the semantic fusion and association relationship management module 609 in the federation management center 30 can be installed in the tablet.

Embodiments of the present application provide a data collection method, system, and related equipment. The data collection method can be applied to a data collection system, the data collection system includes a plurality of devices and a federation management center, the method includes: the federation management center can receive the data collection task sent by the first device among the multiple devices; the federation management According to the data collection task and the sensing capability information of multiple devices, the center determines one or more devices that provide the sensing data required by the data collection task; the federation management center instructs one or more devices to provide the required sensing data. Through the data collection method provided by the embodiment of the present application, multiple (two or more) electronic devices can establish a data collection system, and one electronic device in the data collection system can obtain the raw data collected by the sensor of another electronic device, Or semantic information obtained from raw data collected by sensors.

The following first introduces in detail how to build a data acquisition system with multiple electronic devices.

The federation management center can create a data collection system (also referred to as a sensor federation in this embodiment of the application, which will be referred to as a sensor federation hereinafter). After the electronic device or cloud server or server where the federation management center is located creates a sensing federation, other devices (such as data demand devices and data collection devices) can join the sensing federation.

In some scenarios, the electronic device or cloud server or server where the federation management center is located can form a sensory federation from multiple electronic devices logged in with the same user account. No user operation is required, and the user's electronic device can automatically join the sensor federation created for the account by the federation management center after logging in to the account.

In some scenarios, when electronic device A and electronic device B are connected, one electronic device of electronic device A and electronic device B may create a sensing federation. The electronic device A and/or the electronic device B may display a prompt box, which may be used to prompt the user whether to agree to the sensor federation, and if the user clicks a control for agreeing to create the sensor federation, the sensor federation can be successfully created.

In a possible implementation manner, the federation management center may be deployed in a server or a cloud server (the cloud server is used as an example for illustration below). When the user's electronic device logs into the user account, the electronic device can send the user account information of the electronic device to the federation management center in the cloud server, and the federation management center in the cloud server can determine whether the user account has a corresponding sensor federation; If not, the federation management center in the cloud server creates a sensing federation for the user account; if there is, the federation management center in the cloud server obtains the device information (for example, the device ID, device model, etc.), and based on the device information of the electronic device, add the electronic device to the sensor federation corresponding to the user account.

Specifically, when a user account logs in on the electronic device for the first time, the federation management center in the cloud server has not yet created a sensor federation corresponding to the user account. The federation management center in the cloud server cannot find the sensor federation corresponding to the user account, and the cloud server can create a sensor federation for the user account. Then, the cloud server adds the electronic device to the sensing federation.

It can be understood that the multiple electronic devices in the sensing federation may come from different device manufacturers. Each electronic device that can join the sensory federation can have a standard interface for transmitting data. In this way, data can also be transmitted between electronic devices of different manufacturers in the sensor federation.

It can be understood that the user account can be used by the electronic device to create a sensor federation or join a sensor federation. The user account may be a login account of the electronic device, for example, a Huawei account, a Hongmeng account, and the like.

Taking the mobile phone 104 shown in FIG. 2 as an example, how the federation management center establishes the sensor federation corresponding to the user account of the mobile phone 104 and adds the mobile phone 104 to the sensor federation corresponding to the user account.

Exemplarily, FIG. 7A-FIG. 7E show a cloud server or a schematic diagram of the federation management center in the server side creating a sensor federation and the user's electronic device joining the sensor federation created by the federation management center.

As shown in FIG. 7A , FIG. 7A exemplarily shows a user interface 70A for logging in a user account by the mobile phone 104 . The user interface 70A may include control 7000 , text 7001 and text 7002 , input field 7003 , input field 7004 , text 7005 , text 7006 , control 7007 , and control 7008 . Control 7000 can be used to return to the previous interface. Text 7001 may be used to prompt the user that the user interface is used to log in the user account. The content of the text 7001 may be "user account", and the specific content of the text 7001 is not limited here. The text 7002 can be used to prompt the user to log in the user account to use the service. The content of the text 7002 may be "login the account to use the sensor federation service", and the specific content of the text 7002 is not limited here. The input field 7003 can be used to input user account number, for example "131****7837". The input column 7004 is used to input a password, such as "...". Text 7005 can be used to prompt the user for other login methods. The text 7005 may be "SMS verification code login", and the specific content of the text 7005 is not limited here. Text 7006 can be used for user to retrieve password. The text 7006 may be "forgot password", and the specific content of the text 7006 is not limited here. Control 7007 may be used to log in a user account. Control 7008 may be used to register a user account.

After the user enters the account number in the input field 7003 and the password in the input field 7004, the user can click the control 7007, and in response to the user operation, the mobile phone 104 can successfully log in the user account "131****7837". The mobile phone 104 can send a message to the federal management center, and the message can carry the user account "131****7837". Based on the message, the federation management center can query whether a sensor federation corresponding to the user account "131****7837" has been established in the federation management center. If there is no sensor federation corresponding to the user account "131****7837" in the federation management center, the federation management center will create a sensor federation corresponding to the user account "131****7837". If the sensor federation corresponding to the user account "131****7837" has been established in the federation management center, then the federation management center inquires whether the mobile phone 104 has joined the sensor federation, if not, invites the mobile phone 104 to join the federation The sensor federation corresponding to the user account "131****7837".

In a possible example, if there is no sensor federation corresponding to the user account "131****7837" in the federation management center, the federation management center may send a request message to the mobile phone 104, and the request message is used to request to establish a user account The sensor federation corresponding to "131****7837". The mobile phone 104 may receive the request message and display the user interface 70B.

As shown in FIG. 7B , FIG. 7B exemplarily shows a user interface 70B of the mobile phone 104 . The user interface 70B may include a prompt message 7101 and a bullet box 7102 . Wherein, the prompt message 7101 may be used to prompt the user to log in the user account successfully. The prompt message 7101 may be "login successful", and the specific content of the prompt message 7101 is not limited here. The bullet box 7102 may include a control 7103 and a control 7104 . The user can click the control 7103 to agree to the federation management center to establish a sensor federation corresponding to the user account "131****7837". The user can click on the control 7104, which is used to reject the establishment of the sensor federation corresponding to the user account "131****7837" by the federation management center.

After the user clicks the control 7103, the mobile phone 104 may send a notification message agreeing to establish a sensor federation corresponding to the user account "131****7837" to the federation management center. Moreover, the mobile phone 104 may also send the device information (eg, device ID, device model, etc.) of the mobile phone 104 to the federation management center. The federation management center may establish a sensor federation corresponding to the user account "131****7837" based on the notification message, and add the mobile phone 104 to the sensor federation. After the federal management center establishes the sensor federation corresponding to the user account "131****7837", it can send a message to the mobile phone 104. The message can be used to prompt the mobile phone 104 that the user account "131****7837" has been established. sensor federation. After receiving the message, mobile phone 104 may display user interface 70C.

As shown in FIG. 7C , FIG. 7C exemplarily shows a user interface 70C of the mobile phone 104 . The user interface 70C may include a message prompt box 7301 . The message prompt box 7301 is used to remind the user that the federation management center has successfully created the sensor federation corresponding to the user account "131****7837". The content in the message prompt box 7301 may be "the sensor federation of account 131****7837 is successfully created, and your device can enjoy the sensor federation service", and the specific content of the message prompt box 7301 is not limited here.

In another possible example, after the user clicks the control 7007 on the user interface 70A of the mobile phone 104, the federation management center determines that a sensor federation corresponding to the user account "131****7837" has been established, but the mobile phone 104 It has not joined the sensor federation corresponding to the user account "131****7837". The federation management center may send a request to the mobile phone 104, and the request is used to request the mobile phone 104 to join the sensor federation corresponding to the user account "131****7837". After receiving the request, the mobile phone 104 may display the user interface 70D.

As shown in FIG. 7D , FIG. 7D exemplarily shows a user interface 70D of the mobile phone 104 . The user interface 70D may include a prompt message 7401 and a bullet box 7102. Wherein, the prompt message 7401 may be used to prompt the user to log in the user account successfully. The prompt message 7401 may be "login successful", and the specific content of the prompt message 7401 is not limited here. The bullet box 7402 may include a control 7403 and a control 7404 . The user can click the control 7403 to agree to join the sensor federation corresponding to the user account "131****7837". The user can click on the control 7404 to refuse to join the sensor federation corresponding to the user account "131****7837".

After the user clicks the control 7403, the mobile phone 104 may send a notification message agreeing to join the sensor federation corresponding to the user account "131****7837" to the federation management center. After receiving the notification message, the federation management center can add the mobile phone 104 to the sensor federation corresponding to the user account "131****7837". Optionally, the federation management center may also send a notification message to the mobile phone 104 indicating that the mobile phone 104 has been successfully added to the sensor federation corresponding to the user account "131****7837". After the mobile phone 104 receives the notification message, the mobile phone 104 can display the user interface 70E.

As shown in FIG. 7E , FIG. 7E exemplarily shows a user interface 70E of the mobile phone 104 . The user interface 70E may include a message prompt box 7501 . The message prompt box 7501 is used to remind the user that the federation management center has added the mobile phone 104 to the sensor federation corresponding to the user account "131****7837". The content in the message prompt box 7501 can be "Your device has successfully joined the sensor federation of account 131****7837, and can enjoy the sensor federation service", here is the specific content of the message prompt box 7501 Not limited.

It can be understood that the user interface shown in the above-mentioned Figures 7A-7E is only an example, and the user interface shown in the above-mentioned Figures 7A-7B may contain more or less interface elements, or The position on the user interface may vary, which is not limited in this embodiment of the present application.

In some possible examples, some electronic devices do not have user-operable display screens, eg, air conditioners, earphones, temperature sensors, and the like. Users cannot log in user accounts on these electronic devices that do not have user-operable display screens, but electronic devices such as air conditioners, earphones, and temperature sensors can communicate with other types of electronic devices that have user-operable functions through wireless local area networks, Bluetooth, or distributed soft buses. An electronic device (for example, a mobile phone, a computer, a tablet, etc.) that operates a display screen establishes a communication connection. When the user logs in the account on the electronic device A with an operable display screen, the electronic device A can also send the device information of the electronic device B without an operable display screen that establishes a communication connection with the electronic device A to the federation management center. The federation management center may send a request to electronic device A for adding electronic device A and electronic device B to the sensor federation. The user can click on the electronic device A to agree to add electronic device A and electronic device B to the control of the sensor federation. In response to the user operation, electronic device A can send the consent to join the electronic device A and electronic device B to the federation management center Notification message to sensor federation. After the federation management center receives the notification message, the federation management center can add electronic device A and electronic device B to the sensing federation.

For example, the air conditioner 101 establishes a communication connection with the mobile phone 104. When the user logs in the user account 131****7837 on the mobile phone 104, the mobile phone 104 can send the equipment information of the mobile phone 104 and the equipment information of the air conditioner 101 to the federal management center. The federation management center can add the mobile phone 104 and the air conditioner 101 to the sensor federation corresponding to the user account 131****7837.

It can be understood that the mobile phone 104 and the air conditioner 101 may not be produced by the same manufacturer, but both the mobile phone 104 and the air conditioner 101 have interfaces that can transmit data to each other.

In some scenarios, users can choose one of their own electronic devices as the federation management center. The user may install a federation management center application in the electronic device serving as the federation management center. Users can create a sensor federation in the application of the federation management center, and invite other electronic devices to join the created sensor federation. Other electronic devices that can be used as data demand devices or data collection devices can be installed with a sensor federation engine application, and users can search for sensor federations that can be joined in the sensor federation engine application, and apply to join the sensor federation. Feel the Federation.

Taking the user establishing a sensor federation in the mobile phone 104 as an example, FIGS. 8A-8I exemplarily show the process in which the user establishes a sensor federation through the federation management center application program in the mobile phone 104 .

As shown in FIG. 8A , FIG. 8A exemplarily shows a user interface 80A of the mobile phone 104 . The user interface 80A may be the main interface of the mobile phone 104 . The user interface 80A may include an icon 8001 for the federation management center application. The user interface 80A may also include icons of other application programs, such as icons of weather application programs, icons of mail application programs, etc., which will not be described in detail here. A user may click on the federation management center application icon 8001 in the user interface 80A. In response to this user action, handset 104 may display user interface 80B.

As shown in FIG. 8B , FIG. 8B exemplarily shows a user interface 80B of the mobile phone 104 . The user interface 80B may include a control 8101 , a control 8102 and a control 8103 . This control 8101 can be used to create new sensory federations. The control 8102 can be used to view the sensor federations that have been established in the mobile phone 104 . The control 8103 can be used to view and manage the semantic model in the mobile phone 104 . It can be understood that there may be one or more semantic models in the mobile phone 104 . Semantic models can be used to derive semantic information from one or more sensor data. A semantic model for analyzing sleep quality, which can obtain semantic information such as high or low sleep quality of the user according to the user's sleep duration. The user may click on the control 8101 in the user interface 80B, and in response to the user operation, the mobile phone 104 may display the user interface 80C.

As shown in FIG. 8C , FIG. 8C exemplarily shows a user interface 80C of the mobile phone 104 . The user interface 80C may include a control 8301 , text information 8302 , an input box 8303 , an option box 8304 , a control 8316 and a control 8317 . Control 8301 can be used to return to the previous interface. Text information 8302 may indicate that the current interface is used to create a sensory federation. The content of the text information 8302 may be "create a sensor federation", and the specific content of the text information 8302 is not limited here. The input box 8303 can be used to input the name of the sensor federation created, such as "xxx". There is no limit to the content that can be input in the input box 8303. The option box 8304 may include a sliding bar 8305, a control 8306, a control 8307, a control 8308, and a control 8309. Among them, the sliding bar 8305 can be used to view the content not displayed in the option box 8304 . Control 8306 may be used to select "acceleration" as the cell phone 104 may provide sensor data. Control 8307 may be used to select "location" as sensor data that the handset 104 may provide. Control 8308 may be used to select "heart rate" as sensor data that cell phone 104 may provide. Control 8309 may be used to select "body temperature" as sensor data that the handset 104 may provide. The control 8316 can be used to save the content entered and set by the user in the current interface. Control 8317 may be used to indicate that the creation of a sensory federation is complete.

It can be understood that the above-mentioned user interface 80C is only an example, and the user interface 80C may include fewer or more interface elements. For example, the user interface 80C may also include a control for setting the semantic model that may be provided in the sensing federation . Alternatively, the user interface 80C may also include an option box for setting data required by the mobile phone 104 . The user interface for creating a sensor federation is not limited here.

In a possible implementation manner, after the user creates a sensor federation in the federation management center application program of the mobile phone 104, the user can view the sensor federation created by the user in the federation management center application program.

As shown in FIG. 8D , FIG. 8D exemplarily shows a user interface 80D of the mobile phone 104 . The user interface 80D may include a control 8101 , a control 8102 and a control 8103 . Regarding the control 8101, the control 8102, and the control 8103, reference may be made to the description in FIG. 8B above, and details will not be repeated here. The user can click on the control 8102 to view sensor federations that have been created in the handset 104 . In response to an action to tap control 8102, handset 104 may display user interface 80E.

As shown in FIG. 8E , FIG. 8E exemplarily shows a user interface 80E of the mobile phone 104 . The user interface 80E may include a control 8501 , text information 8502 , text information 8503 , control 8504 , control 8505 , and control 8506 , text information 8507 , control 8508 , control 8509 , and control 8510 . Control 8501 can be used to return to the previous interface. Text message 8502 may indicate that the current interface may be used to view a list of sensory federations that have been created in cell phone 104 . The text information 8503 is used to indicate the name and creation time of a sensor federation created in the mobile phone 104 . For example, the content of the text information 8503 may be "sensing federation xxx (1 minute ago)", that is, the text information 8503 may indicate that the name of the sensing federation is "xxx", and the sensing federation xxx was created by the mobile phone 104 1 minute ago . The control 8504 can be used to view the members of the sensory federation xxx (that is, the electronic devices joined in the sensory federation xxx). Control 8505 may be used to invite other electronic devices to join the sensory federation xxx. Control 8506 may be used to delete the sensory federation xxx. The text message 8507 may be used to indicate the name and creation time of another sensory federation that has been created in the handset 104 . For example, the specific content of the text information 8507 may be "sensing federation 111 (3 days ago)", that is, the text information 8507 may indicate that the name of the sensing federation is "111", and the sensing federation 111 was created by the mobile phone 104 three days ago . The control 8508 can be used to view the members in the sensory federation 111 (ie, the electronic devices joined in the sensory federation 111). Control 8509 may be used to invite other electronic devices to join the sensory federation 111. Control 8510 may be used to delete the sensory federation 111.

It will be appreciated that the user interface 80E shown in FIG. 8E is merely an example. The user interface 80E may contain more or fewer interface elements, for example, the user interface 80E may contain more created sensor federations. Alternatively, the user interface 80E may also include a sensor federation joined by the mobile phone 104 (not a sensor federation created in the mobile phone 104). The embodiment of the present application does not limit the display of the sensor federation list created and joined by the mobile phone 104 .

A user may click on control 8505 in user interface 80E, and in response to the user action, handset 104 may display user interface 80F.

As shown in FIG. 8F , FIG. 8F exemplarily shows a user interface 80F of the mobile phone 104 . The user interface 80F may include a control 8601 , text information 8602 , an option box 8603 , and a control 8610 . Control 8601 can be used to return to the previous interface. Text information 8602 may indicate that the current interface is used to invite other electronic devices to join the sensory federation xxx. The content of the text message 8602 may be "sensing federation xxx. invitation", and the specific content of the text message 8602 is not limited here. Invitable electronic devices and controls for inviting electronic devices may be displayed in option box 8603. For example, in the option box 8603, the air conditioner 101 and the control 8604 corresponding to the air conditioner 101, the camera 102 and the control 8605 corresponding to the camera 102, the watch 103 and the control 8606 corresponding to the watch 103, the tablet 105 and the control 8607 corresponding to the tablet 105, and the air quality sensor 106 and the control 8608 corresponding to the air quality sensor 106, and the sliding bar 8609. The control 8604 can be used to invite the air conditioner 101 to join the sensor federation xxx. Control 8605 can be used to invite camera 102 to join sensory federation xxx. Control 8606 may be used to invite watch 103 to join sensory federation xxx. Control 8607 can be used to invite tablet 105 to join sensory federation xxx. Control 8608 may be used to invite air quality sensors 106 to join sensory federation xxx. The sliding bar 8609 may be used to display electronic devices not displayed in the inviteable device list and controls corresponding to the electronic devices. Control 8610 is used to determine the electronic device selected by the inviting user.

The user can select one or more electronic devices in the option box 8603, and then click the control 8610 to invite one or more electronic devices selected by the user. In the following, the user chooses to invite the tablet 105 to join the sensor federation xxx as an example for illustration. The user clicks the control 8607 in the user interface 80F, and then clicks the control 8610. In response to the user operation, the mobile phone 104 may send a notification message to the tablet 105 inviting to join the sensor federation xxx. Tablet 105 may receive the notification message and display user interface 80G.

As shown in FIG. 8G , FIG. 8G exemplarily shows a user interface 80G of the tablet 105 . The user interface 80G may include a message notification box 8701 . The message notification box 8701 may include text information and a control 8702 and a control 8703 . The text information in the message notification box 8701 is used to prompt the user's mobile phone 104 to invite the tablet 105 to join the sensor federation xxx. The content of the text message may be "the mobile phone 104 invites you to join the sensor federation xxx", and the specific content of the text message is not limited here. Control 8702 may be used to agree to join the sensory federation xxx. Control 8703 can be used to refuse to join the sensing federation xxx.

In some possible examples, some electronic devices do not have user-operable display screens, eg, air conditioners, earphones, temperature sensors, and the like. Electronic devices that do not have user-operable displays can also be invited by the cell phone 104 to join the sensory federation xxx.

In a possible implementation, the mobile phone 104 can directly send a notification message inviting to join the sensor federation xxx to this type of electronic device that does not have a user-operable display screen. After receiving the message, this type of electronic device can prompt the user ( For example, the indicator light flashes, or the electronic device vibrates, rings, etc. to prompt the user), and the user can agree to join the sensor federation xxx through voice commands or by clicking on the physical control on this type of electronic device.

In a possible implementation, the user can directly add such electronic devices that do not have a user-operable display screen and are connected to the mobile phone 104 to the sensor federation xxx through the mobile phone 104 .

Optionally, in another possible implementation manner, such electronic devices without user-operable display screens (for example, air conditioners, earphones, temperature sensors, etc.) , tablet, mobile phone, etc.) to establish a communication connection. When the mobile phone 104 invites this type of electronic device without an operable display screen, a message prompt box can be displayed on the electronic device with an operable display screen that establishes a communication connection with this type of electronic device without an operable display screen, and the user can In electronic devices with operable display screens, agree or reject such electronic devices that do not have user-operable display screens to join the sensor federation xxx.

In a possible implementation manner, the user can also check the members of the sensor federation (that is, the electronic devices that join the sensor federation) in the user interface on the mobile phone 104 that can be used to view the list of sensor federations.

As shown in FIG. 8H , FIG. 8H exemplarily shows a user interface 80H of the cell phone 104 . The user interface 80H may include a control 8501 , text information 8502 , text information 8503 , control 8504 , control 8505 , control 8506 , text information 8507 , control 8508 , control 8509 , and control 8510 . For the specific functions of the controls in the user interface 80H, reference may be made to the description in the user interface 80E, which will not be repeated here. The user can view the members in the sensing federation xxx by clicking the control 8504. Cell phone 104 may display user interface 801 in response to the user's operation of clicking control 8504 .

As shown in FIG. 8I , FIG. 8I exemplarily shows a user interface 80I of the mobile phone 104 . The user interface 80I may include a control 8801, text information 8802, and a member list column 8803. Control 8801 can be used to return to the previous interface. Text information 8802 may indicate that the current interface is used to view members in sensing federation xxx. The member list column 8803 can be used to display the members in the sensing federation xxx. The member list bar 8803 may include a member tablet 105 of the sensing federation xxx, a control 8804 and a control 8805 corresponding to the member tablet 105, and a sliding bar 8806. Control 8804 can be used to set the permissions of tablet 105 (eg, the sensor data that tablet 105 can provide, and the sensor data that tablet 105 requires). Control 8805 may be used to withdraw tablet 105 from sensory federation xxx. The sliding bar 8806 may be used to display other members of the sensing federation xxx not displayed in the member list bar 8803.

It can be understood that the above user interfaces shown in FIGS. 8A-8B are only examples. The user interface shown in FIG. 8A-FIG. 8I may include fewer or more interface elements, or the positions of elements of the user interface in the user interface may be changed, which is not limited in this embodiment of the present application.

It can be understood that the user can call the function module in the mobile phone 104, that is, the federation management center, through the application program of the federation management center in the mobile phone 104 . Regarding the federation management center, reference may be made to the description of the federation management center 30 in FIG. 3 and the description of the federation management center 30 in FIG. 5 , and details are not repeated here.

In other possible examples, a sensor federation engine application may be installed in the electronic device. Users can search for and join sensor federations created by other electronic devices through the sensor federation engine application. In the following, the tablet 105 is installed with a sensor federation engine application program, and the user joins the sensor federation xxx created by the mobile phone 104 through the sensor federation engine application program as an example for illustration.

9A-9G exemplarily show the specific process of adding the tablet 105 to the sensor federation xxx created by the mobile phone 104 through the sensor federation engine application program in the tablet 105 .

As shown in FIG. 9A , FIG. 9A exemplarily shows a user interface 90A of the tablet 105 . The user interface 90A may be the main interface of the tablet 105 . The user interface 90A may include an icon 9101 for the sensor federation engine application. The user interface 90A may also include icons of other application programs, such as icons of weather application programs, icons of mail application programs, etc., which will not be described in detail here. A user may click on the sensor federation engine application icon 9101 in the user interface 90A. In response to this user operation, tablet 105 may display user interface 90B.

As shown in FIG. 9B , FIG. 9B exemplarily shows a user interface 90B of the tablet 105 . The user interface 90B may include a control 9201 , a control 9202 and a control 9203 . Control 9201 may be used to search for nearby sensory federations to join. The control 9202 can view the sensing federations that the tablet 105 has currently joined. The control 9203 can be used to view the historical sensor federations that the tablet 105 has joined (the sensor federations that have joined before, but are not currently in the sensor federation). A user may click on control 9201 in user interface 90B, and in response to the user action, tablet 105 may display user interface 90C.

As shown in FIG. 9C , FIG. 9C exemplarily shows a user interface 90C of the tablet 105 . The user interface 90C may include a control 9301 , text information 9302 , and a sensor federation list 9303 . Control 9301 can be used to return to the previous interface. Text information 9302 may indicate that the current interface is used to search for nearby sensor federations that can be joined. The sensor federation list 9303 can be used to display nearby sensor federations. The sensor federation list 9303 may include the sensor federation xxx and the control 9304 corresponding to the sensor federation xxx, the sensor federation aaa and the control 9305 corresponding to the sensor federation aaa, the sensor federation bbb and the control 9306 corresponding to the sensor federation bbb, The sensor federation ccc and the control 9307 corresponding to the sensor federation ccc, the sensor federation ddd and the control 9308 corresponding to the sensor federation ddd, and the sliding bar 9309. The control 9304 can be used to apply for joining the sensor federation xxx. The control 9305 can be used to apply for joining the sensing federation aaa. The control 9306 can be used to apply for joining the sensing federation bbb. Control 9307 can be used to apply for joining the sensing federation ccc. Control 9308 can be used to apply for joining the sensing federation ddd. The sliding bar 9309 may be used to display sensor federations not displayed in the sensor federation list 9303 .

It can be understood that more or fewer sensor federations may be displayed in the sensor federation list 9303 . More or fewer interface elements may be displayed on the user interface 90C, and the embodiment of the present application does not limit the user interface that can be used to search and display nearby sensor federations that can be added.

When the user wants to join the sensing federation xxx created by the mobile phone 104, the user may click on the control 9304 in the user interface 90C. In response to the user operation, the tablet 105 may send a notification message to the mobile phone 104 to apply for joining the sensor federation xxx. Cell phone 104 may receive the notification message and display user interface 90D.

As shown in FIG. 9D , FIG. 9D exemplarily shows a user interface 90D of the mobile phone 104 . The user interface 90D may include a message notification box 9401 . The message notification box 9401 may include text information and a control 9402 and a control 9403 . The text information in the message notification box 9401 may be used to instruct the tablet 105 to apply for joining the sensor federation xxx. The content of the text information in the message notification box 9401 may be "the tablet 105 applies to join the sensor federation xxx". Control 9402 may be used to agree to the tablet 105 joining the sensory federation xxx. The control 9403 can be used to refuse the tablet 105 to join the sensor federation xxx.

In a possible implementation manner, after the user clicks the control 9402 in the user interface 90D, the mobile phone 104 may send a notification message to the tablet 105 agreeing that the tablet 105 joins the sensor federation xxx. Tablet 105 may receive the notification message and display user interface 90E.

As shown in FIG. 9E , FIG. 9E exemplarily shows a user interface 90E of the tablet 105 . The user interface 90E may include a message notification box 9501 , and a control 9201 , a control 9202 , and a control 9203 . The message notification box 9501 can be used to prompt the user that the tablet 105 has successfully joined the sensor federation xxx. The content of the message notification box 9501 may be "successfully joined the sensor federation, you can query and set permissions in the joined sensor federation". The embodiment of this application does not limit the specific content in the message notification box 9501. Regarding the control 9201, the control 9202, and the control 9203, reference may be made to the description in FIG. 9B , which will not be repeated here. A user may click on control 9202, and in response to this user action, tablet 105 may display user interface 90F.

As shown in FIG. 9F , FIG. 9F exemplarily shows a user interface 90F of the tablet 105 . The user interface 90F may include a control 9601 , text information 9602 , control 9603 , control 9604 , text information 9605 , control 9606 and control 9607 . Control 9601 can be used to return to the previous interface. The text information 9602 may be used to indicate the sensor federation xxx joined by the tablet 105 and the time (for example, 1 minute ago) when the tablet 105 joined the sensor federation xxx. The control 9603 can be used to set the authority of the tablet 105 in the sensor federation xxx. Control 9604 may be used to withdraw tablet 105 from sensory federation xxx. The text message 9605 may be used to indicate the sensory federation 111 that the tablet 105 joined and the time (eg, 3 days ago) when the tablet 105 joined the sensory federation 111 . The control 9606 can be used to set the authority of the tablet 105 in the sensor federation 111 . Control 9607 may be used to withdraw tablet 105 from sensory federation 111. The user can click the control 9603 in the user interface 90F, and in response to the user operation, the tablet 105 can display the user interface 90G.

As shown in FIG. 9G , FIG. 9G exemplarily shows a user interface 90G of the tablet 105 . Control 9701 , option box 9702 , and control 9714 . Control 9701 can be used to return to the previous interface. Option box 9702 may include control 9703 , control 9704 , control 9705 , control 9706 , and slide bar 9707 . Control 9703 can be used to select "acceleration" as the tablet 105 can provide sensor data. Control 9704 may be used to select a "location" as sensor data that tablet 105 may provide. Control 9705 may be used to select "heart rate" as sensor data that tablet 105 may provide. Control 9706 can be used to select "body temperature" as sensor data that the tablet can provide. The slide bar 9707 can be used to view content not displayed in the options box 9702. The control 9714 can be used to indicate that the authorization setting of the tablet 105 in the sensor federation xxx is completed.

It can be understood that the above user interface 90G is only an example, and the user interface 90G may include fewer or more interface elements, for example, the user interface 90G may also include controls for setting the semantic model that the tablet 105 may provide. Alternatively, the user interface 90G may also include an option box for setting the data required by the tablet 105 . The user interface for setting the authority of the tablet 105 in the sensor federation is not limited here.

It can be understood that the user can call the function module in the tablet 105, that is, the sensor federation engine module, through the sensor federation engine application program in the tablet 105 . Regarding the federation management center, reference may be made to the description of the sensing federation engine module in FIG. 3 and FIG. 4 , and details will not be repeated here.

In some possible examples, some electronic devices do not have user-operable display screens, eg, air conditioners, earphones, temperature sensors, and the like. Electronic devices that do not have a user-operable display can also apply to join the Sensing Federation xxx.

Optionally, in another possible implementation manner, such electronic devices without user-operable display screens (for example, air conditioners, earphones, temperature sensors, etc.) , tablet, mobile phone, etc.) to establish a communication connection. Users can apply to join this type of electronic device without an operable display screen to the sensor federation xxx on the electronic device with an operable display that establishes a communication connection with this type of electronic device without an operable display. For the specific operation process, reference may be made to the descriptions in FIGS. 9A-9G above.

In some scenarios, some electronic devices may have device QR codes, and other electronic devices may invite the electronic devices to join the sensor federation by scanning the device QR codes on the electronic devices.

In some scenarios, some data in the electronic device is obtained by some application programs in the electronic device. For example, some sports application programs may provide data such as the number of steps taken by the user every day and the walking distance. If the electronic device wants to provide data such as the number of steps taken by the user every day and the distance traveled to the sensor federation, the electronic device can set permissions in the sensor federation to allow such applications to obtain data.

It can be understood that the federation management center of the sensor federation may store knowledge graphs, for example, the knowledge graphs shown in (a) and (b) of FIG. 6 above. The knowledge graph stored in the federation management center can be used to represent the data that can be collected in the current sensing federation, as well as the semantic information generated by these data. For example, the data data1 collected by sensor 1 (for example, time: 10 am on weekdays), the data data2 collected by sensor 2 (for example: current location: xxx Science Park) and the data data3 collected by sensor (for example, age: 30 years old) Semantic information X (eg, user is working) can be generated.

Further, when the federation management center receives the new device joining information, the federation management center can determine whether the device provides data of a new type or new precision, and if so, the federation management center can store a complete knowledge graph The cloud server obtains the knowledge graph related to the data of the new type or new precision. The federation management center may then send notification messages to electronic devices in the sensing federation that require this new type of data.

In some scenarios, users can add some of their private electronic devices, such as the user's mobile phone, headset, tablet computer, etc., to the sensor federation. The user can create a sensory federation for the user's electronic device according to the steps shown in FIGS. 7A-7E or 8A-8I and 9A-9G, and add the user's electronic device to the sensory federation.

It can be understood that when an electronic device joins a sensing federation, the federation management center can obtain information about the sensing capabilities of the electronic device, such as the device ID of the electronic device, the model of the electronic device, and the type of sensor that can provide data in the electronic device And the type of data that the sensor can provide, the type of software that can provide data and the type of data that the software can provide, low-level semantic information, high-level semantic information and other information obtained from data collected by the sensor or data generated by the software. The federation management center can record the device information of each electronic device in the sensing federation in the electronic device or the cloud server where the federation management center is located. When the device information in the electronic device changes, for example, a new application is downloaded and a new type of data can be obtained, the electronic device can report the new type of data to the federation management center. The federal management center can refresh the record.

In a possible implementation manner, when the electronic device joins the sensing federation, the electronic device may report the model of the electronic device to the federation management center. Based on the model of the electronic device, the federal management center can obtain the device capability,

Optionally, the federation management center can also obtain non-personal privacy data (for example, base station data, cell data) from other electronic devices in the sensor federation for training a more accurate model (for example, the input position can get the image of surrounding cells Model).

In some scenarios, the electronic devices in the sensing federation can opt out of the sensing federation in the federation management center application program of the federation management center as shown in FIG. 8I above. Alternatively, the electronic device in the sensor federation can also choose to quit the sensor federation in the sensor federation engine application program of the electronic device as shown in FIG. 9F above.

In other scenarios, when the electronic equipment in the sensor federation is offline for a preset period of time (for example, the communication connection with the federation management center is interrupted, or the electronic equipment is not connected to the Internet, or the electronic equipment is turned off, etc.) , the electronic device exits the sensing federation.

In some scenarios, electronic devices in the sensor federation actively report to the federation management center after collecting abnormal data. The federal management center can make relevant recommendations or decisions based on the abnormal data, or modify the corresponding judgment results. Optionally, the federation management center can also send relevant notifications or recommendations to other electronic devices in the sensor federation based on the abnormal data. For example, if a watch in the sensing federation collects body temperature data "the user's body temperature is 39°", the watch judges that the body temperature data is abnormal data, and can report the abnormal body temperature data of "the user's body temperature is 39°" to the federation management center, and based on this The semantic information obtained from body temperature data is "the user is sick". The federal management center receives the abnormal body temperature data sent by the watch, and the semantic information "user is sick" based on the body temperature data. The federal management center can adjust the user's original body temperature data and health status data stored in the federal management center. The federal management center can also send the abnormal body temperature data and the semantic information "user is sick" obtained based on the body temperature data to electronic devices that need such data. After receiving the data of this type, the electronic device adjusts the output of the model in the electronic device based on the data of this type. For example, after the electronic device receives the semantic information of "the user is sick", the model used in the electronic device for intelligent route recommendation may plan the user's route from the current location to the hospital instead of planning the user's home from the current location. The electronic device can also recommend appropriate medicines or cooling methods to the user according to "the user's body temperature is 39°". The electronic device can also record other data obtained according to the user's body temperature data and semantic information.

Further, in a possible implementation manner, the user can set an importance level for sensor data or semantic information that can be provided by electronic devices in the sensor federation. For example, important level (data or semantic information such as accidental injury, serious illness, elderly fall/children’s abnormal position), and general level (data or semantic information such as general illness and overtime). When important data is collected by an electronic device in the sensor federation, it can be reported to the federation management center in the sensor federation in real time, and the federation manager center can send notification messages to other electronic devices in the sensor federation. Optionally, in a possible implementation manner, the user can set the type of data that one or more electronic devices in the sensor federation can provide, for example, data of physical health, data of abnormal location, and so on. Optionally, this setting can be shared with other federated users. Optionally, after receiving the reminder message, you can send a voice/notification/other information to the other party for feedback, such as reminding to pay attention to rest when working overtime. Optionally, members in the same sensor federation can set reminders, schedules and other information with each other. For example: a family dinner was originally scheduled at 7:00 p.m., and a meeting is temporarily added to the calendar of one of the family members’ father’s electronic devices, then other users’ mother’s electronic devices can receive reminder messages, and the mother’s electronic devices can receive reminders. Respond to reminders, such as eating early, taking a break, etc.

In some scenarios, some communities or units that need certain specific data can establish a sensory federation kkk in the server, and the server can be the federation management center of the sensory federation, and generate an application program or a website for the sensory federation, or The two-dimensional code is released, and users who are willing to provide data can install the application program in the electronic device, and can open the URL by opening the URL. The user's electronic device can report data to the server in the sensor federation kkk through the application program or website. If the electronic device has also joined another sensory federation xxx, the electronic device can also obtain authorization from the sensory federation xxx, and upload the data in the sensory federation xxx to the server in the sensory federation kkk.

In a feasible example, the cardiovascular and cerebrovascular research community establishes a sensory federation kkk on the server, and releases an application program or website for joining the sensory federation kkk. Users who are willing to provide can download the application program or join the sensor federation kkk through the URL.

In another possible example, the organizer of an event (e.g., a marathon) can create a sensor federation in the server, which is used to monitor the physical conditions of the event participants (e.g., whether they are hypothermic , sudden myocardial infarction, etc.). Before the event starts, the electronic devices of all event participants can join the sensor federation. That is, the electronic devices of all event participants and the server of the event organizer form a sensor federation. The server confirms which sensor data needs to be collected to judge the disease condition. If the electronic devices of the activity participants have their own monitoring functions, they can only report the monitoring results to the server. If the electronic equipment of the activity participant does not have a monitoring function, the corresponding sensor data will be reported according to the requirements of the server. The reporting method can be real-time reporting, or reporting when a certain sensor data is abnormal. The server judges the physical status of the event participants by reasoning the collected sensor data. If there is any abnormality, it will report to the event organizer and provide emergency assistance to the event participants.

In some feasible examples, when several users travel by car in a group, the mobile phones of several users (for example, the mobile phone mobile1 of user 1, the mobile phone of user 2, and the mobile phone of user 3) and the car (for example, The vehicle-machine vehicle1 of user 1, the vehicle-machine vehicle2 of user 2, and the vehicle-machine vehicle3 of user 3) can form a sensor federation according to the steps shown in Figure 7A-Figure 7E or Figure 8A-Figure 8I and Figure 9A-Figure 9G . There is a body state estimation model in mobile1. The body state estimation model does not exist on mobile2 and mobile3. The user can set to send the sensor data in the mobile phone mobile2 and the mobile phone mobile3 that can be used to infer the body state to the mobile phone mobile1. When the mobile phone mobile1 deduces that the user's physical condition is abnormal according to the sensor data sent by the mobile phone mobile2 or mobile phone 3, it can send a reminder message to the mobile phone mobile2 or mobile phone 3. There is an associated model of the gas station on the map and the vehicle oil quantity in mobile2. The association model in the mobile phone mobile2 can remind you to refuel according to the navigation data, the amount of oil in the car, etc. Mobile1, mobile3, vehicle1, vehicle2, and vehicle3 do not have this association model. In the sensor federation, the user can set the vehicle model, fuel consumption and current fuel volume of vehicle1, vehicle2, and vehicle3 to mobile2. The mobile phone mobile2 will judge a suitable gas station based on these comprehensive information and give a reminder.

In some scenarios, when multiple electronic devices form a sensing federation, an electronic device with strong computing and storage capabilities can be selected as the federation management center. Each sensor federation can correspond to a sensor federation ID.

Optionally, in a possible implementation, when the electronic device serving as the federation management center in a sensing federation is in an abnormal state, or cannot communicate with other electronic devices, another electronic device can be selected as the federation control center.

After multiple electronic devices form a sensory federation, one electronic device in the sensory federation can obtain the data collected by the sensors of one or more other electronic devices, or the data collected by the sensors of one or more other electronic devices Get semantic information.

In this embodiment of the present application, the data-demanding device may be referred to as a first device, or a first electronic device, or a sixth device. The data acquisition device may be called as the second device, the third device, the fourth device, the fifth device, the sixth device, the seventh device, or any one of the third electronic device and the fourth electronic device. The federal management center may also be referred to as a second electronic device.

The following will describe in detail how an electronic device in a sensor federation acquires data collected by sensors in other electronic devices or semantic information obtained from data collected by sensors.

FIG. 10 exemplarily shows a schematic flowchart of a data collection method provided by an embodiment of the present application. As shown in Figure 10, a data collection method provided by the embodiment of the present application may include the following steps:

S1001. The data demand device 20 establishes a communication connection with the federation management center 30; the data collection device 40 establishes a communication connection with the federation management center 30.

The data demand device 20 can establish a communication connection with the electronic device where the federation management center 30 is located through WLAN, Bluetooth, cellular network, distributed soft bus, USB cable, etc. For details, refer to the description in FIG. 1 , which will not be repeated here.

The data collection device 40 can establish a communication connection with the electronic device where the federation management center 30 is located through WLAN, Bluetooth, cellular network, distributed soft bus, USB cable, etc. For details, please refer to the description in FIG. 1 and will not go into details here.

It can be understood that the data demand device 20, the federation management center 30, and the data collection device 40 have established a data collection system (also called a sensory federation). Regarding how the data demand device 20, the federal management center 30 and the data collection device 40 establish the data collection system 40, you can refer to the descriptions in the above-mentioned Fig. 7A-Fig. 7E, Fig. 8A-Fig. 8I, and Fig. 9A-Fig. 9G. repeat.

Regarding the data demand device 20, reference may be made to the descriptions in FIG. 1 and FIG. 3, which will not be repeated here. Regarding the federation management center 30, reference may be made to the descriptions in FIG. 1 and FIG. 3, which will not be repeated here. Regarding the data acquisition device, reference may be made to the descriptions in FIG. 1 and FIG. 3 , and details are not repeated here.

Exemplarily, the data demand device 20 may be the air conditioner 101 shown in FIG. 2 . The electronic device where the federation management center 30 is located may be the mobile phone 104 or the tablet 105 shown in FIG. 2 . Optionally, the electronic device where the federation management center 30 is located may also be a server or a cloud server. The data collection device 40 may be any electronic device such as the camera 102 , the watch 103 , the mobile phone 104 , the tablet 105 , and the air quality sensor 106 shown in FIG. 2 .

S1002. The data demand device 20 sends a data collection task request and collection task parameters to the federal management center 30. The collection task parameters include one or more of collection capability requirements, collection cycle, collection duration, collection data accuracy, and collection operator.

When the data demand device 20 is in the working state, the data demand device 20 can generate data collection tasks and data collection parameters. The data collection task is used to indicate the data required by the data demand device 20 . The collection task parameters may include one or more of collection capability requirements, collection period, collection duration, collection data accuracy, and collection operator. The collection capability requirement is used to indicate the sensing capability information of the data collection device corresponding to the data required in the data collection task. The collection period is used to indicate the period of collecting the data required in the data collection task (for example, the data required in the data collection task needs to be collected once every 5 minutes). The collection duration is used to indicate the duration of collecting the data required in the data collection task. The Acquisition Data Accuracy is used to indicate the accuracy of the data required in the Acquisition Data task. The collection operator is used to process the sensor data collected by the data collection device. For example, the average value of sensor data collected in 5 seconds.

It can be understood that, the embodiment of the present application does not limit the specific content of the data collection task. The embodiment of the present application does not limit the specific content of the task collection parameters.

Specifically, in a possible implementation manner, for example, an application program (for example, APP2011) in the application layer of the data demand device 20 may initiate a data collection request. The sensor federation engine module 202 in the data demand device 20 can generate data collection tasks and collection task parameters from data collection requests initiated by applications in the application layer.

Exemplarily, the above-mentioned data collection task and collection task parameters will be described by taking the air conditioner 101 shown in FIG. 2 as an example where the data demand device 20 is used. The air conditioner 101 may have software for adjusting the intelligent mode, which can automatically adjust the working mode (cooling, heating, sweeping, etc.) of the air conditioner 101 and the cooling or heating temperature of the air conditioner according to the temperature in the environment. When the smart mode of the air conditioner 101 is turned on, the software for adjusting the smart mode in the air conditioner 101 needs temperature data in the environment. The software used to adjust the smart mode can be based on temperature data in the environment every 30 minutes. The sensing federation engine module 202 in the air conditioner 101 can generate a data collection task for obtaining temperature data in the environment (ie, collect temperature data in the environment) and collection task parameters (ie, the collection cycle is once every 30 minutes).

It can be understood that the data collection task may include collecting one or more types of data. When the data collection task is to collect multiple types of data (for example, outdoor temperature data, user identity, user health and other types of data), the data collection task may correspond to one or more data collection devices 40 . That is, multiple types of data to be collected in the data collection task may be collected by one data collection device 40 , or may be collected by multiple data collection devices 40 . It can be understood that each type of data that needs to be collected in the data collection task can also be collected by one or more data collection devices 40 . This embodiment of the present application does not limit it. In the following, a data collection task that requires one type of data is used as an example to illustrate.

The data demand device 20 can send the data collection task and the collection task parameters to the federation management center 30 .

In a possible implementation manner, FIG. 11 exemplarily shows a flow chart of steps in which the data demand device 20 sends a data collection task request and collection task parameters to the federation management center 30 . As shown in Figure 11, the data demand device 20 sends the data collection task request and the collection task parameters to the federation management center 30 may include the following steps:

S10011. The data demand device 20 determines whether there is a data collection device 40 that meets the parameters of the collection task. If yes, execute step S10012a; if not, execute step S10012b.

If the data demand device 20 records a historical record that the data collection device 40 has completed the data collection task and satisfies the parameters of the collection task, step S10012 can be executed; otherwise, step S10012b can be executed.

S10012a, the data demand device 20 requests to establish a communication connection with the data collection device 40 , and sends a collection task request and collection task parameters to the data collection device 40 .

The data demand device 20 may directly request to establish a communication connection with the data collection device 40 . The data collection device 40 has completed the data collection task sent by the data demand device 20 and complies with the collection parameters of the task. The data demand device 20 may directly send the collection task request and the collection task parameters to the data collection device 40 . In this way, the following steps S1003-S1005 and S1006a and S1006b do not need to be performed.

S10012b. The data demand device 20 sends a data collection task request and collection task parameters to the federation management center 30 .

Here, reference may be made to the description above, and details are not repeated here.

S1003. The federation management center 30 determines the data collection device 40 corresponding to the data collection task based on the collection task parameters.

The federation management center 30 may receive the data collection task and collection task parameters sent by the data demand device 20 . The federal management center 30 may store a list of different sensing capability information, which may include different sensing capability information, the ID of the semantic information contained in each sensing capability information, and an ID that can provide semantic information. or multiple devices and so on. The federation management center 30 can determine the semantic information x1 required to complete the data collection task based on the collection task parameters of the data collection task, and query the corresponding data collection device 40 that can provide the semantic information x1 in the sensing capability information list. The sensing capability information list may be specifically shown in Table 1 below.

Table 1 exemplarily shows a list of sensing capability information in the federation management center 30 .

Table 1 Sensing capability information list

As shown in Table 1, the sensing capability information of multiple devices is exemplarily shown in Table 1. The sensing capability information of each device may include information such as semantic information that each device can provide, a semantic information ID, data accuracy of the provided semantic information, a location of the device, and the like.

As shown in Table 1, Table 1 can include semantic information such as environmental temperature data, user identity, user's exercise steps, user body temperature, user fatigue, etc., as well as environmental temperature data, user identity, user's exercise steps, user body temperature, etc. Semantic information ID, electronic device that provides semantic information, ID of electronic device, data accuracy, location of electronic device, etc. corresponding to sensing capabilities.

As shown in Table 1, the semantic information ID corresponding to the semantic information "ambient temperature data" may be "CGLB0001". The electronic device providing the semantic information "ambient temperature data" may be a temperature sensor, the device model of the temperature sensor may be "XH01", and the device ID may be "Device 0001". The data accuracy of the ambient temperature data provided by the temperature sensor may be "±0.01°C". The location where the temperature sensor is located may be "user A's bedroom".

As shown in Table 1, the semantic information ID corresponding to the semantic information "user identity" may be "CGLB0002". Electronic devices that provide the semantic information "user identity" can be mobile phones, tablets, and televisions. The device model of the mobile phone can be "XH02", and the device ID can be "Device 0002". The data accuracy of the semantic information "user identity" provided by the mobile phone can be "98%". The location of the mobile phone can be a null value (such as Null). The device model of the tablet can be "XH03", and the device ID can be "Device0003". The data accuracy of the semantic information "user identity" provided by the tablet can be "99%". The location of the tablet may be "user A's bedroom". The TV model can be "XH04" and the device ID can be "Device 0004". The data accuracy of the "user identity" provided by the television may be "95%". The location of the television may be "user A's living room".

As shown in Table 1, the semantic information ID corresponding to the semantic information "user's motion steps" may be "CGLB0003". Electronic devices that provide the semantic information "number of user's motion steps" can be mobile phones, watches, and bracelets. The device model of the mobile phone can be "XH02", and the device ID can be "Device 0002". The data accuracy of the semantic information "user's movement steps" provided by the mobile phone can be "0.1 step". The location of the mobile phone can be a null value (such as Null). The model of the watch can be "XH05" and the device ID can be "Device 0005". The data accuracy of the semantic information "user's motion steps" provided by the watch may be "0.5 steps". The location of the watch may be "user A's bedroom". The model of the bracelet can be "XH06", and the device ID can be "Device 0006". The data accuracy of the semantic information "user's movement steps" provided by the bracelet can be "0.8 steps". The location of the wristband can be a null value (such as Null).

As shown in Table 1, the semantic information ID corresponding to the semantic information "user body temperature" may be "CGLB0004". Devices that provide the semantic information "user body temperature" can be watches and wristbands. The model of the watch can be "XH05" and the device ID can be "Device0005". The data accuracy of the semantic information "user body temperature" provided by the watch can be "±0.01°C". The location of the watch may be "user A's bedroom". The model of the bracelet can be "XH06", and the device ID can be "Device 0006". The data accuracy of the semantic information "user body temperature" provided by the bracelet can be "±0.02°C". The location of the wristband can be a null value (such as Null).

As shown in Table 1, the semantic information ID corresponding to the semantic information "user fatigue" may be "CGLB0006". Electronic devices that provide the semantic information "user fatigue" may be mobile phones and wristbands. The device model of the mobile phone can be "XH02", and the device ID can be "Device 0002". The data accuracy of the semantic information "user fatigue" provided by the mobile phone can be "95%". The location of the mobile phone can be a null value (such as Null). The model of the wristband can be "XH06", and the device ID can be "Device0006". The data accuracy of the semantic information "user fatigue" provided by the wristband can be "92%". The location of the wristband can be a null value (such as Null).

It can be understood that the above Table 1 is only an example. Further sensing capabilities may be stored in federation management center 30 . And more information corresponding to the sensing capability can also be stored in the federal management center 30 (for example, the resource of the electronic device that provides the sensing capability, and the wearing state of the electronic device, the power state, and the sensors and sensor information included in the electronic device. ID, model, type of raw data that the sensor can provide, accuracy, etc.). The embodiment of the present application does not limit the content contained in the sensing capability information list stored in the federation management center 30 .

It can be understood that, when the position of the electronic device is fixed for a period of time (for example, a TV set or an air conditioner), the position information of the electronic device may be recorded in the sensing capability information list. If the position of the electronic device is not fixed (for example, a handheld device or a wearable device), the position information of the electronic device may not be recorded in the sensing capability information list.

Optionally, the electronic device may periodically report location information to the federation management center.

It can be understood that the sensing capability information in the sensing capability information list in the federation management center 30 may be reported to the federation management center 30 by sensing each electronic device in the federation. Specifically, when the data demand device 20 and the data collection device 40 join the sensor federation formed by the federation management center 30, the data demand device 30 and the data collection device 40 can report the sensing capability information in the device to the federation management center 30 respectively . The federation management center 30 may update the sensor capability information list stored in the federation management center 30 based on the sensor capability information reported by the data demand device 20 and the data collection device 40 .

Optionally, when a device in the sensor federation joins the sensor federation, it can report the device model of the device to the federation management center. The federal management center can obtain the sensing capability information of the device in the cloud server based on the device model of the device. Sensing capability information corresponding to different types of devices is stored in the cloud server.

Optionally, the sensing capability information list generated from the sensing capability information of each device in a sensing federation may be stored in a cloud server or a server. The information in the federation management can obtain the sensing capability information list from the cloud server or the server.

Optionally, the federation management center may also use the storage resources of another device in the sensing federation to store data or information that the federation management center needs to store (for example, a list of sensing data or sensing capability information).

In a possible implementation, whenever a new device model joins the sensor federation, the federation management center can obtain the sensing capability information of the new device model. The federation management center can update the sensing capability information list in the sensing federation based on the full amount of knowledge graph stored in the cloud server and the sensing capability information of the new device model, and update the Local knowledge graph.

In the embodiment of the present application, the cloud or cloud server, or the entire knowledge graph stored in the server may be referred to as the second knowledge graph. The local knowledge graph in the sensor federation can be called the first knowledge graph.

Optionally, the federal management center can push out new types of sensing capability information based on the full knowledge graph stored in the cloud server and the sensing capability information of the new device model, and/or obtain information for A derivation model for deriving new types of sensory capability information.

For example, when the wristband with the device model "XH06" in Table 1 is not added to the sensor federation, the sensor capability information list in the sensor federation may not include the wristband with the model "XH06". The sensing capability information corresponding to the ring (for example, the device that provides the semantic information corresponding to the semantic information "user body temperature" does not include the model "XH06" wristband), and using the sensor information corresponding to the "XH06" wristband A new type of sensory capability information (eg, semantic information "user fatigue") is derived from the sensory capability information. When the wristband joins the sensing federation, the federation management center can obtain the sensing capability information of the wristband, and can derive new sensing information based on the semantic information "user's movement steps" and semantic information "user's body temperature" provided by the wristband. Capability information, such as the semantic information "user fatigue", and the ID of the semantic information "user fatigue", can provide the semantic information "user fatigue" device, device model and other sensing capability information.

Optionally, when a device joins a sensing federation, the federation management center may generate a device ID in the sensing federation for the device.

In a possible implementation, when a device in the sensing federation withdraws, the federation management center can delete the sensing capability information of the device in the sensing capability information list and the information derived based on the sensing capability information of the device. New sensing capability information.

In a possible implementation manner, when the sensor federation is dissolved, the federation management center may delete the sensing capability information list in the sensor federation.

In a possible implementation manner, FIG. 12 exemplarily shows a specific flowchart of the federation management center 30 determining the data collection device 40 corresponding to the data collection task based on the collection task parameters. As shown in Figure 12, the federation management center 30 determines the data collection device 40 corresponding to the data collection task based on the collection task parameters may include the following steps:

S10021. The federal management center 30 determines the semantic information N corresponding to the data collection task, and determines M1 data collection devices 40 corresponding to the sensing capability N in the sensing capability information list, and the sensing capability information list contains multiple semantic information And one or more data collection devices 40 respectively corresponding to multiple semantic information.

The federation management center 30 can determine the semantic information N corresponding to the data to be collected in the data collection task, and determine the M1 data collection devices 40 corresponding to the semantic information N in the sensing capability information list. M1 can be an integer. The sensing capability information list may include a plurality of semantic information and one or more data collection devices 40 respectively corresponding to the plurality of semantic information. For a list of sensing capability information, reference may be made to Table 1 above, which will not be repeated here.

It can be understood that there may be one or more electronic devices providing semantic information N. For example, the electronic device that provides the semantic information "environmental temperature data" in Table 1 above is a temperature sensor. The electronic devices that provide the semantic information "user identity" in Table 1 above may be mobile phones, tablets, and televisions.

It can be understood that the data required in the above data collection task may be one or more of environmental temperature data, user identity, user's exercise steps, user's body temperature and other data. The embodiment of the present application does not limit the specific data collection task.

In a possible implementation manner, when the data described in the data collection task is multiple types of data, the semantic information required by the data collection task may also be multiple.

In a possible implementation, when the semantic information N corresponds to a data collection device 40, that is, when M1 is 1, the federation management center 30 may determine to send the data collection task to the data collection device 40, and the federation management center 30 may The following steps S10021 to S10025 are not executed. When M1 is greater than 1, it means that there are multiple data acquisition devices 40 in the sensing capability information list that can provide the semantic information N, and the federation management center 30 can follow the steps S10022-Step S10025 below to select from multiple data collection devices 40 that can provide the semantic information N. A target data collection device 40 is selected from the collection devices 40 .

In a possible implementation manner, M1 may be equal to 0, that is, there is no device providing the semantic information in the sensor federation. The federation management device may inform the data demand device 20 that there is no device in the sensing federation that can provide the semantic information. The federal management center may not perform the following steps.

It can be understood that if the collection task parameters indicate that the data collection task can be completed by multiple data collection devices 40, the federation management center 30 can directly send the data collection task and the collection task parameters to multiple data collection devices that can provide semantic information N Acquisition device 40 . That is, the federation management center 30 does not need to execute the following steps S10022-step S10025.

S10022. The federal management center 30 determines M2 wakeable data collection devices 40 from the M1 data collection devices 40 .

The federal management center 30 may determine M2 wakeable data collection devices 40 from the M1 data collection devices 40 . M2 is an integer, and M2 is less than or equal to M1. In this embodiment of the application, the wakeable data collection device 40 may mean that the federation management center 30 can receive the message sent by the data collection device 40 within a preset time, and the data collection device 40 does not report an abnormal state (for example, data The collection device 40 is shut down, the data collection device 40 is powered off, the power of the data collection device is lower than the preset power threshold, and the data collection device is not in the wearing state (one or more items).

In a possible implementation, the electronic devices that can provide semantic information N in the sensor capability information list in the federation management center 30 may not be in the sensor federation where the data demand device 20 and the federation management center 30 are located. The electronic device providing the semantic information N may be another sensor federation that has joined the federation management center 30 before step S1001.

In a possible implementation manner, M2 may be greater than 1 or equal to 1. When M2 is equal to 1, it means that there is only one wakeable data collection device 40 among the M1 data collection devices 40 . The federal management center 30 can send the data collection task and data collection parameters to the wakeable data collection device 40 . The federation management center 30 may not execute the following steps S10023-step S10025. When M2 is greater than 1, the federation management center 30 may execute the following step S10023.

In a possible implementation manner, M2 may be equal to 0, which means that there is no wakeable data collection device 40 among the M1 data collection devices 40 . At this time, the federation management center 30 may not perform the following steps. The federation management center 30 may inform the data demand device 20 that there is no device in the sensor federation that can complete the data collection task.

S10023. The federation management center 30 determines M3 data collection devices 40 from the M2 data collection devices 40 that meet the accuracy of the collected data.

The collection task parameters may include the accuracy range of the data required by the data collection task. The federal management center 30 can determine M3 data collection devices 40 from the M2 data collection devices 40 that meet the data accuracy in the collection task parameters. M3 is an integer, and M3 may be less than or equal to M2.

In a possible implementation manner, M3 may be greater than 1, or equal to 1, or equal to 0. When M3 is equal to 1, the federation management center 30 may not execute the following steps S10024-step S10025. When M3 is greater than 1, the federation management center 30 may execute the following step S10024.

In a possible implementation manner, when M3 is equal to 0, it means that no data collection device 20 can provide the semantic information N. The federation management center 30 may send a message to remind the data demand device 20 that currently no data collection device 20 can receive the data collection task. The federation management center 30 does not execute the following steps S10024-S10025.

S10024. The federation management center 30 determines from the M3 data collection devices 40 that M4 data collection devices 40 whose distance to the data demand device 20 is less than a preset distance threshold.

The task collection parameters may include a distance range between the data collection device 40 and the data demand device 20, and the distance between the data collection device 40 and the data demand device 20 should be smaller than a preset distance threshold. The federation management center 30 may determine M4 data collection devices 40 whose distance from the data demand device 20 is less than a preset distance threshold from the M3 data collection devices 40 . M4 is an integer, where M4 may be less than or equal to M3.

In a possible implementation manner, M4 may be greater than 1 or equal to 1. When M4 is equal to 1, it means that the distance between only one data collection device 40 among the M3 data collection devices 40 and the data demand device 20 is smaller than the preset distance threshold. Then the federation management center 30 can send the data collection task and the parameters of the collection task to the data collection device 40 . The federation management center 30 may not execute the following step S10025. When M4 is greater than 1, it means that among the M3 data collection devices 40, the distance between multiple data collection devices 40 and the data demand device 20 is smaller than the preset distance threshold. The federation management center 30 may execute the following step S10025.

In a possible implementation manner, M4 may be equal to 0, which means that the distance between the data collection device 40 and the data demand device 20 among the M3 data collection devices 40 is less than the preset distance threshold.

S10025. The federation management center 30 is based on the device states of the M4 data collection devices 40. The device states of the data collection devices 40 include the power state of the data collection devices 40, the device type, the wearing state, and the data accuracy of the semantic information N in the data collection devices 40. One or more items of , distance from the data demand device 20 , and determine the target data collection device 40 .

The federation management center 30 can compare the device states of the M4 data collection devices 40 , that is, determine the target data collection device 40 by the device states of the M4 data collection devices. The device state of the data collection device 40 may include the power state of the data collection device 40, the device type (whether it has computing resources or storage resources), the power state, the wearing state, the data accuracy of the semantic information N in the data collection device 40, and the data requirements The distance of the device 20 and whether it has one or more of the standard sensing federation data transmission interface.

In a possible implementation, the federal management center 30 determines the active device according to the power status (active device and passive device) of the M4 data collection devices 40, and the data collection device 40 connected to the power supply is the target Data Acquisition Device 40 .

For example, take M4 equal to 2 as an example for illustration. The M4 data collection devices 40 include a first data collection device 40 and a second data collection device 40 . The first data collection device 40 is an active device (for example, a notebook computer), and the first data collection device 40 is connected to a power supply, the second data collection device is a passive device (for example, a mobile phone), and the first data collection device 40 Power is not connected. Then, based on the power status in the device status, the federation management center 30 can determine that the first data collection device 40 is the target data collection device 40 .

In a possible implementation, the federation management center 30 selects from the M4 data collection devices 40 based on the device types of the M4 data collection devices 40, a data collection device 40 with the most computing resources and the most storage resources as the target Data Acquisition Device 40 .

In a possible implementation, the federation management center 30 may select a data collection device 40 with the most power from the M4 data collection devices 40 as the target data collection device 40 based on the power status of the M4 data collection devices 40 .

In a possible implementation, if the M4 data collection devices 40 are wearable devices (watches, wristbands, earphones, smart glasses, etc.), the federation management center 30 may base on the equipment types of the M4 data collection devices 40 (whether There are computing resources or storage resources), battery status, wearing status, data accuracy of semantic information N in the data collection device 40, distance from the data demand device 20, and whether there is a standard sensor federation data transmission interface, from M4 Select one of the data acquisition devices 40 with the most computing resources, the most storage resources, the most power, the wearing state, the highest data accuracy of semantic information N, the closest distance to the data demand device 20, and a standard sensor federation data transmission interface. One data collection device 40 serves as the target data collection device 40 .

In a possible implementation, if the M4 data collection devices 40 are non-wearable devices, the federation management center 30 can base the M4 data collection devices 40 on the power status, device type (whether has computing resources or storage resources), power state, the data accuracy of the semantic information N in the data collection device 40, the distance from the data demand device 20, and whether there is a standard sensor federation data transmission interface, select one of the M4 data collection devices 40 to connect to the power supply, and calculate The target data collection device 40 is a data collection device 40 with the most resources, the most storage resources, the most power, the highest data accuracy of the semantic information N, and the closest distance to the data demand device 20 , and equipped with a standard sensor federation data transmission interface.

S1004. The federation management center 30 sends the collection task parameters to the data collection device 40.

The federation management center 30 may send the data collection task and the collection task parameters to the determined target data collection device 40 . Specifically, if there is a sensor federation engine module in the data collection device 40 , the federation management center 30 can directly send the data collection task and the collection task parameters to the data collection device 40 . If there is no sensing federation engine module in the data collection device 40, the federation management center 30 can decompose the data collection task (for example, obtaining user identity data before air conditioning) and the collection task parameters into the collection subtasks of the data collection device 40 (for example, Take a user image), and then send the collection subtask to the data collection device 40.

The sensor federation engine module can parse the collection task parameters into data collection parameters in the data collection device 40 . For example, the “user identity” in the task collection parameters of the sensor federation engine module is resolved to the data collection parameter “user face image” in the data collection device 40 . For the specific functions of the sensor federation engine module, please refer to the description in FIG. 4 above, which will not be repeated here.

Regarding how the federation management center 30 sends the collection task parameters to the data collection device 40, please refer to the descriptions in FIGS. 13A-13C below, and details will not be repeated here.

S1005, the data collection device 40 determines whether to accept the data collection task based on the collection task parameters and the device capabilities of the data collection device 40; if accepted, execute step S1006a; if not, execute step S1006b.

The data collection device 40 can evaluate the resources (computing resources and storage resources) to be used if the data collection device 40 needs to execute the data collection task based on the collection task parameters. Then, the data collection device 40 determines whether the remaining resources in the data collection device 40 are larger than the resources to be used; if the remaining resources are larger than the resources to be used, the data collection device 40 can accept the data collection task. When the data collection device 40 accepts the data collection task, the data collection device 40 may execute step S1006a. If the remaining resources are less than the resources to be used, the data collection device 40 may not accept the data collection task. When the data collection device 40 does not accept the data collection task, the data collection device 40 may execute step S1006b.

As for whether the data collection device 40 accepts the data collection task specifically, please refer to the description in FIG. 14A-14B below, and details will not be repeated here.

Optionally, in a possible implementation manner, the remaining resources of the data collection device 40 may refer to remaining resources in the resources reserved for sensor federation in the data collection device 40 . It can be understood that, after the data acquisition device 40 joins the sensor federation, certain computing resources and storage resources may be reserved for the sensor federation.

S1006a, the data collection device 40 establishes a communication connection with the data demand device 20 , and the data collection device 40 sends the collected sensor data to the data demand device 20 .

After the data collection device 40 accepts the data collection task of the data demand device 20, if a communication connection has been established between the data collection device 40 and the data demand device 20, the data collection device 4 can collect the collected sensor data, or based on the collected sensor data The obtained semantic information is sent to the data demand device 20 . If there is no communication connection established between the data collection device 40 and the data demand device 20, the data collection device 40 may establish a connection with the data demand device 20, and then send the collected sensor data or the semantic information based on the collected sensor data to To the data demand device 20. Alternatively, if no communication connection is established between the data collection device 40 and the data demand device 20, the data collection device 40 can send the collected sensor data, or the semantic information based on the collected sensor data, to the data demand device through the federal management center 30. device 20.

Here, how the data collection device 40 sends the collected sensor data to the data demand device 20 may refer to the descriptions of FIGS. 15A-15C below, and details will not be repeated here.

S1006b. The data collection device 40 sends a message of refusing to accept the data collection task to the federation management center 30 .

When the remaining resources in the data collection device 40 are less than the resources to be used required by the data collection task, the data collection device 40 may refuse to accept the data collection task. Or, when the sensor in the data collection device 40 used to collect the data required in the data collection task is in working state and is occupied by another task, the data collection device 40 can also refuse to accept the data collection task.

The data collection device 40 may send a message of refusing to accept the data collection task to the federation management center 30 . The federation management center 30 may also send a message to the data demand device 30 that the data collection device 40 refuses to accept the data collection task. Alternatively, the federation management center 30 may determine another data collection device 40 from the sensor federation to complete the data collection task.

In some scenarios, the data collection device 40 may have a sensory federation engine module. The sensor federation engine module in the data collection device 40 can analyze the collection task parameters and the data collection task, and can determine whether to accept the data collection task according to the collection task parameters and the device status of the data collection device 40 .

Fig. 13A exemplarily shows how the federation management center 30 sends a data collection task to the data collection device 40 when there is a sensor federation engine module in the data collection device 40, and how the data collection device 40 determines whether to Flowchart of specific steps for accepting data collection tasks. As shown in Figure 13A, the federal management center 30 sends the collection task parameters to the data collection device 40, and the data collection device 40 determines to accept the data collection task may include the following steps:

S10031a, the federation management center 30 determines that the data collection device 40 has a sensor federation engine module.

The federation management center 30 may store the device information of the data demand device 20 and the device information of the data collection device 40 in the sensor federation. The federation management center 30 may determine according to the device information of the data collection device 40 that the data collection device 40 has a sensor federation engine module. Alternatively, when the data collection device 40 joins the sensor federation created by the federation management center 30, the data collection device 40 may report to the federation management center 30 that the data collection device 40 has a sensor federation engine module.

S10032a, the federation management center 30 sends the task collection parameters to the sensor federation engine module in the data collection device 40 .

The federation management center 30 can send the task collection parameters to the sensing federation causing module in the data collection device 40 .

S10033a, the sensor federation engine module in the data collection device 40 analyzes the task collection parameters, and maps the task collection parameters to the data collection parameters in the data collection device 40 .

The sensor federation engine module in the data collection device 40 can analyze the task collection parameters. Specifically, the task analysis module in the sensor federation engine module can map the task collection parameters to the data collection parameters in the data collection device 40 . For example, the task parsing module in the sensor federation engine module can decompose the semantic information "user identity" required in the data collection task into the specific collection parameters of the data collection device 40 (for example, the specific collected data is "image collected by camera") ", as well as shooting several images, resolution, image compression ratio and other parameters). The task parsing module in the sensor federation engine module can map the semantic information "average heart rate" required by the data collection task to the collection cycle and collection frequency of the device. For example, the PPG sensor of the watch has a fixed collection frequency of 1Hz, so the parameters may be Yes: The collection time is 5s, the collection frequency is 1Hz, and the collection operator is to average the data collected within 5s. It can be understood that the task analysis module in the sensor federation engine can analyze the semantic information in the task collection parameters to obtain the data or low-level semantic information collected by the sensor that generates or derives the semantic information.

S10034a, the sensing federation engine module in the data collection device 40 determines the resources to be used corresponding to the data collection task based on the data collection parameters; The management center 30 replies to accept the data collection task.

The sensor federation engine module in the data collection device 40 can determine the resources (which may be referred to as resources to be used) to be used by the data collection task based on the data collection parameters. The sensor federation engine module in the data collection device 40 can determine the remaining resources in the data collection device 40 based on the working status of the data collection device 40 (for example, the number of tasks being executed). In the case that the remaining resources in the data collection device 40 are greater than the resources to be used, the data collection device 40 may reply to the federation management center 30 to accept the data collection task. Alternatively, the data collection device 40 may directly establish a communication connection with the data demand device 20 . In the case that the remaining resources in the data collection device 40 are greater than the resources to be used, the data collection device 40 may reply to the federation management center 30 that the data collection task is not accepted.

In other scenarios, the data collection task in the data demand device 20 has been sent to the data collection device 40 times, and the number of times the data collection device 40 refuses to execute the data collection task is greater than a preset threshold. At this time, the federal management center 30 determines whether the data collection device 40 accepts the data collection task.

FIG. 13B exemplarily shows a specific flowchart of how to determine whether the data collection device 40 accepts the data collection task this time when the number of times the data collection device 40 has refused to execute the data collection task is greater than a preset threshold. As shown in FIG. 13B , in the case that the number of times the data collection device 40 has refused to execute the data collection task is greater than the preset threshold, determining that the data collection device 40 accepts the data collection task this time may include the following steps:

S10031b. The federal management center 30 determines that the data collection task request carries an arbitration mode identifier, and the arbitration mode identifier is used to indicate that the data collection device 40 refuses to accept the data collection task in the data demand device 20. The number of times the federation management center 30 based on The device capability of the data collection device 40 determines whether the data collection device 40 accepts the data collection task.

The data collection task sent by the data demand device 20 may carry an arbitration mode identifier. When the data collection device 40 corresponding to the data collection task in the data collection device 20 rejects the threshold of the number of times the data collection task is greater than the preset threshold, when the data demand device 20 sends the data collection task to the federation management center 30 again, it can carry Arbitration mode flag. The arbitration mode flag may be used to indicate that the data collection device 40 rejects the data collection task in the data demand device 20 for a number of times exceeding a preset threshold.

Optionally, the arbitration mode identifier may carry the device information of the data collection device 40 that rejects the data collection task and the number of times of rejection.

The federation management center 30 can determine the arbitration mode identifier carried in the data collection task sent by the data demand device 20 .

S10032b. The federation management center 30 sends the task collection parameters and the arbitration mode identifier to the sensor federation engine module in the data collection device 40 .

The federation management center 30 may send the task collection parameters and the arbitration mode identifier to the data collection device 40 . The arbitration mode identifier may instruct the data collection device 40 to send the remaining resources and the resources to be used required by the data collection task to the federation management center 30 .

Optionally, when the data collection device 40 does not have a sensor federation engine module, the federation management center 30 can send an instruction a to the data collection device 40, and the instruction a can be used to instruct the data collection device 40 to send the data collection device 40 The remaining resources are sent to the federation management center 30.

S10033b. The sensor federation engine module in the data collection device 40 analyzes the task collection parameters, and maps the task collection parameters to the data collection parameters in the data collection device 40 .

For this step, reference may be made to the above step S10033a, which will not be repeated here.

S10034b. The sensor federation engine module in the data collection device 40 determines the resources to be used corresponding to the data collection task based on the data collection parameters; the data collection device 40 sends the resources to be used corresponding to the data collection task and the remaining resources of the data collection device 40 to Federal Management Center 30.

The sensor federation engine module in the data collection device 40 can determine the to-be-used resources corresponding to the data collection task based on the data collection parameters. Here, reference can be made to the description in the above step S10034a, which will not be repeated here.

The data collection device 40 may send the resources to be used corresponding to the data collection task and the remaining resources of the data collection device 40 to the federation management center 30 .

S10035b. The federation management center 30 determines that the data collection device 40 accepts the data collection task based on the resources to be used corresponding to the data collection task and the remaining resources of the data collection device 40 .

Specifically, if the remaining resources in the data collection device 40 are greater than the resources to be used corresponding to the data collection task; the federation management center 30 may determine that the data collection device 40 can receive the data collection task. Further, the federation management center 30 may notify the data collection device 40 to accept the data collection task.

If the remaining resources in the data collection device 40 are less than the resources to be used corresponding to the data collection task, the federation management center 30 may determine that the data collection device can reject the data collection task. Further, the federation management center 30 may notify the data collection device 40 that it does not need to receive the data collection task.

Optionally, the data collection device 40 may report the remaining resources and the list of tasks being executed in the data collection device 40 to the federation management center 30 . When the remaining resources in the data collection device 40 are less than the resources to be used corresponding to the data collection task, the federation management center 30 can compare the priority of the data collection task with the tasks in the task list being executed in the data collection device 40 . If the priority of the data collection task is higher than the priority of the tasks in the task list being executed in the data collection device 40, the federation management center 30 can notify the data collection device 40 to suspend or delete the task with lower priority in the task list being executed. task, and receive the data collection task.

In this way, some data collection devices 40 in the sensing federation can be prevented from maliciously rejecting the data collection tasks sent by the data demand device 20 .

In some scenarios, the data collection device 40 does not have a sensor federation engine module, the data collection device 40 needs to resolve the data collection task through the federation management center 30, and needs to send the collected sensor data to the data demand device through the federation management center 30 20. Then the federation management center 30 needs to determine whether the data collection device 40 accepts based on the resource status of the federation management center 30 and the new resources to be used that the federation management center 30 helps the data collection device 40 to forward the collected sensors to the data demand device 20. data collection tasks.

FIG. 13C exemplarily shows a flow chart of specific steps of how the federation management center 30 determines whether the data collection device 40 accepts the data collection task when there is no sensor federation engine module in the data collection device 40 . As shown in Figure 13C, the federal management center 30 determines that the data collection device 40 accepts the data collection task may include the following steps:

S10031c, the federation management center 30 determines that the data collection device 40 does not have a sensor federation engine module.

The federation management center 30 may determine that the data collection device 40 does not have a sensor federation engine module according to the stored device information of the data collection device 40 .

S10032c. The federation management center 30 parses the task collection parameters and data collection tasks, and parses the data collection tasks into a list of data collection subtasks of the data collection device 40 .

The federation management center module in the federation management center 30 can resolve task collection parameters and data collection tasks, and can parse the data collection tasks into the data collection subtask list of the data collection device 40. For example, the federation management center 30 may parse the data collection task “user identity” into a list of data collection subtasks in the data collection device 40 . The data collection subtask list may include the data collection subtask "collect user's face image" and the data collection subtask "according to the user's face image to determine user identity (children, young and middle-aged people, old people, etc.)".

S10033c, the federation management center 30 evaluates new resources to be used of the federation management center 30 based on the data collection subtask list.

The federation management center 30 can evaluate the resources that the federation management center 30 needs to invest in the data collection device 40 when the data collection device 40 completes the data collection subtasks in the collection subtask list based on the data collection subtask list (which can be referred to as new Incrementally use resources). The newly added resources to be used may include the resources that the federation management center 30 needs to use to send the sensor data collected by the data collection device 40 to the data demand device 20, and the federation management center 30 converts the sensor data collected by the data collection device 40 into data collection Semantic information required by a task requires resources to be used.

S10034c, the federal management center 30 determines to start the proxy data collection mode based on the newly added resources to be used and the resource status of the federal management center 30, and sends the collection subtask list to the data collection device 40; the proxy data collection mode refers to the federal management center 30 Send the data collected by the data collection device 40 to the data demand device 20 ;

The federation management center 30 may determine whether to enable the proxy data collection mode based on newly added resources to be used and the resource status of the federation management center 30 . The federal management center 30 may also send the collection subtask list to the data collection device 40 .

In a possible implementation manner, when the new resources to be used in the federation management center 30 are less than the remaining resources in the federation management center 30, the federation management center 30 determines to enable the proxy data collection mode. In this embodiment of the application, the proxy data collection mode may mean that the federal management center 30 sends the data collected by the data collection device 40 to the data demand device 20; The processed data (for example, converting sensor data into semantic information conforming to data collection parameters) is sent to the data demand device 20 .

In a possible implementation, when the federation management center 30 is in the cloud server, and the newly added resources to be used by the federation management center 30 are greater than the remaining resources in the federation management center 30, the federation management center 30 can apply to the cloud server More resources. After the federation management center 30 applies for more resources, the federation management center can start the proxy data collection mode.

In a possible implementation, the federation management center 30 is deployed in an electronic device, and can apply for resources from other electronic devices in the sensor federation for proxy data collection or proxy model training or derivation of new semantics.

In this way, when the data collection device 40 cannot directly establish a communication connection with the data demand device 20, and the data collection device 40 does not have a sensor federation engine module, the data collection device 40 can also complete the data collection task in the data demand device 20 .

In some scenarios, the data collection device 40 may directly send the sensor data collected based on the data collection task to the data demand device 20 . When the federal management center 30 determines that the data collection device 40 completes the data collection task in the data demand device 20 , the data demand device 20 can directly establish a data transmission channel with the data collection device 40 .

FIG. 14A exemplarily shows specific steps for establishing a data transmission channel between the data collection device 40 and the data demand device 20 . As shown in Figure 14A, after the federation management center 30 determines that the data collection task of the data demand device 20 is completed by the data collection device 40, the federation management center 30, the data collection device 40 and the data demand device 20 can perform the following steps:

S10041a, the federation management center 30 triggers the sensor federation engine module in the data demand device 20 and the data collection device 40 to start.

The federation management center 30 can trigger the sensory federation engine module in the data demand device 20 to start. Specifically, the federation management center 30 can send an instruction b to the data demand device 20, and the instruction b can be used to instruct the data demand device 20 to start the sensor federation engine module. The data demand device 20 may receive the instruction b, and start the sensor federation engine module based on the instruction b.

The federation management center 30 can trigger the sensor federation engine module in the data collection device 40 to start. Specifically, the federation management center 30 may send an instruction c to the data collection device 40, and the instruction c may be used to instruct the data collection device 40 to start the sensor federation engine module. The data collection device 40 may receive the instruction c, and start the sensor federation engine module based on the instruction c.

S10042a, the sensor federation engine module of the data collection device 40 and the sensor federation engine module of the data demand device 20 establish a data transmission channel.

The sensor federation engine module of the data collection device 40 can establish a data transmission channel with the sensor federation engine module of the data demand device 20 . In a possible implementation manner, the sensor federation engine module in the data collection device 40 may send a request for establishing a data transmission channel to the sensor federation engine module in the data demand device 20 . After the sensor federation engine module in the data demand device 20 receives the request for establishing a data transmission channel, the sensor federation engine module in the data demand device 20 can reply to the sensor federation engine module in the data collection device 40 for approval Establish a data transmission channel. Then, the sensor federation engine module in the data collection device 40 and the sensor federation engine in the data demand device 20 can successfully establish a data transmission channel.

S10043a, the data collection device 40 obtains the device information of the data demand device 20 , and the data demand device 20 obtains the device information of the data collection device 40 .

Optionally, after the sensor federation engine module in the data collection device 40 and the sensor federation engine in the data demand device 20 can successfully establish a data transmission channel, the data collection device 40 can send the data collection device 40 to the data demand device 20. device information. The data demand device 20 may also send the device information of the data demand device 20 to the data collection device 40 .

In some scenarios, the data collection device 40 may send the sensor data collected based on the data collection task to the federation management center 30 , and then the federation management center 30 sends the sensor data to the data demand device 20 . When the federal management center 30 determines that the data collection device 40 completes the data collection task in the data demand device 20, the federation management center 30 needs to establish data transmission channels with the data demand device 20 and the data collection device 40 respectively.

FIG. 14B exemplarily shows specific steps for establishing data transmission channels between the federation management center 30 and the data demand device 20 and the data collection device 40 respectively. As shown in Figure 14B, after the federation management center 30 determines that the data collection task of the data demand device 20 is completed by the data collection device 40, the federation management center 30, the data collection device 40 and the data demand device 20 can perform the following steps:

S10041b. The federation management center 30 triggers the sensor federation engine in the data demand device 20 and the data collection device 40 to start.

For step S10041b, reference may be made to the description in the above step S10041a, which will not be repeated here.

S10042b. The federation management center 30 establishes a data transmission channel with the data demand device 20 , and the federation management center 30 establishes a data transmission channel with the data collection device 40 .

The federation management center module (or the sensory federation engine module) in the federation management center 30 can establish a communication connection with the sensory federation engine module in the data demand device 20 . In a possible implementation manner, the federation management center module (or the sensory federation engine module) in the federation management center 30 may send a request for establishing a data transmission channel to the sensory federation engine module in the data demand device 20 . After the sensing federation engine module in the data demand device 20 receives the request for establishing a data transmission channel, the sensor federation engine module in the data demand device 20 can send a federation management center module in the federation management center 30 (or the federation management center module in the federation management center 30). Sense federation engine module) replies to agree to establish a data transmission channel. Then, the federation management center module (or sensory federation engine module) in the federation management center 30 and the sensory federation engine in the data demand device 20 can successfully establish a data transmission channel.

S10043b, the data collection device 40 obtains the device information of the data demand device 20 through the federation management center 30 , and the data demand device 20 obtains the device information of the data collection device 40 through the federation management center 30 .

Optionally, after the federation management center 30 establishes a data transmission channel with the data demand device 20, the data demand device 20 may send the device information of the data demand device 20 to the federation management center 30 through the data transmission channel. Then, the federation management center 30 can send the device information of the data demand device 20 to the data collection device 40 . The data collection device 40 can also send the device information of the data collection device 40 to the federal management center 30, and after the federation management center 30 receives the device information of the data collection device 40, it can send the device information of the data collection device 40 to the data demander. device 20.

In some possible implementations, after the data demand device 20, the federation management center 30 and the data collection device 40 perform the above step S10043b, the data collection device 40 can send the collected sensor data to the data demand device through the federation management center 30 20. FIG. 15A exemplarily shows specific steps for the data collection device 40 to send the collected sensor data to the data demand device 20 through the federation management center 30 . As shown in Figure 15A, the data collection device 40 sends the collected sensor data to the data demand device 20 through the federation management center 30 may include the following steps:

S10051a. The sensors in the data collection device 40 collect data based on the data collection subtask list, and send the sensor data collected by the sensors to the federation management center 30 .

When the federation management center 30 starts the proxy data collection mode, the federation management center 30 can parse the data collection task into a data collection subtask list of the data collection device 40 . The data collection subtask list may include one or more data collection subtasks. Reference can be made here to the description of FIG. 13C above. The sensors in the data collection device 40 can collect data based on the data collection subtask list, and send the sensor data collected by the sensors, such as sensor data data100 , to the federation management center 30 .

S10052a. The federation management center 30 processes the sensor data based on the collection task parameter collection operator to obtain a data collection result.

In the case that the acquisition task parameters include the acquisition operator, after the federation management center 30 receives the sensor data, such as the sensor data data100, it can process the sensor data based on the acquisition operator to obtain the data acquisition result, such as the sensor data data200 . The collection operator may be averaging the sensor data collected within 10 seconds, or taking the maximum value of the sensor data collected within 10 seconds, etc. The embodiment of the present application does not limit the collection operator.

Optionally, the federal management center 30 can also convert the sensor data data200 into semantic information.

S10053a, the federation management center 30 sends the data collection result to the data demand device 20 .

The federation management center 30 can send the data collection result (for example, sensor data data200 ) to the data demand device 20 . Alternatively, the federal management center 30 can convert the sensor data data200 into semantic information, and then send the semantic information to the data demand device 20 .

In some scenarios, when the data collection device 40 collects data based on the data collection task, and the data collection device 40 is abnormal, the data collection device 40 may report the abnormal event to the federation management center 30 .

FIG. 15B exemplarily shows the specific steps of how the data collection device 40 and the federation management center 30 handle abnormal events occurring in the data collection device 40 . As shown in FIG. 15B, when the data collection device 40 is abnormal in collecting data, the data collection device 40 and the federation management center 30 can perform the following steps:

S10051b. The data collection device 40 sends the collected sensor data to the data demand device 20 .

The data collection device 40 can directly send the collected sensor data to the data demand device 20 .

Optionally, the data collection device 40 may send the collected sensor data to the federation management center 30, and the federation management center 30 processes the sensor data based on the collection operator, and then sends the processed sensor data to the data demand device. Specifically, reference may be made to the descriptions in the above step S10051a-step S10053a, which will not be repeated here.

S10052b. The data collection device 40 reports the abnormal state to the federal management center 30. The abnormal state includes the shutdown of the data collection device 40, the power failure of the data collection device 40, the power of the data collection device is lower than the preset power threshold, and the data collection device is not in the wearing state. one or more of .

When the data collection device 40 is abnormal when collecting data, the data collection device 40 cannot continue to collect data normally based on the data collection task. The data collection device 40 can report the specific abnormal state to the federation management center. The abnormal state may include one or more of the data collection device 40 being shut down, the data collection device 40 being powered off, the power of the data collection device 40 being lower than a preset power threshold, and the data collection device 40 not being worn.

S10053b. The federation management center 30 searches for a data collection device 40 that meets the data collection parameters.

The federation management center 30 can search for other data collection devices 40 that meet the data collection parameters again. For details, reference may be made to the description of FIG. 12 above, and details are not repeated here.

Optionally, in some scenarios, the data collection device 40 may cache the collected sensor data, and delete sensor data exceeding the collection time. The data collection device 40 can also send the cached sensor data to the federation management center 30 . FIG. 15C exemplarily shows a schematic flow chart of processing the collected sensor data by the data collection device 40 . As shown in FIG. 15C, the data acquisition device 40 may perform the following steps:

S10051c. The data collection device 40 caches the collected sensor data in the data collection device 40, and deletes the sensor data exceeding the collection time.

The data collection device 40 may cache the collected sensor data in the data collection device 40, and delete the sensor data exceeding the collection time.

Optionally, the data collection device 40 may execute the following step S10052c, and the federation management center 30 may execute the following step S10053c.

S10052c, the data collection device 40 sends the cached sensor data to the federation management center 30 .

In some possible implementations, when the sensor data cache time in the data collection device 40 exceeds the collection time, or the storage space occupied by the sensor data is greater than the preset storage space, the data collection device 40 can send the cached sensor data to the federation management center 30 .

S10053c. The federation management center 30 receives and saves the sensor data, and releases the saved sensor data after the collection period.

The federation management center 30 can receive the sensor data sent by the data collection device 40 and save the sensor data. Optionally, the federation management center 30 may delete the sensor data after the collection period.

Optionally, in a possible implementation manner, the federation management center may store semantic information required in the data collection task, and the federation management center may directly send the stored semantic information to the data demand device. The federal management center does not need to perform the above steps.

In a possible implementation manner, the data collection device 40 may proactively report the collected abnormal data.

In a possible implementation manner, the federation management center 30 may store some sensing data sent by one or more devices in the sensing federation.

Further, the federation management center can also periodically refresh the stored sensory data sent by one or more devices in the sensory federation. Specifically, the federation management center can allow one or more devices to periodically send some sensing data to the federation management center.

The following describes the data collection method provided by the embodiment of the present application in some specific scenarios.

Fig. 16 exemplarily shows a specific usage scenario of the data collection method provided by the embodiment of the present application. As shown in FIG. 16 , a data collection system (or called sensor federation) provided in the embodiment of the present application may include a mobile phone 104, an air conditioner 101, a camera 102, and a TV 107 (the TV 107 includes a camera). Wherein, the mobile phone 104 may be an electronic device where the federation management center is located. The air conditioner 101 may be a data demand device in the data collection system. The camera 102 and the TV set 107 may be data collection devices in the data collection system. Regarding how the mobile phone 104, the air conditioner 101, the camera 102, and the television 107 set up a data acquisition system, reference can be made to the description above, and details will not be repeated here.

As shown in FIG. 16 , the air conditioner 101 can send a data collection task to the mobile phone 104 . The data collection task may include data collection parameters, for example, the data to be collected is user identity and number of users, which needs to be collected in real time and allows multi-device collection. The mobile phone 104 can receive the data collection task, and analyze the data collection task into data collection subtask 1 (collect image data in real time) and data collection subtask 2 (collect user identities and numbers in real time).

The mobile phone 104 can send the data collection subtask 1 to the camera 102 , and send the data collection subtask 2 to the TV 107 . Regarding how the mobile phone 104 determines that the data collection device corresponding to the data collection subtask 1 is the camera 102, and how to determine that the data collection device corresponding to the data collection subtask 2 is the TV 107, please refer to the descriptions in FIGS. 10-12 above. I won't repeat them here.

After receiving the data collection subtask 1, the camera 102 may collect user images based on the data collection subtask 1. In a possible example, the camera 102 does not have a sensor federation engine module, and the mobile phone 104 needs to convert the user image captured by the camera into corresponding semantic information "user identity and number of users" and send it to the air conditioner 101. Here, for details, reference may be made to the proxy data collection mode described above in FIG. 13C , which will not be repeated here.

After receiving the data collection subtask 2, the television 107 may collect user images based on the data collection subtask 2. In a possible example, the TV 107 is equipped with a sensor federation engine module. The TV 107 can convert the user image into corresponding semantic information "user identity and number of users". The television set 107 sends the obtained semantic information "user identity and number of users" to the air conditioner 101 . Here, reference may be made to the description of the data collection device 40 directly sending the collected data to the data demand device 20 in FIG. 13A above, which will not be repeated here. The data collection shown in Figure 16 will be described in detail below in conjunction with the data demand device, data collection device, sensor federation engine module, and software architecture of the federation management center in the data collection system shown in Figures 3-6 above. The specific software module implementation process in the scenario.

FIG. 17 shows a schematic diagram of a specific software model implementation process of the data collection scenario provided in FIG. 16 .

As shown in FIG. 17 , the air conditioner 101 may include a sensor federation engine module 1600 . The air conditioner 101 may generate a data collection task through the sensor federation engine module 1600 . Specifically, the air conditioner 101 can generate a data collection task through the task generation module 1602 in the sensor federation engine module 1600.

As shown in FIG. 17 , the sensor federation engine module 1600 in the air conditioner 101 can send the data collection task to the federation management center 1700 in the mobile phone 104 .

As shown in FIG. 17 , the mobile phone 104 may have a federal management center 1700 . The federation management center 1700 can receive the data collection task sent by the sensor federation engine module 1600 in the air conditioner 101 . The federation management center 1700 can analyze the data collection task, and analyze the semantic information required by the data collection task. Then, the semantic information corresponding to the semantic information and the data collection device that provides the semantic information can be searched based on the parsed semantic information. Then, the federation management center 1700 can send the data collection task to the corresponding data collection device.

Specifically, in a possible implementation, the task parsing module 1701 in the federation management center 1700 can perform data collection tasks (for example, the data to be collected is user identity and number of users, real-time collection is required, and multi-device collection is allowed ) is parsed into data collection subtask 1 (for example, collecting image data in real time) and data collection subtask 2 (for example, collecting user identities and numbers in real time). The task analysis module 1701 can send the parsed data collection subtask 1 (for example, collect image data in real time) and data collection subtask 2 (for example, collect user identities and numbers in real time) to the task assignment and scheduling module 1702 . Then, the task allocation and scheduling module 1702 may send the received semantic information required by the data collection subtask 1 and the semantic information required by the data collection subtask 2 to the semantic search module 1703 .

The semantic search module 1703 can search in the sensing capability management model 1704 based on the semantic information required in the data collection subtask 1 and the semantic information required in the data collection subtask 2, and determine the information required in the data collection subtask 1. The semantic information corresponding to the semantic information of the semantic information and the data acquisition device (for example, camera 102) that provides the semantic information, and the semantic information corresponding to the semantic information required for the data acquisition subtask 2 and the data acquisition device that provides the semantic information ( For example, television 107).

Further, in a possible implementation, the semantic search module 1703 may send the device information of the data collection device corresponding to the data collection subtask 1 and the device information of the data collection device corresponding to the data collection subtask 2 to the task assignment Scheduling module 1702 .

The task allocation and scheduling module 1702 may send the data collection subtask 1 to the data collection device (for example, the camera 102 ) corresponding to the data collection subtask 1 based on the device information of the data collection device corresponding to the data collection subtask 1 . The task allocation and scheduling module 1702 may send the data collection subtask 2 to the data collection device corresponding to the data collection subtask 2 (for example, the TV 107 ) based on the device information of the data collection device corresponding to the data collection subtask 2 .

As shown in FIG. 17 , the camera 102 may include a data collection module 1801 and a data transmission interface 1802 . The camera 102 may receive the data collection subtask 1 sent by the mobile phone 104 . Specifically, the data collection module 1801 in the camera 102 may receive the data collection subtask 1 sent by the federal management center 1700 in the mobile phone 104 . The data collection module 1801 can collect user images based on the data collection subtask 1 . The data collection module 1801 may send the user image to the data transmission interface 1802 . The data transmission interface 1802 can establish a data transmission channel with the federation management center 1700 of the mobile phone 104 .

Optionally, in a possible implementation manner, specifically, the data transmission interface 1802 may establish a data transmission channel with the interface adaptation module 1705 in the federation management center 1700 of the mobile phone 104 . The data transmission interface 1802 can send the user image to the interface adaptation module 1705 . The interface adaptation module 1705 may send the user image to the subscription and query module 1706 .

Optionally, in a possible implementation manner, the federation management center 1700 includes a data analysis module (not shown in FIG. 17 ). For details about the data analysis module, reference may be made to the description in FIG. 4 above. The interface adaptation module 1705 can send the user image to the data analysis module. The data analysis module can convert the user image into user identity and number of users, and send it to the subscription and query module 1706 .

As shown in Figure 17, the subscription and query module 1706 can convert the user image into user identity and number of users, and send the user identity and number of users to the data access module in the sensing federation engine module 1600 of the air conditioner 101 1601.

As shown in FIG. 17 , the TV 107 can receive the data collection subtask 2 sent by the mobile phone 104 . Specifically, the TV 107 may include a sensor federation engine module 1900 and a data collection module 1905 . The sensor federation engine module 1900 may include a task analysis module 1901 , a subscription and query module 1902 , a data analysis module 1903 , and a data access module 1904 .

In a possible implementation manner, the task parsing module 1901 in the sensing federation engine module 1900 of the TV set 107 may receive the data collection subtask 2 sent by the mobile phone 104 . The task parsing module 1901 can parse the data collection subtask 2 into a task of collecting user images, and instruct the data collection module 1905 to collect user images. The data collection module 1905 can send the collected user images to the data access module 1904 . The data access module 1904 can send the user image to the data analysis module 1903 . The data analysis module 1903 can convert user images into user identities and user numbers. The data analysis module 1903 can send the user identity and the number of users to the subscription and query module 1902 , and the subscription and query module 1902 sends data such as the user identity and the number of users to the air conditioner 101 .

As shown in FIG. 17 , the air conditioner 101 may receive data such as user identity and number of users sent by the television 107 . Specifically, the data access module 1601 in the sensing federation engine module 1600 of the air conditioner 101 can receive data such as user identity and number of users sent by the subscription and query module 1902 in the TV set 107 .

After the air conditioner 101 collects the target data, it can transmit the target data to the upper-layer intelligent mode adjustment module. The intelligent adjustment module in the air conditioner 101 intelligently selects a working mode suitable for the user according to the received result information. Through the data acquisition method provided by the embodiment of the present application, the intelligent application of the air conditioner does not need to preset the device type and signal that can provide effective information, nor does it need to preset the ability to access the target interface, and the target data can be determined through the federal management center (i.e. data collection The data required in the task) corresponds to the data acquisition device and obtains the target data. In this way, the preparation cost of the air conditioner with high-order intelligent functions can be greatly reduced.

After the interface adaptation module in the federal management center of the mobile phone 104 receives the adaptation acquisition task parameters, the interface adaptation module can pass the interface of the target data acquisition device (for example, the data transmission interface 1802 of the camera 102 ) for data access. The federation management center of the mobile phone 104 processes the returned result parameters according to the configured preprocessing operator, and synchronizes the target data to the sensor federation engine module of the air conditioner 101 through the data subscription module of the federation management center. The sensor federation engine module of the air conditioner 101 can provide the target data to the upper intelligent application of the air conditioner 101 for use.

Through the data collection method provided in the embodiment of the present application, the decoupling of the data access capability of the electronic device (for example, the camera 102 ) that does not have a federation engine module can be realized. At the same time, the operator adaptation capability provided by the federal management center also reduces the requirements for air-conditioning bandwidth and computing power.

In some scenarios, after the federation management center of the mobile phone 104 searches for the target data collection devices as the TV 107 and the camera 102, the federation management center can initiate a data collection request to the two target data collection devices. The television 107 that receives the data collection request can make a decision to determine whether to accept the data collection task. Specifically, the TV 107 can obtain the relevant parameters of the data collection task through the task parsing module in the sensory federation engine module, and then send the request parameters to the task allocation and scheduling module in the sensory federation engine module. The task allocation and scheduling module in the sensing federation engine module can decide whether to accept the task based on the remaining resources of the federated resource pool reserved by the device.

In the embodiment of this application, the purpose of the data collection device (for example, TV 107) making a decision whether to accept the data collection task is that the data collection device accepts the data collection task in the sensor federation without affecting the normal function of the data collection device itself. run. Especially for complex smart devices, due to the variety of computing tasks of the device itself and the fluctuation of computing and storage resources, too many collection tasks may have a great impact on the operating efficiency of the device. In various possible embodiments, the task scheduling module in the data collection device may use different public or private algorithms to make decisions, and may also rely entirely or partially on the remaining resources of the federated resource pool to make decisions. The embodiment of the present application does not limit the specific decision-making manner.

In this embodiment, after receiving the task request parameters, the task scheduling module in the sensor federation engine of the data collection device will inquire about the remaining resources and task queues of the current federated resource pool, and estimate that new tasks need to be added. LF. The resources include: computing resources (CPU resources), storage resources (RAM, ROM, etc.), and sensor access channel resources. For example. Commercial camera sensors generally only support two-channel signal access, and do not support simultaneous multi-channel signal access. When the remaining resources can meet the new resource requirements, the task scheduling module decides to accept the task, then feeds back to the federation management center and starts the corresponding collection action through the operating system service of the device itself. When the remaining resources are not enough to meet the new resource requirements, for example, the camera channel of the current TV 107 is being occupied by the AI fitness application of the TV 107 and the collection cannot be started, then the sensing federation engine module in the TV 107 can report the rejection result and the reason The code is fed back to the Federal Management Center. The federation management center can record the result of the task application to the database module, and feed it back to the data-demanding device (for example, the air conditioner 101).

In the data sending method provided in the embodiment of the present application, when the task request of the data-demanding device fails, the federation management center can determine that other data collection devices in the sensor federation that can meet the data collection task requirements complete the data collection task. For example, when the TV 107 refuses to accept the data collection task of the air conditioner 101, the federal management center calls another data collection device that meets the data collection task during the capability search process.

Through the data collection method provided by the embodiment of the present application, a mechanism for guaranteeing the resources of the data collection device can be realized, and the stable operation of the main functions of the data collection device can be guaranteed.

It can be understood that, in the embodiment of the present application, the data acquisition tasks include not only tasks requiring real-time acquisition of sensing data, but also tasks requiring non-real-time and long-term acquisition of sensing data.

The real-time data collection task in this application is not limited to the data collection task that the air conditioner obtains the identity information and the number of users in real time in the above example. Exemplarily, in some scenarios, some devices may initiate a real-time data collection task, and determine whether the user is in a hypothermia state according to the collected sensor data. Specifically, the user's mobile phone, body fat scale, watch, bracelet, and server can be combined into a data collection system.

Among them, the body fat scale provides the sensor data of the physiological characteristics measured by the user with the body fat scale, such as body weight, body fat rate, heart rate, skeletal muscle mass, basal metabolic rate, moisture rate, bone salt mass, protein, fat-free body mass, Predict body age, predict body score, etc. The body fat scale can transmit the user's physiological data to the mobile phone.

The mobile phone may have a user profile registration module for the user to manually input data, such as age, gender, height, and menstrual cycle. The mobile phone can also learn and process the heart rate, blood pressure and other data collected by the smart wearable device, and obtain the user's prediction of heart health and sleep quality.

Smart wearable devices such as wristbands and watches can continuously collect sensory data of users' real-time physiological characteristics. Physiological data can include the physiological characteristics of the user's daily life, exercise, and official competition, such as heart rate, blood pressure, pulse, body temperature, blood oxygen saturation, blood sugar concentration, inertial measurement unit (Inertial Measurement Unit, IMU) signal and photoelectric volume Pulse wave method (Photoplethysmography, PPG) signal, etc. The body temperature sensor on the wristband and watch can monitor the body surface temperature of the wrist. Among them, watches and bracelets can use PPG signals to further extract sensing data related to physiological characteristics of the cardiovascular system such as the user's breathing rate, pulse rate, blood flow rate, blood temperature, arterial pressure, hardness index, and so on. Watches and watches can use IMU signals and PPG signals to further identify the user's shivering degree and determine whether the user's gait is stumbling. The wristband and watch also include sensors that can collect sensory data such as environmental characteristics, such as ambient temperature sensors, humidity sensors, wind speed sensors, altitude sensors, air pressure sensors, and so on.

In some scenarios, the user's mobile phone can initiate a data collection task, and the user's wristband, watch and other wearable devices can collect the sensing data of the user's physiological characteristics and environmental characteristics in real time to determine whether the user is suffering from hypothermia state. If it is judged that the user is in a state of hypothermia, various interactive methods can be used to remind the user and ask whether an alarm is required. Optional interactive methods such as vibration from the bracelet, ringtones, and voice prompts are available. When the user does not respond, the user will automatically Alarm and send GPS positioning information. In other scenarios, the server of the competition party can initiate a data collection task, and can send the data collection task to the mobile phone or wearable device such as a watch or bracelet of each user participating in the competition. This data collection task can be used to obtain sensing data of the contestants’ physiological characteristics and environmental characteristics that can be collected in real time on wearable devices such as bracelets and watches of each user, that is, sensing data based on the user’s physiological characteristics And the sensor data of environmental characteristics can judge whether the contestant is in a state of hypothermia, and if so, can contact the contestant in time and provide rescue in time.

In other scenarios, the data-demanding device can also initiate a long-term data collection task to obtain sensory data collected by one or more data collection devices for a long time.

Exemplarily, in a possible scenario, multiple devices of the user may form a sensing federation, and receive a data collection task instructing the user's devices to collect sensory data related to the user's physiological characteristics for a long time. Based on this data collection task, the user's watch or bracelet can collect sensory data such as heart rate, HRV, blood oxygen saturation, and sleep quality in different scenarios such as sports activities, meals, sedentary, and rest. The user's earphones or smart glasses collect sensory data such as the user's heart rate, HRV, and blood oxygen saturation in leisure scenarios (such as listening to music). The user's mobile phone can collect sensory data such as heart rate, HRV, and blood oxygen saturation of the user in leisure scenes (such as watching videos, listening to music, playing games, and meditating) and work scenes. The mobile phone can also obtain sensory data such as the user's age, gender, and BMI. The user's car machine can collect sensory data such as the user's acceleration operation, brake operation, and steering operation on the vehicle in scenarios such as traffic jams and traffic accidents. The user's smart headband can collect sensory data such as the user's heart rate, HRV, brain wave and other physiological signals in the meditation scene. The user's computer and large screen can collect sensory data such as user status (working status, leisure status, indoor sports). The federal management center in the sensing federation (for example, a mobile phone, or a computer, or a smart health monitoring device, etc.) The sensory data collected by devices such as large screens within a certain period of time (for example, within a day, within a week, within a month, etc., not limited here) determines the user's stress state. In this way, sensory data related to the user's physiological characteristics can be collected through various devices, and the user's stress state can be evaluated more accurately and objectively. In addition, it can also provide timely health reminders to identified high-stress users to ensure that users can adjust their living conditions in a timely manner and avoid the occurrence of diseases. In addition, the long-term tracking of the user's psychological stress state can also be realized through multiple devices of the user user, so as to provide quantitative reference for doctors to assess the stress state of patients in daily life, and to grasp the patient's condition changes more accurately.

The hardware structure of the electronic devices in the data collection system provided by the embodiment of the present application (for example, the data demand device, the data collection device, and the electronic device where the federation management center is located) can be shown as the electronic device 100 in FIG. 18 . FIG. 18 is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application.

Hereinafter, the embodiment will be specifically described by taking the electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2. Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194 and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

Wherein, the controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The SIM interface can be used to communicate with the SIM card interface 195 to realize the function of transmitting data to the SIM card or reading data in the SIM card.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger.

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .

The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves and radiate them through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD). The display panel can also use organic light-emitting diodes (organic light-emitting diodes, OLEDs), active-matrix organic light-emitting diodes or active-matrix organic light-emitting diodes (active-matrix organic light emitting diodes, AMOLEDs), flexible light-emitting diodes ( flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application required by a function (such as a face recognition function, a fingerprint recognition function, a mobile payment function, etc.) and the like. The data storage area can store data created during use of the electronic device 100 (such as face information template data, fingerprint information template, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also known as "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication.

It can be understood that, in the embodiment of the present application, the electronic device where the federation management center 30 is located may include more or less components shown in the above-mentioned electronic device 100 . The data demanding device 20 may include more or less of the components shown in the electronic device 100 described above. The data collection device may also include fewer or more components than those shown in the electronic device 100 described above. The embodiment of the present application does not limit the hardware structure of the data demand device 20 , the electronic device where the federation management center 30 is located, and the data collection device 40 , and the specific components included therein.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions described in each embodiment are modified, or some of the technical features are replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the various embodiments of the application.

As used in the above embodiments, depending on the context, the term "when" may be interpreted to mean "if" or "after" or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrase "in determining" or "if detected (a stated condition or event)" may be interpreted to mean "if determining..." or "in response to determining..." or "on detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.

Claims

A data collection method, characterized in that the data collection method is applied to a data collection system, the data collection system includes a plurality of devices and a federation management center, the plurality of devices includes a first device, and the method includes :

The federation management center receives the data collection task sent by the first device;

The federation management center determines one or more devices that provide the sensing data required by the data collection task according to the data collection task and the sensing capability information of the plurality of devices;

The federation management center instructs the one or more devices to provide required sensory data.
The method according to claim 1, characterized in that, the federation management center is deployed in one device, or deployed in multiple devices distributed in the data collection system, and the devices include electronic devices and servers.
The method according to any one of claims 1 or 2, characterized in that, the federation management center determines the information needed to provide the data collection task according to the data collection task and the sensing capability information of the plurality of devices. Sensing data from one or more devices, including:

The federation management center determines one or more devices that provide the sensing data required by the data collection task according to the data collection task and the sensing capability information of the plurality of devices;

The federation management center determines to provide the data according to the sensing capability information of the multiple devices, the association relationship between the sensing data of the multiple devices and/or the device status of the multiple devices One or more devices that collect sensory data required for a task perform the data collection task.
The method according to claim 3, wherein the device status includes the power status of the multiple devices, the wearing status, the distance from the first device, and the sensor data provided by the multiple devices. One or more of precision.
The method according to claim 4, characterized in that, according to the data collection task and the sensor data of the plurality of devices, the federation management center determines one of the sensor data required for the data collection task. or multiple devices, including:

The federation management center parses out the data collection task, and determines that the sensory data required for the data collection task includes first sensory data and second sensory data;

The federation management center determines, from the multiple devices, a second device that provides the first sensor data and provides the second sensor data according to the first sensor data and the second sensor data. third device for data;

The federation management center sends a first collection subtask to the second device, and sends a second collection subtask to the third device; the first collection subtask is used to instruct the second device to provide the first collection subtask. Sensing data, the second collection subtask is used to instruct the third device to provide the second sensing data.
The method according to claim 5, wherein the federal management center sends the first collection subtask to the second device, and after sending the second collection subtask to the third device, the method further includes :

The second device accepts or rejects the first acquisition subtask;

The third device accepts or rejects the second acquisition subtask.
The method according to claim 6, wherein the second device accepting or rejecting the first acquisition subtask comprises:

The second device determines to accept or reject the first collection subtask based on remaining resources in the second device;

The second device determines to accept or reject the first acquisition subtask based on remaining resources in the second device, including:

When the second device determines that the remaining resources in the second device are greater than the resources to be used required by the first collection subtask, the second device sends a first message to the federation management center, The first message is used to instruct the second device to accept the first collection subtask; the remaining resources in the second device include remaining computing resources and storage resources in the second device;

When the second device determines that the remaining resources in the second device are less than the resources to be used required by the first acquisition subtask, the second device sends a second message to the federation management center, The second message is used to instruct the second device to refuse to accept the first acquisition subtask.
The method according to any one of claims 1-7, wherein the federation management center instructs the one or more devices to provide the required sensing data, including:

The one or more devices establish a data transmission channel with the first device;

The one or more devices send the required sensing data to the first device through the data transmission channel.
The method according to any one of claims 1-7, wherein the federation management center instructs the one or more devices to provide the required sensing data, including:

The federation management center sends the required sensing data sent by one or more devices to the first device.
The method according to claim 9, wherein the sensing data required for the data collection task includes first semantic information; the federation management center sends the required sensing data to one or more devices, sent to the first device, including:

The federation management center converts the original signal collected by the sensor or the second semantic information sent by one or more devices into first semantic information, and sends the first semantic information to the first device.
The method according to claim 7, wherein when the second device determines that the remaining resources in the second device are less than the resources to be used required by the first collection subtask, the The second device sends a second message to the federation management center, the second message is used to instruct the second device to refuse to accept the first acquisition subtask, including:

The federation management center determines a fourth device that provides the first sensing data according to the first collection subtask and the sensing capability information of the plurality of devices.
The method according to claim 7, wherein when the second device determines that the remaining resources in the second device are greater than the resources to be used required by the first collection subtask, the The second device sends a first message to the federation management center, where the first message is used to instruct the second device to accept the first collection subtask, including:

The second device collects the first sensory data based on the first collection subtask;

The federal management center determines that the second device is in an abnormal state, and the federal management center determines that the fourth device that provides the first sensor data; the abnormal state includes that the battery power of the second device is lower than the power One or more of a threshold value, disconnection of the second device from the federal management center, disconnection of the second device from a power supply, and disconnection of the second device from a power source.
The method according to any one of claims 1-12, wherein the data collection task carries an arbitration flag, and the arbitration flag is used to indicate that the fifth device among the one or more devices refuses to accept the The number of data collection tasks exceeds a preset threshold; the federal management center instructs the one or more devices to provide the required sensing data, including:

The federation management center instructs the fifth device to provide required sensing data based on the remaining resources of the fifth device.
The method according to claim 13, wherein the federation management center instructs the fifth device to provide required sensing data based on the remaining resources of the fifth device, including:

The federation management center sends a third message to the fifth device, and the required third message is used to instruct the fifth device to send the remaining resources in the fifth device and the fifth device to the federation management center. 5. The estimated resources to be used for performing the data collection task evaluated by the equipment;

The federation management center receives the remaining resources in the fifth device sent by the fifth device, and the resources to be used estimated by the fifth device required to perform the data collection task;

In the case that the remaining resources are greater than the required resources to be used, the federation management center instructs the fifth device to provide the data required by the data collection task.
The method according to claim 13, wherein the federation management center instructs the fifth device to provide required sensing data based on the remaining resources of the fifth device, including:

The federation management center sends a fourth message to the fifth device, and the required fourth message is used to instruct the fifth device to send the remaining resources in the fifth device to the federation management center;

The federation management center receives the remaining resources in the fifth device sent by the fifth device;

The federation management center evaluates the resources to be used required by the fifth device to perform the data collection task based on the data collection task;

In the case that the remaining resources are greater than the required resources to be used, the federation management center instructs the fifth device to provide the data required by the data collection task.
The method according to any one of claims 1-15, wherein before the federation management center receives the data collection task sent by the first device, the method further includes:

The federal management center receives the sensing capability information of the plurality of devices;

The federal management center derives new sensing capability information based on the sensing capability information of the plurality of devices and the downloaded first knowledge graph.
The method according to claim 16, wherein the federation management center receives the sensing capability information of the plurality of devices, comprising:

The federation management center receives a message that the sixth device among the plurality of devices joins the data collection system and the device model of the sixth device;

The federation management center obtains the sensor capability information of the sixth device from the first server based on the device model of the sixth device; the first server stores different device models and corresponding Sensing capability information.
The method according to claim 17, wherein after the federation management center acquires the capability information of the sixth device from the first server based on the device model of the sixth device, the method further comprises:

The federal management center derives new sensing capability information based on the sensing capability information of the sixth device and a second knowledge graph; the second knowledge graph includes a first knowledge graph, and the second knowledge graph The sensory capability information contained in is more than the sensory capability information contained in the first knowledge map;

Based on the new sensing capability information, update the sensing capability information stored in the federation management center.
The method according to claim 18, characterized in that, after the federation management center derives new sensing capability information based on the sensing capability information of the sixth device and the second knowledge map, the method Also includes:

The federal management center determines that the sixth device exits the data collection system;

The federation management center deletes the stored sensing capability information of the sixth device, and deletes new sensing capability information derived based on the sensing capability information of the sixth device.
The method according to claim 19, wherein the federal management center determines that the sixth device exits the data collection system, comprising:

If the federation management center does not receive the fifth message sent by the sixth device within a preset time period, the federation management center determines that the sixth device exits the data collection system; the fifth The message is used to indicate that the sixth device is in the data collection system.
The method according to claim 19, wherein the federal management center determines that the sixth device exits the data collection system, comprising:

The federation management center receives a sixth message sent by the sixth device, and the sixth message is used to instruct the sixth device to quit the data collection system.
The method according to claims 1-21, wherein the one or more devices further include a seventh device, and the method further includes:

The federation management center sends first indication information to the seventh device, where the first indication information is used to indicate that when the third sensor data collected by the seventh device is determined to be in the first state, the seventh The device sends the third sensor data to the federation management center;

The federation management center receives the third sensing data sent by the seventh device;

The federation management center sends the third sensing data to one or more devices subscribed to the third sensing data.
The method according to any one of claims 1-22, wherein after the federation management center receives the data collection task sent by the first device, the method further comprises: the federation management center based on the Data collection tasks, and sensory data stored by the federation management center;

The federation management center determines that the federation management center stores the sensory data required for the data collection task;

The federal management center sends the required sensing data to the first device.
The method according to any one of claims 1-23, wherein the sensing capability information includes sensing data in the device and the accuracy of the sensing data in the device;

The first knowledge graph is generated by the federation management center based on the sensor capability information of the multiple devices; the second knowledge graph is stored in the server, and the first knowledge graph includes the passed information of the multiple devices. Sensing data, and an association relationship between the sensing data of the multiple devices.
A data acquisition method, the data acquisition method is applied to a data acquisition system, the data acquisition system includes a plurality of devices and a federation management center, the plurality of devices includes a first device; the method includes:

The first device sends a first data collection task to the federal management center, and the first data collection task indicates that when the sensor data required by the first data collection task meets a first condition, acquire the first data collection task. - sensory data required for data acquisition tasks;

The federation management center determines one or more devices that provide the sensing data required by the first data collection task based on the data collection task and the sensing capability information of the plurality of devices;

The federation management center sends first indication information to one or more devices, where the first indication information is used to indicate that the required sensor data collected by the one or more devices meets the first condition case, said one or more devices send said required sensor data to said federation management center.
The method according to claim 25, further comprising:

The second device sends a second data collection task to the federation management center, the second collection task includes a collection cycle parameter, and the collection cycle parameter indicates a cycle in which the second device needs the fourth sensing data;

The federation management center determines one or more devices to provide the fourth sensing data based on the second data collection task and the sensing capability information of the plurality of devices;

The federation management center instructs the one or more devices to collect the fourth sensory data based on the collection period parameter.
The method according to any one of claims 25 or 26, characterized in that the federation management center is deployed in one device, or distributed in multiple devices in the data collection system, and the devices include electronic devices ,server.
The method according to any one of claims 25-27, wherein the sensing capability information includes the sensing data in the device and the accuracy of the sensing data in the device; the first condition includes the The required sensor data exceeds a first threshold, and the difference between the required sensor data collected at the second moment and the required sensor data collected at the first moment is greater than a second threshold; the second The second moment is later than the first moment.
A data acquisition system, characterized in that the data acquisition system includes a first electronic device, a second electronic device, and a third electronic device, the first electronic device establishes a communication connection with the second electronic device, and the The second electronic device establishes a communication connection with the third electronic device; the first electronic device, the second electronic device, and the third electronic device are used to execute the method according to claims 1-24 or 25-28. any one of the methods described.
An electronic device, characterized in that it includes one or more processors, one or more memories, and a transceiver; wherein the transceiver, the one or more memories are coupled to the one or more processors , the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the functions described in claim 1- 24 or the method described in any one of 25-28.
A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer is made to perform the operation described in any one of claims 1-24 or 25-28. Methods.