CN115442392A - Data processing platform and data acquisition method - Google Patents

Abstract

The invention provides a data processing platform and a data acquisition method, and relates to the technical field of Internet of things. In the invention, a plurality of target Internet of things terminal devices are determined from a plurality of Internet of things terminal devices, wherein the number of the target Internet of things terminal devices is less than that of the Internet of things terminal devices; acquiring Internet of things image data sent by a target Internet of things terminal device aiming at each target Internet of things terminal device in a plurality of target Internet of things terminal devices; and performing data deduplication processing on the internet of things image data aiming at the internet of things image data corresponding to each target internet of things terminal device to obtain target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is smaller than or equal to that of the corresponding internet of things image data. Based on the method, the problem that data is easy to lose in the data acquisition process in the prior art can be solved.

Description

Data processing platform and data acquisition method

Technical Field

The invention relates to the technical field of Internet of things, in particular to a data processing platform and a data acquisition method.

Background

In the technical field of the internet of things, a large number of terminal devices of the internet of things are generally involved to collect data and send the collected data to a background for processing. Therefore, due to the existence of a large number of terminal devices of the internet of things, the background can receive more data synchronously, and thus, when receiving resources conflict, the problem of data loss can be caused.

Disclosure of Invention

In view of this, the present invention aims to provide a data processing platform and a data acquisition method, so as to solve the problem that data is easily lost in the data acquisition process in the prior art.

In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:

a data acquisition method is applied to a data processing platform, the data processing platform is in communication connection with a plurality of terminal devices of the Internet of things, and the data acquisition method comprises the following steps:

determining a plurality of target Internet of things terminal devices from the plurality of Internet of things terminal devices, wherein the number of the plurality of target Internet of things terminal devices is less than that of the plurality of Internet of things terminal devices, and the plurality of Internet of things terminal devices are respectively arranged in a plurality of different data acquisition areas;

acquiring Internet of things image data sent by each target Internet of things terminal device in the target Internet of things terminal devices, wherein the Internet of things image data is acquired based on the corresponding target Internet of things terminal device and corresponding data acquisition regions;

and performing data deduplication processing on the internet of things image data aiming at the internet of things image data corresponding to each target internet of things terminal device in the target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is less than or equal to that of the corresponding internet of things image data.

In some preferred embodiments, in the data acquisition method, the step of determining a plurality of target internet of things terminal devices from among the plurality of internet of things terminal devices includes:

for each piece of internet-of-things terminal equipment in the plurality of pieces of internet-of-things terminal equipment, determining whether the piece of internet-of-things terminal equipment belongs to first piece of internet-of-things terminal equipment, wherein the first piece of internet-of-things terminal equipment belongs to piece of internet-of-things terminal equipment which has been used as a historical target within a preset time period, and sending historical internet-of-things image data to the data processing platform;

and aiming at each of the plurality of Internet of things terminal devices, if the Internet of things terminal device does not belong to the first Internet of things terminal device, determining whether the Internet of things terminal device is used as a target Internet of things terminal device or not so as to determine the plurality of target Internet of things terminal devices.

In some preferred embodiments, in the data acquisition method, the step of determining, for each of the plurality of internet of things terminal devices, whether to use the internet of things terminal device as a target internet of things terminal device if the internet of things terminal device does not belong to the first internet of things terminal device, so as to determine the plurality of target internet of things terminal devices includes:

for each piece of internet-of-things terminal equipment in the plurality of pieces of internet-of-things terminal equipment, if the piece of internet-of-things terminal equipment does not belong to the first piece of internet-of-things terminal equipment, determining the piece of internet-of-things terminal equipment as second piece of internet-of-things terminal equipment, acquiring historical time information of the second piece of internet-of-things terminal equipment which is determined as a historical target piece of internet-of-things terminal equipment for the last time, and determining a first screening coefficient corresponding to the second piece of internet-of-things terminal equipment based on the historical time information corresponding to the second piece of internet-of-things terminal equipment, wherein the first screening coefficient and the historical time information have a negative correlation relationship;

and determining a plurality of target Internet of things terminal equipment in a plurality of second Internet of things terminal equipment based on the first screening coefficient corresponding to each second Internet of things terminal equipment.

In some preferred embodiments, in the data acquisition method, the step of determining, based on the first screening coefficient corresponding to each of the second networked terminal devices, a plurality of target internet-of-things terminal devices among the second networked terminal devices includes:

for each two second networked terminal devices in the plurality of second networked terminal devices, determining a device correlation coefficient between the two second networked terminal devices;

for each second networking terminal device in the second networking terminal devices, calculating an average value of device correlation coefficients between the second networking terminal device and each other second networking terminal device to obtain a correlation coefficient average value corresponding to the second networking terminal device;

for each second networking terminal device in the second networking terminal devices, determining a second screening coefficient corresponding to the second networking terminal device based on a correlation coefficient mean value corresponding to the second networking terminal device, performing weighted summation calculation processing on the second screening coefficient corresponding to the second networking terminal device and a first screening coefficient corresponding to the second networking terminal device, and determining a weighted screening coefficient corresponding to the second networking terminal device, wherein the second screening coefficient and the correlation coefficient mean value have positive correlation;

and determining a plurality of second networked terminal devices with the maximum corresponding weighted screening coefficients in the plurality of second networked terminal devices as target internet of things terminal devices based on the pre-configured target quantity value so as to obtain a plurality of target internet of things terminal devices.

In some preferred embodiments, in the data collecting method, the step of determining, for each two second networked terminal devices of the plurality of second networked terminal devices, a device correlation coefficient between the two second networked terminal devices includes:

for each second networking terminal device in the plurality of second networking terminal devices, determining the position of a data acquisition area corresponding to the second networking terminal device, obtaining area position information corresponding to the second networking terminal device, and obtaining historical internet of things image data sent by the second networking terminal device which is determined as a historical target internet of things terminal device last time;

for every two second networked terminal devices in the plurality of second networked terminal devices, determining area position distance information between the two second networked terminal devices based on area position information corresponding to the two second networked terminal devices, and determining a first correlation coefficient corresponding to the two second networked terminal devices based on the area position distance information, wherein a negative correlation relationship is formed between the first correlation coefficient and the area position distance information;

for every two second networked terminal devices in the plurality of second networked terminal devices, calculating the similarity between the historical internet-of-things image data corresponding to the two second networked terminal devices to obtain the data similarity corresponding to the two second networked terminal devices;

for every two second networked terminal devices in the plurality of second networked terminal devices, determining second correlation coefficients corresponding to the two second networked terminal devices based on the data similarity corresponding to the two second networked terminal devices, and performing fusion processing on the second correlation coefficients corresponding to the two second networked terminal devices and the first correlation coefficients corresponding to the two second networked terminal devices to obtain device correlation coefficients corresponding to the two second networked terminal devices, wherein the second correlation coefficients and the data similarity have positive correlation.

In some preferred embodiments, in the data acquisition method, the step of acquiring, for each of the target internet of things terminal devices, internet of things image data sent by the target internet of things terminal device includes:

generating data acquisition notification information, and respectively sending the data acquisition notification information to each of the target internet of things terminal devices, wherein each of the target internet of things terminal devices is used for sending internet of things image data acquired and stored in a corresponding data acquisition area to the data processing platform after receiving the data acquisition notification information, and clearing the internet of things image data after sending the internet of things image data to the data processing platform;

and acquiring, for each target internet of things terminal device of the plurality of target internet of things terminal devices, internet of things image data acquired by the target internet of things terminal device for acquiring the corresponding data acquisition region based on the data acquisition notification information.

In some preferred embodiments, in the data acquisition method, the step of performing data deduplication processing on the internet of things image data for each of the target internet of things terminal devices to obtain target internet of things image data corresponding to the internet of things image data includes:

respectively calculating the similarity between every two adjacent frames of Internet of things monitoring images included in the Internet of things image data aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the plurality of target Internet of things terminal equipment to obtain the image similarity between every two adjacent frames of Internet of things images;

and aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the target Internet of things terminal equipment, carrying out data deduplication processing on the Internet of things monitoring images included by the Internet of things image data based on the image similarity between every two adjacent frames of Internet of things monitoring images included by the Internet of things image data to obtain the target Internet of things image data corresponding to the Internet of things image data.

The embodiment of the invention also provides a data processing platform, wherein the data processing platform is in communication connection with a plurality of terminal devices of the internet of things and is used for executing a preconfigured data acquisition method, and the data acquisition method comprises the following steps:

determining a plurality of target Internet of things terminal devices from the plurality of Internet of things terminal devices, wherein the number of the target Internet of things terminal devices is less than that of the plurality of Internet of things terminal devices, and the plurality of Internet of things terminal devices are respectively arranged in a plurality of different data acquisition areas;

acquiring Internet of things image data sent by each target Internet of things terminal device in the target Internet of things terminal devices, wherein the Internet of things image data is acquired based on the corresponding target Internet of things terminal device and corresponding data acquisition regions;

and performing data deduplication processing on the internet of things image data aiming at the internet of things image data corresponding to each target internet of things terminal device in the target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is less than or equal to that of the corresponding internet of things image data.

In some preferred embodiments, in the data processing platform, the step of determining a plurality of target internet of things terminal devices from among the plurality of internet of things terminal devices includes:

determining whether each piece of terminal equipment of the internet of things belongs to first terminal equipment of the internet of things or not, wherein the first terminal equipment of the internet of things belongs to terminal equipment of the internet of things which is already used as a historical target in a preset time span, and sending historical image data of the internet of things to the data processing platform;

and aiming at each of the plurality of terminal equipment of the Internet of things, if the terminal equipment of the Internet of things does not belong to the first terminal equipment of the Internet of things, determining whether the terminal equipment of the Internet of things is used as target terminal equipment of the Internet of things so as to determine the plurality of target terminal equipment of the Internet of things.

In some preferred embodiments, in the data processing platform, the step of performing data deduplication processing on the internet of things image data for each of the target internet of things terminal devices to obtain target internet of things image data corresponding to the internet of things image data includes:

respectively calculating the similarity between every two adjacent frames of Internet of things monitoring images included in the Internet of things image data aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the plurality of target Internet of things terminal equipment to obtain the image similarity between every two adjacent frames of Internet of things images;

and aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the target Internet of things terminal equipment, carrying out data deduplication processing on the Internet of things monitoring images included by the Internet of things image data based on the image similarity between every two adjacent frames of Internet of things monitoring images included by the Internet of things image data to obtain the target Internet of things image data corresponding to the Internet of things image data.

According to the data processing platform and the data acquisition method provided by the embodiment of the invention, a plurality of target internet of things terminal devices can be determined in a plurality of internet of things terminal devices, and then, internet of things image data sent by the target internet of things terminal devices can be obtained for each target internet of things terminal device in the plurality of target internet of things terminal devices, so that the internet of things image data can be subjected to data deduplication processing aiming at the internet of things image data corresponding to each target internet of things terminal device, and the target internet of things image data corresponding to the internet of things image data can be obtained. Based on this, because the quantity of target thing networking terminal equipment is less than the quantity of thing networking terminal equipment for can reduce the quantity of the data of synchronous receipt, reduce the risk of data resource conflict to a certain extent, ensure the effective receipt of data, thereby improve the problem that data loss appears easily among the prior art data acquisition's the in-process.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

Fig. 1 is a block diagram of a data processing platform according to an embodiment of the present invention.

Fig. 2 is a schematic flowchart of steps included in a data acquisition method according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of modules included in a data acquisition system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, an embodiment of the present invention provides a data processing platform. Wherein the data processing platform may include a memory and a processor.

In detail, the memory and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, they may be electrically connected to each other via one or more communication buses or signal lines. The memory can have stored therein at least one software function (computer program) which can be present in the form of software or firmware. The processor may be configured to execute the executable computer program stored in the memory, thereby implementing the data acquisition method provided by the embodiments of the present invention (as described below).

By way of example, in some possible embodiments, the Memory may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like.

For example, in some possible embodiments, the Processor may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), a System on Chip (SoC), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

Also, the structure shown in fig. 1 is only an illustration, and the data processing platform may further include more or less components than those shown in fig. 1, or have a different configuration than that shown in fig. 1, for example, may include a communication unit for information interaction with other devices.

For example, in some possible embodiments, the data processing platform may be a server with data processing capabilities.

With reference to fig. 2, an embodiment of the present invention further provides a data acquisition method, which can be applied to the data processing platform. The method steps defined by the flow related to the data acquisition method can be realized by the data processing platform. And the data processing platform is in communication connection with a plurality of terminal devices of the internet of things. The specific process shown in FIG. 2 will be described in detail below.

And step S110, determining a plurality of target terminal equipment of the Internet of things in the plurality of terminal equipment of the Internet of things.

In the embodiment of the invention, the data processing platform can determine a plurality of target terminal devices of the internet of things from the plurality of terminal devices of the internet of things. The number of the target internet of things terminal devices is smaller than that of the internet of things terminal devices, and the internet of things terminal devices are respectively arranged in a plurality of different data acquisition areas.

Step S120, aiming at each target Internet of things terminal device in the target Internet of things terminal devices, acquiring Internet of things image data sent by the target Internet of things terminal device.

In the embodiment of the present invention, the data processing platform may acquire, for each of the plurality of target internet of things terminal devices, internet of things image data sent by the target internet of things terminal device. And the Internet of things image data is acquired from the corresponding data acquisition area based on the corresponding target Internet of things terminal equipment.

Step S130, aiming at the Internet of things image data corresponding to each target Internet of things terminal device in the target Internet of things terminal devices, data deduplication processing is carried out on the Internet of things image data to obtain target Internet of things image data corresponding to the Internet of things image data.

In the embodiment of the present invention, the data processing platform may perform data deduplication processing on the internet of things image data for the internet of things image data corresponding to each of the plurality of target internet of things terminal devices, so as to obtain the target internet of things image data corresponding to the internet of things image data. And the data volume of the target Internet of things image data is less than or equal to that of the corresponding Internet of things image data.

Based on the steps included in the data acquisition method, a plurality of target internet of things terminal devices can be determined in a plurality of internet of things terminal devices, and then, internet of things image data sent by the target internet of things terminal devices can be acquired for each target internet of things terminal device in the plurality of target internet of things terminal devices, so that the internet of things image data can be subjected to data deduplication processing for the internet of things image data corresponding to each target internet of things terminal device, and the target internet of things image data corresponding to the internet of things image data can be obtained. Based on this, because the quantity of the target terminal devices of the internet of things is smaller than that of the terminal devices of the internet of things, the quantity of the data which are synchronously received can be reduced, the risk of data resource conflict is reduced to a certain extent, the effective receiving of the data is guaranteed, and the problem that the data are easily lost in the data acquisition process in the prior art is solved.

Illustratively, in some possible embodiments, step S110 may include the following:

firstly, determining whether each piece of internet-of-things terminal equipment belongs to first internet-of-things terminal equipment or not for each piece of internet-of-things terminal equipment, wherein the first internet-of-things terminal equipment belongs to internet-of-things terminal equipment which has been used as a historical target within a preset time span, and sending historical internet-of-things image data to the data processing platform;

secondly, for each of the plurality of internet of things terminal devices, if the internet of things terminal device does not belong to the first internet of things terminal device, whether the internet of things terminal device is used as a target internet of things terminal device is determined so as to determine the plurality of target internet of things terminal devices.

For example, in some possible embodiments, the step of determining, for each of the plurality of terminal devices of the internet of things, whether to use the terminal device of the internet of things as a target terminal device of the internet of things if the terminal device of the internet of things does not belong to the first terminal device of the internet of things, so as to determine the plurality of target terminal devices of the internet of things may include the following steps:

firstly, for each of the plurality of pieces of internet-of-things terminal equipment, if the piece of internet-of-things terminal equipment does not belong to first piece of internet-of-things terminal equipment, determining the piece of internet-of-things terminal equipment as second piece of internet-of-things terminal equipment, acquiring historical time information of the second piece of internet-of-things terminal equipment which is determined as a historical target internet-of-things terminal equipment at the latest time, and determining a first screening coefficient corresponding to the second piece of internet-of-things terminal equipment based on the historical time information corresponding to the second piece of internet-of-things terminal equipment, wherein the first screening coefficient and the historical time information have a negative correlation relationship (namely the earlier the historical time information is, the larger the corresponding first screening coefficient is);

secondly, determining a plurality of target internet of things terminal devices in the plurality of second internet of things terminal devices based on the first screening coefficient corresponding to each second internet of things terminal device.

For example, in some possible embodiments, the step of determining, based on the first screening coefficient corresponding to each of the second networked terminal devices, a plurality of target internet-of-things terminal devices from among the second networked terminal devices may include the following steps:

firstly, aiming at every two second networking terminal devices in the plurality of second networking terminal devices, determining a device correlation coefficient between the two second networking terminal devices;

secondly, for each second networking terminal device in the second networking terminal devices, calculating an average value of device correlation coefficients between the second networking terminal device and each of other second networking terminal devices (except the second networking terminal device in the second networking terminal devices), and obtaining a correlation coefficient average value corresponding to the second networking terminal device;

then, for each second networking terminal device in the second networking terminal devices, determining a second screening coefficient corresponding to the second networking terminal device based on a correlation coefficient mean value corresponding to the second networking terminal device, performing weighted summation calculation processing on the second screening coefficient corresponding to the second networking terminal device and a first screening coefficient corresponding to the second networking terminal device, and determining a weighted screening coefficient corresponding to the second networking terminal device, wherein a positive correlation relationship exists between the second screening coefficient and the correlation coefficient mean value;

and finally, based on a preset target quantity value (the target quantity value can have positive correlation with the data receiving capacity of the data processing platform, and the data receiving capacity can have positive correlation with factors such as a data receiving port of the data processing platform), determining a plurality of second networking terminal devices with the maximum corresponding weighted screening coefficients in the plurality of second networking terminal devices as target internet-of-things terminal devices to obtain the plurality of target internet-of-things terminal devices.

For example, in some possible embodiments, the step of determining, for each two second networked terminal devices of the plurality of second networked terminal devices, a device correlation coefficient between the two second networked terminal devices may include the following steps:

firstly, aiming at each second networking terminal device in the plurality of second networking terminal devices, determining the position of a data acquisition area corresponding to the second networking terminal device, obtaining area position information corresponding to the second networking terminal device, and obtaining historical internet of things image data sent by the second networking terminal device which is determined as a historical target internet of things terminal device at the latest time;

secondly, for every two second networked terminal devices in the second networked terminal devices, determining area position distance information between the two second networked terminal devices based on area position information corresponding to the two second networked terminal devices, and determining a first correlation coefficient corresponding to the two second networked terminal devices based on the area position distance information, wherein the first correlation coefficient and the area position distance information have a negative correlation relationship;

then, for every two second networked terminal devices in the plurality of second networked terminal devices, calculating the similarity between the historical internet-of-things image data corresponding to the two second networked terminal devices to obtain the data similarity corresponding to the two second networked terminal devices;

then, for each two second networked terminal devices in the plurality of second networked terminal devices, based on the data similarity corresponding to the two second networked terminal devices, determining second correlation coefficients corresponding to the two second networked terminal devices, and performing fusion processing (for example, performing weighted summation calculation processing, where the corresponding weighting coefficients may be configured according to an actual application scenario) on the second correlation coefficients corresponding to the two second networked terminal devices and the first correlation coefficients corresponding to the two second networked terminal devices to obtain device correlation coefficients corresponding to the two second networked terminal devices, where the second correlation coefficients and the data similarity have a positive correlation relationship.

For example, in some possible embodiments, the step of calculating, for each two second networked terminal devices in the plurality of second networked terminal devices, a similarity between historical internet of things image data corresponding to the two second networked terminal devices to obtain a data similarity corresponding to the two second networked terminal devices may include the following steps:

firstly, respectively determining historical internet-of-things image data corresponding to the two second networked terminal devices as first historical internet-of-things image data and second historical internet-of-things image data, performing target image recognition processing on the first historical internet-of-things image data to obtain a first historical image set corresponding to the first historical internet-of-things image data, and performing target image recognition processing on the second historical internet-of-things image data to obtain a second historical image set corresponding to the second historical internet-of-things image data, wherein the first historical image set is formed based on historical internet-of-things monitoring images with target objects (such as target persons, animals or devices and the like) in the first historical internet-of-things image data, and the second historical image set is formed based on historical internet-of-things monitoring images with target objects in the second historical internet-of-things image data;

secondly, calculating the image similarity between every two frames of historical internet-of-things monitoring images in the first historical image set, screening out one frame of historical internet-of-things monitoring image in the two frames of historical internet-of-things monitoring images when the image similarity corresponding to the two frames of historical internet-of-things monitoring images is larger than or equal to a preset image similarity threshold value, taking the historical internet-of-things monitoring image as a similar monitoring image of the other frame of historical internet-of-things monitoring image which is not screened out, and constructing and obtaining a target first historical image set corresponding to the first historical image set based on each frame of historical internet-of-things monitoring image which is not screened out in the first historical image set;

then, for every two frames of historical internet-of-things monitoring images in the second historical image set, calculating the image similarity between the two frames of historical internet-of-things monitoring images, when the image similarity corresponding to the two frames of historical internet-of-things monitoring images is greater than or equal to the image similarity threshold, screening out one frame of historical internet-of-things monitoring image in the two frames of historical internet-of-things monitoring images, taking the historical internet-of-things monitoring image as a similar monitoring image of the other frame of historical internet-of-things monitoring image which is not screened out, and constructing and obtaining a target second historical image set corresponding to the second historical image set based on each frame of historical internet-of-things monitoring image which is not screened out in the second historical image set;

fourthly, counting the number of other historical internet-of-things monitoring images of which the image similarity in the first historical image set is greater than or equal to the image similarity threshold value aiming at each frame of historical internet-of-things monitoring image in the target first historical image set to obtain the counted number of first similar images corresponding to the historical internet-of-things monitoring images;

fifthly, respectively counting the number of other historical internet-of-things monitoring images of which the image similarity in the first historical image set is greater than or equal to the image similarity threshold value and the image similarity of each similar monitoring image corresponding to the historical internet-of-things monitoring image in the first historical image set aiming at each historical internet-of-things monitoring image in the target first historical image set to obtain the statistical number of the similar monitoring images corresponding to each similar monitoring image corresponding to the historical internet-of-things monitoring image, and calculating the average value of the statistical number of the similar monitoring images corresponding to each similar monitoring image corresponding to the historical internet-of-things monitoring image to obtain the statistical number of the second similar images corresponding to the historical internet-of-things monitoring image;

sixthly, performing weighted summation calculation on the first similar image statistical quantity corresponding to the historical internet of things monitoring image and the second similar image statistical quantity corresponding to the historical internet of things monitoring image aiming at each frame of historical internet of things monitoring image in the target first historical image set to obtain the weighted statistical quantity corresponding to the historical internet of things monitoring image, wherein the weighting coefficient corresponding to the first similar image statistical quantity is greater than the weighting coefficient corresponding to the second similar image statistical quantity;

seventhly, determining at least one frame of historical internet-of-things monitoring image in the target first historical image set based on the weighted statistical quantity corresponding to each frame of historical internet-of-things monitoring image in the target first historical image set, wherein the historical internet-of-things monitoring image is used as at least one frame of first historical internet-of-things monitoring image corresponding to the target first historical image set;

eighthly, determining each frame of historical internet-of-things monitoring image with the image similarity between the historical internet-of-things monitoring image and the first historical internet-of-things monitoring image being greater than or equal to the image similarity threshold value in the second historical image set as a related monitoring image corresponding to the second historical image set, and determining a first image quantity ratio between the quantity of the related monitoring image and the quantity of the at least one frame of first historical internet-of-things monitoring image;

ninth, in the target second historical image set, determining each frame of historical internet-of-things monitoring image with the image similarity between the frame of historical internet-of-things monitoring image and the first historical internet-of-things monitoring image being greater than or equal to the image similarity threshold value as a related monitoring image corresponding to the target second historical image set, and determining a second image quantity ratio between the quantity of the related monitoring image and the quantity of the at least one frame of first historical internet-of-things monitoring image;

and finally, carrying out weighted summation calculation on the first image quantity ratio and the second image quantity ratio, and determining the data similarity between the historical internet of things image data corresponding to the two second networked terminal devices based on the obtained weighted quantity ratio, wherein the data similarity and the weighted quantity ratio have positive correlation, and the weighting coefficient corresponding to the first image quantity ratio is greater than the weighting coefficient corresponding to the second image quantity ratio.

For example, in some possible embodiments, step S120 may include the following:

firstly, generating data acquisition notification information, and respectively sending the data acquisition notification information to each of the target internet of things terminal devices, wherein each of the target internet of things terminal devices is used for sending internet of things image data acquired and stored in a corresponding data acquisition area to the data processing platform after receiving the data acquisition notification information, and clearing the internet of things image data after sending the internet of things image data to the data processing platform (so that the internet of things image data acquired each time is acquired at the minimum);

and secondly, acquiring the Internet of things image data, which is acquired by the target Internet of things terminal equipment based on the data acquisition notification information and sent by the target Internet of things terminal equipment to the corresponding data acquisition area, for each target Internet of things terminal equipment in the plurality of target Internet of things terminal equipment.

Illustratively, in some possible embodiments, step S130 may include the following:

firstly, for the internet of things image data corresponding to each target internet of things terminal device in the target internet of things terminal devices, respectively calculating the similarity between every two adjacent frames of internet of things monitoring images included in the internet of things image data (referring to the existing image similarity calculation mode), and obtaining the image similarity between every two adjacent frames of internet of things images;

secondly, for the internet of things image data corresponding to each target internet of things terminal device of the plurality of target internet of things terminal devices, based on the image similarity between every two adjacent frames of internet of things monitoring images included in the internet of things image data, data deduplication processing is performed on the internet of things monitoring images included in the internet of things image data, so as to obtain target internet of things image data corresponding to the internet of things image data (for example, one frame of the two frames of internet of things monitoring images with the image similarity larger than a threshold value is screened out).

With reference to fig. 3, an embodiment of the present invention further provides a data acquisition system, which can be applied to the data processing platform. Wherein, the data acquisition system can comprise the following modules:

the target equipment determining module is used for determining a plurality of target Internet of things terminal equipment from the plurality of Internet of things terminal equipment, wherein the number of the plurality of target Internet of things terminal equipment is less than that of the plurality of Internet of things terminal equipment, and the plurality of Internet of things terminal equipment are respectively arranged in a plurality of different data acquisition areas;

the image data acquisition module is used for acquiring the internet of things image data sent by each target internet of things terminal device in the target internet of things terminal devices, wherein the internet of things image data is acquired based on the corresponding target internet of things terminal device to the corresponding data acquisition area;

the image data deduplication module is used for performing data deduplication processing on the internet of things image data according to the internet of things image data corresponding to each target internet of things terminal device in the multiple target internet of things terminal devices to obtain target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is smaller than or equal to that of the corresponding internet of things image data.

In summary, according to the data processing platform and the data acquisition method provided by the invention, a plurality of target internet of things terminal devices can be determined in a plurality of internet of things terminal devices, and then, internet of things image data sent by the target internet of things terminal devices can be obtained for each of the plurality of target internet of things terminal devices, so that the internet of things image data corresponding to each target internet of things terminal device can be subjected to data deduplication processing, and the target internet of things image data corresponding to the internet of things image data is obtained. Based on this, because the quantity of the target terminal devices of the internet of things is smaller than that of the terminal devices of the internet of things, the quantity of the data which are synchronously received can be reduced, the risk of data resource conflict is reduced to a certain extent, the effective receiving of the data is guaranteed, and the problem that the data are easily lost in the data acquisition process in the prior art is solved.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A data acquisition method is characterized by being applied to a data processing platform, wherein the data processing platform is in communication connection with a plurality of terminal devices of the Internet of things, and the data acquisition method comprises the following steps:

determining a plurality of target Internet of things terminal devices from the plurality of Internet of things terminal devices, wherein the number of the target Internet of things terminal devices is less than that of the plurality of Internet of things terminal devices, and the plurality of Internet of things terminal devices are respectively arranged in a plurality of different data acquisition areas;

acquiring Internet of things image data sent by each target Internet of things terminal device in the target Internet of things terminal devices, wherein the Internet of things image data is acquired based on the corresponding target Internet of things terminal device and corresponding data acquisition regions;

and performing data deduplication processing on the internet of things image data aiming at the internet of things image data corresponding to each target internet of things terminal device in the target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is less than or equal to that of the corresponding internet of things image data.

2. The data acquisition method of claim 1, wherein the step of determining a plurality of target internet of things terminal devices among the plurality of internet of things terminal devices comprises:

determining whether each piece of terminal equipment of the internet of things belongs to first terminal equipment of the internet of things or not, wherein the first terminal equipment of the internet of things belongs to terminal equipment of the internet of things which is already used as a historical target in a preset time span, and sending historical image data of the internet of things to the data processing platform;

and aiming at each of the plurality of Internet of things terminal devices, if the Internet of things terminal device does not belong to the first Internet of things terminal device, determining whether the Internet of things terminal device is used as a target Internet of things terminal device or not so as to determine the plurality of target Internet of things terminal devices.

3. The data acquisition method as claimed in claim 2, wherein the step of determining, for each of the plurality of terminal devices of the internet of things, whether the terminal device of the internet of things is a target terminal device of the internet of things if the terminal device of the internet of things does not belong to the first terminal device of the internet of things, so as to determine the plurality of target terminal devices of the internet of things, includes:

for each piece of internet-of-things terminal equipment in the plurality of pieces of internet-of-things terminal equipment, if the piece of internet-of-things terminal equipment does not belong to the first piece of internet-of-things terminal equipment, determining the piece of internet-of-things terminal equipment as second piece of internet-of-things terminal equipment, acquiring historical time information of the second piece of internet-of-things terminal equipment which is determined as a historical target piece of internet-of-things terminal equipment for the last time, and determining a first screening coefficient corresponding to the second piece of internet-of-things terminal equipment based on the historical time information corresponding to the second piece of internet-of-things terminal equipment, wherein the first screening coefficient and the historical time information have a negative correlation relationship;

and determining a plurality of target Internet of things terminal equipment in a plurality of second Internet of things terminal equipment based on the first screening coefficient corresponding to each second Internet of things terminal equipment.

4. The data acquisition method according to claim 3, wherein the step of determining a plurality of target internet of things terminal devices among the plurality of second internet of things terminal devices based on the first screening coefficient corresponding to each of the second internet of things terminal devices comprises:

for each two second networked terminal devices in the plurality of second networked terminal devices, determining a device correlation coefficient between the two second networked terminal devices;

for each second networking terminal device in the plurality of second networking terminal devices, calculating an average value of device correlation coefficients between the second networking terminal device and each other second networking terminal device to obtain a correlation coefficient average value corresponding to the second networking terminal device;

for each second networking terminal device in the second networking terminal devices, determining a second screening coefficient corresponding to the second networking terminal device based on a correlation coefficient mean value corresponding to the second networking terminal device, performing weighted summation calculation processing on the second screening coefficient corresponding to the second networking terminal device and a first screening coefficient corresponding to the second networking terminal device, and determining a weighted screening coefficient corresponding to the second networking terminal device, wherein a positive correlation relationship exists between the second screening coefficient and the correlation coefficient mean value;

and determining a plurality of second networking terminal devices with the maximum corresponding weighted screening coefficients in a corresponding number from the plurality of second networking terminal devices as target Internet of things terminal devices based on the preset target quantity value so as to obtain a plurality of target Internet of things terminal devices.

5. The data collection method of claim 4, wherein the step of determining, for each two of the plurality of second networked terminal devices, a device correlation coefficient between the two second networked terminal devices comprises:

for each second networking terminal device in the plurality of second networking terminal devices, determining the position of a data acquisition area corresponding to the second networking terminal device to obtain area position information corresponding to the second networking terminal device, and acquiring historical internet of things image data sent by the second networking terminal device which is determined as a historical target internet of things terminal device last time;

for every two second networked terminal devices in the plurality of second networked terminal devices, determining region position distance information between the two second networked terminal devices based on the region position information corresponding to the two second networked terminal devices, and determining a first correlation coefficient corresponding to the two second networked terminal devices based on the region position distance information, wherein the first correlation coefficient and the region position distance information have a negative correlation relationship;

for every two second networked terminal devices in the plurality of second networked terminal devices, calculating the similarity between the historical internet-of-things image data corresponding to the two second networked terminal devices to obtain the data similarity corresponding to the two second networked terminal devices;

for every two second networked terminal devices in the plurality of second networked terminal devices, determining second correlation coefficients corresponding to the two second networked terminal devices based on the data similarity corresponding to the two second networked terminal devices, and performing fusion processing on the second correlation coefficients corresponding to the two second networked terminal devices and the first correlation coefficients corresponding to the two second networked terminal devices to obtain device correlation coefficients corresponding to the two second networked terminal devices, wherein the second correlation coefficients and the data similarity have positive correlation.

6. The data acquisition method of claim 1, wherein the step of obtaining, for each of the plurality of target internet of things terminal devices, internet of things image data sent by the target internet of things terminal device comprises:

generating data acquisition notification information, and respectively sending the data acquisition notification information to each of the target internet of things terminal devices, wherein each of the target internet of things terminal devices is used for sending internet of things image data acquired and stored in a corresponding data acquisition area to the data processing platform after receiving the data acquisition notification information, and clearing the internet of things image data after sending the internet of things image data to the data processing platform;

and acquiring, by each of the plurality of target internet of things terminal devices, internet of things image data acquired by the target internet of things terminal device acquiring the corresponding data acquisition region based on the data acquisition notification information.

7. The data acquisition method as claimed in any one of claims 1 to 6, wherein the step of performing data deduplication processing on the internet of things image data for the internet of things image data corresponding to each of the plurality of target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data comprises:

for the Internet of things image data corresponding to each target Internet of things terminal device in the target Internet of things terminal devices, respectively calculating the similarity between every two adjacent frames of Internet of things monitoring images included in the Internet of things image data to obtain the image similarity between every two adjacent frames of Internet of things images;

and aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the target Internet of things terminal equipment, carrying out data deduplication processing on the Internet of things monitoring images included by the Internet of things image data based on the image similarity between every two adjacent frames of Internet of things monitoring images included by the Internet of things image data to obtain the target Internet of things image data corresponding to the Internet of things image data.

8. The data processing platform is characterized in that the data processing platform is in communication connection with a plurality of terminal devices of the internet of things and is used for executing a preconfigured data acquisition method, wherein the data acquisition method comprises the following steps:

determining a plurality of target Internet of things terminal devices from the plurality of Internet of things terminal devices, wherein the number of the target Internet of things terminal devices is less than that of the plurality of Internet of things terminal devices, and the plurality of Internet of things terminal devices are respectively arranged in a plurality of different data acquisition areas;

acquiring Internet of things image data sent by each target Internet of things terminal device in the target Internet of things terminal devices, wherein the Internet of things image data is acquired based on the corresponding target Internet of things terminal device and corresponding data acquisition regions;

and performing data deduplication processing on the internet of things image data aiming at the internet of things image data corresponding to each target internet of things terminal device in the target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data, wherein the data volume of the target internet of things image data is less than or equal to that of the corresponding internet of things image data.

9. The data processing platform of claim 8, wherein the step of determining a plurality of target internet of things terminal devices among the plurality of internet of things terminal devices comprises:

for each piece of internet-of-things terminal equipment in the plurality of pieces of internet-of-things terminal equipment, determining whether the piece of internet-of-things terminal equipment belongs to first piece of internet-of-things terminal equipment, wherein the first piece of internet-of-things terminal equipment belongs to piece of internet-of-things terminal equipment which has been used as a historical target within a preset time period, and sending historical internet-of-things image data to the data processing platform;

and aiming at each of the plurality of Internet of things terminal devices, if the Internet of things terminal device does not belong to the first Internet of things terminal device, determining whether the Internet of things terminal device is used as a target Internet of things terminal device or not so as to determine the plurality of target Internet of things terminal devices.

10. The data processing platform of claim 8, wherein the step of performing data deduplication processing on the internet of things image data for the internet of things image data corresponding to each of the plurality of target internet of things terminal devices to obtain the target internet of things image data corresponding to the internet of things image data comprises:

for the Internet of things image data corresponding to each target Internet of things terminal device in the target Internet of things terminal devices, respectively calculating the similarity between every two adjacent frames of Internet of things monitoring images included in the Internet of things image data to obtain the image similarity between every two adjacent frames of Internet of things images;

and aiming at the Internet of things image data corresponding to each target Internet of things terminal equipment in the target Internet of things terminal equipment, carrying out data deduplication processing on the Internet of things monitoring images included by the Internet of things image data based on the image similarity between every two adjacent frames of Internet of things monitoring images included by the Internet of things image data to obtain the target Internet of things image data corresponding to the Internet of things image data.

Images