CN113704341A - Data acquisition system and method - Google Patents
Data acquisition system and method Download PDFInfo
- Publication number
- CN113704341A CN113704341A CN202111006904.1A CN202111006904A CN113704341A CN 113704341 A CN113704341 A CN 113704341A CN 202111006904 A CN202111006904 A CN 202111006904A CN 113704341 A CN113704341 A CN 113704341A
- Authority
- CN
- China
- Prior art keywords
- data
- channel
- management server
- acquisition
- internet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000007726 management method Methods 0.000 claims abstract description 103
- 238000013523 data management Methods 0.000 claims abstract description 79
- 230000000295 complement effect Effects 0.000 claims abstract description 76
- 239000013589 supplement Substances 0.000 claims abstract description 17
- 230000001502 supplementing effect Effects 0.000 claims abstract description 9
- 230000005540 biological transmission Effects 0.000 claims description 48
- 230000002159 abnormal effect Effects 0.000 claims description 24
- 230000004044 response Effects 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 230000008439 repair process Effects 0.000 claims description 9
- 230000000694 effects Effects 0.000 claims description 8
- 230000037361 pathway Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 11
- 230000008569 process Effects 0.000 abstract description 5
- 238000013480 data collection Methods 0.000 description 32
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Abstract
The embodiment of the application discloses a data acquisition system and a method, which belong to the technical field of Internet of things, wherein the data acquisition system comprises a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices; the data management server is used for determining the collected data items and the non-collected data items based on the mark information corresponding to each data item; generating a data complementary acquisition command for acquiring the non-acquired data items; sending a data supplementing and collecting command to a data collecting server; the data acquisition server is used for sending a data acquisition instruction to the channel management server based on the data supplement and acquisition command; the channel management server is used for acquiring complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending complementary acquisition data to a data acquisition server; and the data acquisition server is also used for sending the complementary acquisition data to the data management server. According to the data acquisition method and device, the integrity of data acquisition is guaranteed, and the processing overhead in the data complementary acquisition process is reduced.
Description
Technical Field
The application relates to the technical field of Internet of things, in particular to a data acquisition system and a data acquisition method.
Background
At present, the application in each project scene of data acquisition is wide. In the related art, data transmission is performed between a plurality of computer devices through a communication channel. Illustratively, the first computer device sends a data acquisition request to the second computer device through the communication channel, and the second computer device provides data for the first computer device through the communication channel according to the data acquisition request after acquiring the data acquisition request.
However, in the above related art, during the process of collecting data, the computer device cannot reasonably handle the situation that data is not collected successfully due to various factors, and the situation that data is not collected completely is likely to occur.
Disclosure of Invention
The embodiment of the application provides a data acquisition system and a data acquisition method, which can ensure the integrity of data acquisition. The technical scheme is as follows:
on one hand, the embodiment of the application provides a data acquisition system, which comprises a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices;
the data management server is used for determining acquired data items and non-acquired data items based on mark information corresponding to each data item, and the mark information is used for indicating that the data items belong to the acquired data items or the non-acquired data items; generating a data complementary collection command for collecting the non-collected data items; sending the data supplementing and collecting command to the data collecting server;
the data acquisition server is used for sending a data acquisition instruction to the channel management server based on the data supplement and acquisition command, wherein the data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items;
the channel management server is used for acquiring complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending the complementary acquisition data to the data acquisition server;
and the data acquisition server is also used for sending the complementary acquisition data to the data management server.
On the other hand, an embodiment of the present application provides a data acquisition method, which is applied to a data acquisition system, where the data acquisition system includes a data management server, a data acquisition server, a channel management server, and a plurality of internet of things devices, and the method includes:
the data management server determines collected data items and non-collected data items based on mark information corresponding to each data item, wherein the mark information is used for indicating that the data items belong to the collected data items or the non-collected data items; generating a data complementary collection command for collecting the non-collected data items; sending the data supplementing and collecting command to the data collecting server;
the data acquisition server sends a data acquisition instruction to the channel management server based on the data supplement and acquisition command, wherein the data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items;
the channel management server acquires complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending the complementary acquisition data to the data acquisition server;
and the data acquisition server sends the complementary acquisition data to the data management server.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
managing each data item through a data management server, and processing the data items which are not acquired in time to ensure the integrity of data acquisition; and under the condition that the data is not completely collected, only the data items which are not collected are subjected to complementary collection processing, and the collected data items do not need to be collected again, so that the processing overhead in the data complementary collection process is reduced, the data collection rate is increased, and the timeliness of data collection is ensured.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of a data acquisition system provided by one embodiment of the present application;
FIG. 2 is a flow chart of a data collection method provided by an embodiment of the present application;
FIG. 3 illustrates an information transfer schematic between a data management server and a data collection server;
FIG. 4 illustrates a schematic diagram of a data item tagging approach;
FIG. 5 is a flow chart of a data collection method provided by another embodiment of the present application;
FIG. 6 is a flow chart of a data collection method provided by yet another embodiment of the present application;
fig. 7 is a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, a schematic diagram of a data acquisition system according to an embodiment of the present application is shown. The data acquisition system may include: the system comprises a data management server 10, a data acquisition server 20, a channel management server 30 and an internet of things device 40.
The data management server 10 is configured to manage data items corresponding to each internet of things device. Optionally, the data management server 10 may generate information to be collected of each internet of things device. In a possible embodiment, the data management server 10 generates the information to be acquired of each piece of internet-of-things equipment according to a preset rule. In another possible embodiment, after receiving a data acquisition command from a user terminal, the data management server 10 generates information to be acquired of each internet of things device based on the data acquisition command. The information to be collected comprises identification information of the Internet of things equipment, data items corresponding to the Internet of things equipment and collection time of each data item. Optionally, after the data management server 10 generates the information to be acquired of each piece of internet-of-things equipment, the data item of the information to be acquired may be marked to determine the acquired data item and the non-acquired data item in each data item, which is beneficial to timely performing complementary acquisition on the data when the data is not successfully acquired.
The data collection server 20 is used for converting and scheduling various commands from the data management server 10. Optionally, after acquiring each command from the data management server 10, the data acquisition server 20 converts the command into a data acquisition instruction, and then controls the channel management server 30 to acquire data from each internet of things device according to the data acquisition instruction. The commands comprise a data acquisition command and a data supplement and acquisition command, the data acquisition command is a command for acquiring data triggered and generated when the data is not acquired, and the data supplement and acquisition command is a command for supplementing and acquiring the data triggered and generated when the data acquisition fails. Optionally, after acquiring the above commands, the data collection server 20 may also determine the execution order of the commands according to the priority of the commands. Illustratively, the command generated by the user terminal is triggered with a higher priority than the command generated by the data management server 10 automatically. It should be noted that the data management server 10 and the data collection server 20 may be connected through a message middleware 50. The message middleware 50 is configured to store the commands from the data management server 10 and send the commands to the data collection server 20 when the load of the data collection server 20 is normal.
The channel management server 30 is used to provide data to the data collection server 20. Optionally, after acquiring the data acquisition instruction from the data acquisition server 20, the channel management server 30 determines a channel for connecting the internet of things device 40, and acquires data from the internet of things device 40 through the channel. The channel may be a channel established before the data acquisition instruction is received, or may be a channel established after the data acquisition instruction is received. Alternatively, the channel management server 30 may transmit the data to the data collection server 20 after acquiring the data, and the data collection server 20 transmits the data to the data management server 10 through the message middleware 50. It should be noted that the channel management server 30 can also be used to perform maintenance and repair on the established channel. In one possible embodiment, when the channel is in the working state, the channel management server 30 may determine whether the channel is abnormal based on the data acquisition condition, and switch a new channel to acquire data when the channel is abnormal, and repair the abnormal channel. In another possible embodiment, when the channel is in the non-operating state, the channel management server 30 may determine whether the channel is abnormal through heartbeat detection, and if the channel is abnormal, repair the channel, so as to implement maintenance on the channel.
The internet of things device 40 refers to a device that generates various items of data. Alternatively, the internet of things device 40 may be a mobile phone, a wearable device, a fan, an electric vehicle, a battery, smart furniture, a vehicle, and the like. The internet of things device 40 may provide data to the channel management server 30 after detecting the data acquisition request of the channel management server 30; the internet of things device 40 may also actively provide data to the channel management server 30 when there is a data change.
Optionally, the data management server 10, the data collection server 20, the channel management server 30 and the internet of things device 40 communicate with each other through a network.
The data management server 10, the data collection server 20, and the channel management server 30 may be independent computer devices, or may be different virtual devices provided in the same computer device. Of course, the data management server 10, the data collection server 20, and the channel management server 30 may be a cluster formed by a plurality of servers.
Referring to fig. 2, a flow chart of a data collection method according to an embodiment of the present application is shown. The method is applied to a data acquisition system, and the data acquisition system comprises a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices. The method comprises the following steps (201-207):
step 201, the data management server determines the collected data items and the non-collected data items based on the mark information corresponding to each data item.
The data management server refers to a server for managing each data item. Alternatively, the data management server may be one server, or may be a server cluster composed of a plurality of servers. Of course, in actual use, the data management server may also be a virtual server provided in a certain computer device, which is not limited in this embodiment of the present application.
The data item is used for indicating data needing to be collected, and the data management server can obtain the data item corresponding to certain Internet of things equipment to indicate the data needing to be collected from the Internet of things equipment. Optionally, the data items corresponding to different internet of things devices are different, and the same internet of things device may also correspond to one or more data items. In an embodiment of the present application, the data management server may use the tagging information to tag the data item. Wherein the tagging information is used to indicate whether the data item belongs to a collected data item or to an uncollected data item. Optionally, when the tag information corresponding to the target data item is the first tag information, determining that the target data item belongs to the acquired data item, that is, the data management server has acquired the data that needs to be acquired and is indicated by the target data item; and under the condition that the mark information corresponding to the target data item is the second mark information, determining that the target data item belongs to an uncollected data item, namely that the data management server does not acquire the data which is indicated by the target data item and needs to be collected. Wherein the first flag information and the second flag information are two different flag information.
In the embodiment of the application, the data management server determines the collected data items and the non-collected data items based on the marking information corresponding to each data item. Optionally, the data management server may scan data after acquiring data corresponding to each data item, determine acquired data and data that is not acquired, determine a data item corresponding to the acquired data as an acquired data item, and determine a data item corresponding to the data that is not acquired as an unrecovered data item.
Step 202, the data management server generates a data supplement collecting command for collecting the non-collected data items.
In the embodiment of the application, in order to ensure the integrity of data acquisition, the data management server generates a data complementary acquisition command for acquiring the non-acquired data item after determining the acquired data item and the non-acquired data item. The data complementing command is used for controlling the data acquisition server to complement and acquire the data items failed in acquisition.
Step 203, the data management server sends a data complementary acquisition command to the data acquisition server.
In the embodiment of the application, after the data supplementing and acquiring command is generated, the data management server sends the data supplementing and acquiring command to the data acquisition server. Correspondingly, the data acquisition server acquires the data supplement and acquisition command.
Optionally, a message middleware may be included between the data management server and the data collection server, that is, the data management server and the data collection server may communicate through the message middleware. The message middleware may be a server cluster composed of a plurality of servers. Optionally, after generating the data supplement and acquisition command, the data management server sends the data supplement and acquisition command to the message middleware, further, the message middleware stores the data supplement and acquisition command, acquires the state information of the data acquisition server, and sends the data supplement and acquisition command to the data acquisition server when the state information shows that the data acquisition server is working normally (i.e., the workload is not overloaded). Similarly, after acquiring the complementary data, the data acquisition server may also send the complementary data to the message middleware, and then the message middleware sends the complementary data to the data management server.
Illustratively, as shown in fig. 3, the data management server 31 determines an uncollected data item through the data marking module, generates a data complementing command for the uncollected data item through the task assembling module, sends the data complementing command to the message middleware 32 through the application interface module, and sends the data complementing command to the data acquisition server 33 through the message middleware 32.
And step 204, the data acquisition server sends a data acquisition instruction to the channel management server based on the data supplement and acquisition command.
The data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items. In the embodiment of the application, after the data acquisition server obtains the data supplement and acquisition command, the data acquisition server sends a data acquisition instruction to the channel management server based on the data supplement and acquisition command. Optionally, after the data acquisition server obtains the data complementary acquisition command, the data acquisition server converts the data complementary acquisition command into a data acquisition instruction recognizable by the internet of things device, and sends the data acquisition instruction to the channel management server.
For example, as shown in fig. 3, after the application interface module of the data collection server 33 obtains the data collection command through the message middleware 32, the data collection command is converted into a data collection command recognizable by the internet of things device through the protocol parsing module, and the time for sending the data collection command to the channel management server is determined through the task scheduling module.
In step 205, the channel management server obtains the complementary data corresponding to the non-collected data item from the internet of things device based on the data collection instruction.
In the embodiment of the application, after the channel management server obtains the data acquisition instruction, complementary acquisition data corresponding to the data items which are not acquired is obtained from the internet of things equipment. Optionally, the channel management server sends the data acquisition instruction to the internet of things device, and then acquires the complementary acquisition data corresponding to the non-acquired data item sent by the internet of things device. Certainly, in actual application, the internet of things device may also actively provide data to the data management server when data of the internet of things device changes, and at this time, the channel management server may obtain the complementary data from the data stored in the channel management server after obtaining the data acquisition instruction.
And step 206, the channel management server sends the complementary acquisition data to the data acquisition server.
In the embodiment of the present application, after acquiring the complementary data, the channel management server sends the complementary data to the data acquisition server. Correspondingly, the data acquisition server acquires the complementary acquisition data.
And step 207, the data acquisition server sends the complementary acquisition data to the data management server.
In the embodiment of the application, the data acquisition server sends the complementary acquisition data to the data management server after acquiring the complementary acquisition data.
Illustratively, as shown in fig. 3, after acquiring the complementary data, the data acquisition server 33 converts the complementary data through the data parsing module, sends the converted complementary data to the message middleware 32 through the application interface module, and sends the converted complementary data to the data management server 31 through the message middleware 32, so that the data management server 31 acquires the complementary data, and the data management server 31 stores the complementary data through the data storage module.
Optionally, in this embodiment of the application, after the data management server obtains the complementary data, based on the data identifier of the complementary data, the data management server determines a complementary data item in the non-collected data items, and modifies the label information corresponding to the complementary data item from the second label information to the first label information. Illustratively, as shown in fig. 4, the data management server obtains data corresponding to an unpurchased data item 41 from the internet of things device 42 based on the unpurchased data item 41, and further modifies the label information of the unpurchased data item 41 after obtaining the data, so that the unpurchased data item 41 is converted into a collected data item 43.
To sum up, in the technical scheme provided by the embodiment of the application, each data item is managed through the data management server, and the data items which are not successfully acquired are processed in time, so that the integrity of data acquisition is ensured; and under the condition that the data is not completely collected, only the data items which are not collected are subjected to complementary collection processing, and the collected data items do not need to be collected again, so that the processing overhead in the data complementary collection process is reduced, the data collection rate is increased, and the timeliness of data collection is ensured.
In addition, the collected data items and the data items which are not collected are marked by adopting different marking information, so that the data management server can accurately master the data collection progress in the data collection process, and the data management server is convenient to directly perform targeted processing on the data items which are not collected.
In addition, in this embodiment of the application, when acquiring the complementary acquisition data, the channel management server may acquire the complementary acquisition data through a channel connected to the internet of things device. Next, the path is described from the perspective of the path management server.
Referring to fig. 5, a flow chart of a data acquisition method according to another embodiment of the present application is shown. The execution subject of the method can be the channel management server 30 in the embodiment of fig. 1, and the method can include the following steps (501-504):
The data acquisition instruction is used for controlling the channel management server to acquire data. In the embodiment of the application, after the channel management server obtains the data acquisition instruction, a first channel for connecting the internet of things device is determined. The first channel may be any channel used for connecting the internet of things device and the channel management server. Optionally, the data acquisition instruction includes identification information used for indicating the internet of things device, and the channel management server may determine the internet of things device for which data needs to be acquired according to the identification information in the data acquisition instruction, and determine a first channel used for connecting the internet of things device.
In a possible implementation manner, before the channel management server obtains the data acquisition instruction, a channel in an idle state exists between the channel management server and the internet of things device, in this case, after the channel management server obtains the data acquisition instruction, the channel management server determines the first channel from the existing channels. Optionally, in the presence of a channel for the internet of things device, the channel management server determines a first channel from the existing channels based on the data acquisition instruction, and converts the first channel from an idle state to a working state.
In another possible implementation manner, before the channel management server obtains the data acquisition instruction, there is no channel in an idle state between the channel management server and the internet of things device, in this case, after the channel management server obtains the data acquisition instruction, the channel management server establishes a first channel for the internet of things device. Optionally, the channel management server establishes at least one channel for the internet of things device based on the data acquisition instruction when the channel for the internet of things device does not exist. Wherein the at least one channel includes the first channel.
Optionally, in order to ensure timeliness of data transmission, the channel between the channel management server and the internet of things device is not disconnected after being established, that is, when data transmission is not performed, the channel between the channel management server and the internet of things device is in an idle state. In this case, the path management server needs to maintain and repair the path in the idle state. Taking the first channel as an example, when the first channel is in an idle state, the channel detection information is sent to the internet of things device, and at this time, the internet of things device may send feedback information to the channel management server for the detection information. Further, when receiving feedback information aiming at the channel detection information from the internet of things equipment, the channel management server determines that the first channel is normal; otherwise, the channel management server determines that the first channel is abnormal when the channel management server does not receive feedback information aiming at the channel detection information from the internet of things equipment, and then repairs the first channel. Of course, in order to reduce the maintenance energy consumption for the idle channel, the channel between the channel management server and the internet of things device may also be directly disconnected when data transmission is not performed.
In this embodiment, after determining the first channel, the channel management server obtains the complementary data from the first channel.
Optionally, in order to ensure smooth data transmission, the channel management server may monitor the data transmission condition of the first channel in real time when determining that the first channel is in the working state, so as to ensure smooth data transmission.
In the embodiment of the application, when the channel management server is in a working state, the channel management server obtains the data transmission condition of the first channel in the data transmission process. Wherein the data transmission condition is used to indicate a data transmission effect of the first channel, optionally, the data transmission condition includes but is not limited to at least one of the following: data transmission efficiency, maximum interval duration of data transmission, minimum rate of data transmission, and the like.
And step 504, in response to the abnormal data transmission condition, replacing the first channel with the second channel to obtain the complementary data.
In the embodiment of the application, after acquiring the data transmission condition, the computer device detects the data transmission condition, and when determining that the data transmission condition is abnormal, the computer device acquires the complementary acquisition data by replacing the first channel with the second channel. The second channel is any channel except the first channel and used for connecting the Internet of things equipment. Illustratively, as shown in fig. 1, a primary channel 60 and a backup channel 70 exist between the channel management server 30 and the internet of things device 40. In the data transmission process, if the main channel 60 is abnormal, the standby channel 70 is adopted to replace the main channel 60 for data transmission.
In a possible implementation manner, a standby channel exists between the channel management server and the internet of things device, and the channel management server may obtain the second channel from the standby channel when determining that data transmission is abnormal. Optionally, when it is determined that the data transmission condition is abnormal, the channel management server obtains a standby channel for the internet of things device. The standby channel refers to other channels except the first channel for the internet of things equipment. Further, the channel management server acquires the channel with the highest priority from the standby channels, and determines the channel as the second channel. The priority is in direct proportion to the data transmission effect of the channel, and the better the data transmission effect of the channel is, the higher the priority of the channel is; that is, when the old channel is replaced with the new channel, the channel with the better data transmission effect is preferentially selected. It should be noted that the priority may be determined according to a preset rule when the channel is established, for example, the priority of the 4G channel is higher than that of the 2G channel; alternatively, the priority may be a priority that is determined by the channel management server collecting the data transmission effect of each time in the data transmission process and comprehensively according to the data transmission effect of each time.
In another possible implementation, if a standby channel does not exist between the channel management server and the internet of things device, the channel management server may establish a second channel for the internet of things device when determining that data transmission is abnormal. Optionally, when it is determined that the data transmission condition is abnormal, a standby channel for the internet of things device is established, and the standby channel is determined to be the second channel.
It should be noted that, in this embodiment of the present application, when determining that the first channel is abnormal, the channel management server may repair the first channel, and after the first channel is repaired, replace the second channel with the first channel to obtain the complementary data.
In summary, in the technical scheme provided by the embodiment of the application, when data transmission is performed, a channel for data transmission is arranged between the channel management server and the internet of things device, which is beneficial to ensuring data transmission efficiency; and when the channel is abnormal, the new channel is used for replacing the old channel, so that the stability and the integrity of data transmission are ensured.
The above is an introduction of the data acquisition method in terms of data complementary acquisition, and the following is an introduction of the data acquisition method in the present application in terms of data acquisition.
Referring to fig. 6, a flow chart of a data collection method according to still another embodiment of the present application is shown. The method is applied to a data acquisition system, and the data acquisition system comprises a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices. The method comprises the following steps (601-611):
step 601, the data management server generates a data item corresponding to the internet of things device. And the data item corresponding to the Internet of things equipment is an uncollected data item. Optionally, the data management server may generate a data item corresponding to a certain internet of things device based on a preset rule, or may generate a data item corresponding to a certain internet of things device after receiving a data acquisition command triggered by the user terminal.
Step 602, the data management server generates a data acquisition command for acquiring a data item corresponding to the internet of things device.
Step 603, the data management server sends a data acquisition command to the data acquisition server.
Step 604, the data acquisition server sends a data acquisition instruction to the channel management server based on the data acquisition command. The data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the data items corresponding to the Internet of things equipment.
Step 605, the channel management service determines a first channel for connecting the internet of things device based on the data acquisition instruction.
In step 606, the channel management server obtains data from the first channel.
Step 607, under the condition that the first channel is in the working state, the channel management server obtains the data transmission condition of the first channel.
Step 608, in response to the abnormal data transmission condition, the channel management server replaces the first channel with the second channel to obtain the data.
Step 609, the channel management server sends the data to the data collection server.
Step 610, the data acquisition server sends data to the data management server.
Step 611, the data management server determines the collected data items and the non-collected data items based on the acquired data and performs data complementary collection.
The following are embodiments of the system of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the system of the present application, reference is made to the embodiments of the method of the present application.
An exemplary embodiment of the present application further provides a data acquisition system, where the data acquisition system includes a data management server, a data acquisition server, a channel management server, and a plurality of internet of things devices;
the data management server is used for determining acquired data items and non-acquired data items based on mark information corresponding to each data item, and the mark information is used for indicating that the data items belong to the acquired data items or the non-acquired data items; generating a data complementary collection command for collecting the non-collected data items; sending the data supplementing and collecting command to the data collecting server;
the data acquisition server is used for sending a data acquisition instruction to the channel management server based on the data supplement and acquisition command, wherein the data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items;
the channel management server is used for acquiring complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending the complementary acquisition data to the data acquisition server;
and the data acquisition server is also used for sending the complementary acquisition data to the data management server.
In an exemplary embodiment, the data management server is configured to determine that a target data item belongs to the acquired data item when tag information corresponding to the target data item is first tag information;
determining that the target data item belongs to the non-collected data item under the condition that the mark information corresponding to the target data item is second mark information; wherein the first label information and the second label information are two different label information.
In an exemplary embodiment, the data management server is further configured to determine, based on the data identification of the complementary data, a complementary data item in the non-acquired data items; and modifying the mark information corresponding to the supplemented data item into the first mark information from the second mark information.
In an exemplary embodiment, the channel management server is configured to determine, based on the data acquisition instruction, a first channel for connecting the internet of things device; and acquiring the complementary data from the first channel.
In an exemplary embodiment, the channel management server is configured to determine, based on the data acquisition instruction, the first channel from existing channels when a channel for the internet of things device exists; converting the first channel from an idle state to a working state; or, on the condition that no channel for the internet of things device exists, establishing at least one channel for the internet of things device based on the data acquisition instruction, wherein the at least one channel comprises the first channel.
In an exemplary embodiment, the channel management server is further configured to obtain a data transmission condition of the first channel when the first channel is in a working state; and in response to the abnormal data transmission condition, replacing the first channel with a second channel to obtain the complementary data.
In an exemplary embodiment, the channel management server is configured to, in response to the data transmission condition being abnormal, obtain a standby channel for the internet of things device, where the standby channel refers to a channel other than the first channel for the internet of things device; acquiring a channel with the highest priority from the standby channels, and determining the channel as the second channel, wherein the priority is in direct proportion to the data transmission effect of the channel; or, in response to the data transmission condition being abnormal, establishing a standby channel for the internet of things equipment; determining the backup channel as the second channel.
In an exemplary embodiment, the channel management server is configured to repair the first channel; and in response to the completion of the repair of the first channel, replacing the second channel with the first channel to acquire the complementary data.
In an exemplary embodiment, the channel management system is further configured to send channel detection information to the internet of things device when the first channel is in an idle state; determining that the first channel is normal in response to receiving feedback information for the channel detection information from the internet of things device; and in response to not receiving feedback information aiming at the channel detection information from the Internet of things equipment, determining that the first channel is abnormal, and repairing the first channel.
Referring to fig. 7, a block diagram of a computer device according to an embodiment of the present application is shown. The computer equipment can be used for realizing the functions of the data acquisition method. The computer device can be any one of a data management server, a data acquisition server and a channel management server. Specifically, the method comprises the following steps:
the computer device 700 includes a Central Processing Unit (CPU) 701, a system Memory 704 including a Random Access Memory (RAM) 702 and a Read Only Memory (ROM) 703, and a system bus 705 connecting the system Memory 704 and the CPU 701. The computer device 700 also includes a basic Input/Output (I/O) system 706 that facilitates information transfer between various devices within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714, and other program modules 712.
The basic input/output system 706 includes a display 708 for displaying information and an input device 709, such as a mouse, keyboard, etc., for a user to input information. Wherein the display 708 and the input device 709 are connected to the central processing unit 701 through an input output controller 710 connected to the system bus 705. The basic input/output system 706 may also include an input/output controller 710 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 710 may also provide output to a display screen, a printer, or other type of output device.
The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer-readable media provide non-volatile storage for the computer device 700. That is, the mass storage device 707 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact disk Read-Only Memory) drive.
Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash Memory or other solid state Memory technology, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 704 and mass storage device 707 described above may be collectively referred to as memory.
According to various embodiments of the present application, the computer device 700 may also operate as a remote computer connected to a network via a network, such as the Internet. That is, the computer device 700 may be connected to the network 712 through the network interface unit 711 connected to the system bus 705, or may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 711.
The memory also includes a computer program stored in the memory and configured to be executed by the one or more processors to implement the data acquisition method described above.
In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which when executed by a processor, implements the above data acquisition method.
Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State drive), or optical disc. The Random Access Memory may include a ReRAM (resistive Random Access Memory) and a DRAM (Dynamic Random Access Memory).
In an exemplary embodiment, a computer program product is also provided, which, when being executed by a processor, is adapted to carry out the above-mentioned data acquisition method.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.
The above description is only exemplary of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A data acquisition system is characterized by comprising a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices;
the data management server is used for determining acquired data items and non-acquired data items based on mark information corresponding to each data item, and the mark information is used for indicating that the data items belong to the acquired data items or the non-acquired data items; generating a data complementary collection command for collecting the non-collected data items; sending the data supplementing and collecting command to the data collecting server;
the data acquisition server is used for sending a data acquisition instruction to the channel management server based on the data supplement and acquisition command, wherein the data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items;
the channel management server is used for acquiring complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending the complementary acquisition data to the data acquisition server;
and the data acquisition server is also used for sending the complementary acquisition data to the data management server.
2. The system of claim 1, wherein the data management server is configured to:
determining that the target data item belongs to the acquired data item under the condition that the marking information corresponding to the target data item is first marking information;
determining that the target data item belongs to the non-collected data item under the condition that the mark information corresponding to the target data item is second mark information;
wherein the first label information and the second label information are two different label information.
3. The system of claim 2, wherein the data management server is further configured to:
determining supplemented data items in the non-collected data items based on the data identification of the supplemented data;
and modifying the mark information corresponding to the supplemented data item into the first mark information from the second mark information.
4. The system according to any one of claims 1 to 3, wherein the path management server is configured to:
determining a first channel for connecting the Internet of things equipment based on the data acquisition instruction;
and acquiring the complementary data from the first channel.
5. The system of claim 4, wherein the path management server is configured to:
determining the first channel from existing channels based on the data acquisition instruction under the condition that the channel for the Internet of things equipment exists; converting the first channel from an idle state to a working state;
alternatively, the first and second electrodes may be,
under the condition that a channel for the Internet of things equipment does not exist, establishing at least one channel for the Internet of things equipment based on the data acquisition instruction, wherein the at least one channel comprises the first channel.
6. The system of claim 4, wherein the path management server is further configured to:
acquiring the data transmission condition of the first channel under the condition that the first channel is in a working state;
and in response to the abnormal data transmission condition, replacing the first channel with a second channel to obtain the complementary data.
7. The system of claim 6, wherein the path management server is configured to:
responding to the abnormal data transmission condition, and acquiring a standby channel for the Internet of things equipment, wherein the standby channel refers to other channels except the first channel for the Internet of things equipment; acquiring a channel with the highest priority from the standby channels, and determining the channel as the second channel, wherein the priority is in direct proportion to the data transmission effect of the channel;
alternatively, the first and second electrodes may be,
responding to the abnormal data transmission condition, and establishing a standby channel for the Internet of things equipment; determining the backup channel as the second channel.
8. The system of claim 6, wherein the path management server is configured to:
repairing the first channel;
and in response to the completion of the repair of the first channel, replacing the second channel with the first channel to acquire the complementary data.
9. The system of claim 4, wherein the pathway management system is further configured to:
under the condition that the first channel is in an idle state, channel detection information is sent to the Internet of things equipment;
determining that the first channel is normal in response to receiving feedback information for the channel detection information from the internet of things device;
and in response to not receiving feedback information aiming at the channel detection information from the Internet of things equipment, determining that the first channel is abnormal, and repairing the first channel.
10. A data acquisition method is applied to a data acquisition system, wherein the data acquisition system comprises a data management server, a data acquisition server, a channel management server and a plurality of Internet of things devices, and the method comprises the following steps:
the data management server determines collected data items and non-collected data items based on mark information corresponding to each data item, wherein the mark information is used for indicating that the data items belong to the collected data items or the non-collected data items; generating a data complementary collection command for collecting the non-collected data items; sending the data supplementing and collecting command to the data collecting server;
the data acquisition server sends a data acquisition instruction to the channel management server based on the data supplement and acquisition command, wherein the data acquisition instruction is an instruction which can be identified by the Internet of things equipment and is used for acquiring the non-acquired data items;
the channel management server acquires complementary acquisition data corresponding to the non-acquired data items from the Internet of things equipment based on the data acquisition instruction; sending the complementary acquisition data to the data acquisition server;
and the data acquisition server sends the complementary acquisition data to the data management server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111006904.1A CN113704341A (en) | 2021-08-30 | 2021-08-30 | Data acquisition system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111006904.1A CN113704341A (en) | 2021-08-30 | 2021-08-30 | Data acquisition system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113704341A true CN113704341A (en) | 2021-11-26 |
Family
ID=78657028
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111006904.1A Pending CN113704341A (en) | 2021-08-30 | 2021-08-30 | Data acquisition system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113704341A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114287897A (en) * | 2021-12-28 | 2022-04-08 | 深圳市汇顶科技股份有限公司 | Blood pressure data acquisition method, chip, electronic device and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101707399A (en) * | 2009-11-20 | 2010-05-12 | 深圳市科陆电子科技股份有限公司 | Method and system for acquiring electric energy information |
| CN201955815U (en) * | 2010-11-11 | 2011-08-31 | 江苏苏源高科技有限公司 | Power utilization information intelligent concentrating system among users |
| JP2016095652A (en) * | 2014-11-13 | 2016-05-26 | 富士通株式会社 | Data storage method, and information processing device |
| CN110489407A (en) * | 2019-07-29 | 2019-11-22 | 南方电网数字电网研究院有限公司 | Data filling mining method, apparatus, computer equipment and storage medium |
| CN111368165A (en) * | 2020-02-25 | 2020-07-03 | 江苏物联网研究发展中心 | Spatio-temporal streaming data integration platform |
| CN112071050A (en) * | 2020-08-12 | 2020-12-11 | 南京南瑞信息通信科技有限公司 | Concentrator terminal and electricity consumption data acquisition system |
| CN112559505A (en) * | 2020-12-21 | 2021-03-26 | 深圳市航天泰瑞捷电子有限公司 | Real-time data complementary acquisition device and method, computer equipment and storage medium |
-
2021
- 2021-08-30 CN CN202111006904.1A patent/CN113704341A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101707399A (en) * | 2009-11-20 | 2010-05-12 | 深圳市科陆电子科技股份有限公司 | Method and system for acquiring electric energy information |
| CN201955815U (en) * | 2010-11-11 | 2011-08-31 | 江苏苏源高科技有限公司 | Power utilization information intelligent concentrating system among users |
| JP2016095652A (en) * | 2014-11-13 | 2016-05-26 | 富士通株式会社 | Data storage method, and information processing device |
| CN110489407A (en) * | 2019-07-29 | 2019-11-22 | 南方电网数字电网研究院有限公司 | Data filling mining method, apparatus, computer equipment and storage medium |
| CN111368165A (en) * | 2020-02-25 | 2020-07-03 | 江苏物联网研究发展中心 | Spatio-temporal streaming data integration platform |
| CN112071050A (en) * | 2020-08-12 | 2020-12-11 | 南京南瑞信息通信科技有限公司 | Concentrator terminal and electricity consumption data acquisition system |
| CN112559505A (en) * | 2020-12-21 | 2021-03-26 | 深圳市航天泰瑞捷电子有限公司 | Real-time data complementary acquisition device and method, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| DA CRUZ, MAURO A. A等: "Performance evaluation of IoT middleware", JOURNAL OF NETWORK AND COMPUTER APPLICATIONS, 1 May 2018 (2018-05-01), pages 53 - 65 * |
| 王敏芳: "移动智能网性能管理若干关键技术的研究与应用", 中国优秀硕士学位论文全文数据库 (信息科技辑), 15 March 2010 (2010-03-15), pages 136 - 484 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114287897A (en) * | 2021-12-28 | 2022-04-08 | 深圳市汇顶科技股份有限公司 | Blood pressure data acquisition method, chip, electronic device and readable storage medium |
| CN114287897B (en) * | 2021-12-28 | 2024-02-06 | 深圳市汇顶科技股份有限公司 | Blood pressure data acquisition method, chip, electronic device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102045192A (en) | Apparatus and system for estimating network configuration | |
| CN110069210B (en) | Storage system, and method and device for allocating storage resources | |
| CN111818159A (en) | Data processing node management method, device, equipment and storage medium | |
| CN109376011A (en) | The method and apparatus of resource are managed in virtualization system | |
| CN110908806A (en) | Mixed flow task management method, device, equipment and storage medium | |
| CN111045811A (en) | Task allocation method and device, electronic equipment and storage medium | |
| US20120072589A1 (en) | Information Processing Apparatus and Method of Operating the Same | |
| CN113704341A (en) | Data acquisition system and method | |
| CN114143023A (en) | Monitoring system for running states of multiple low-voltage transformer areas | |
| US9391916B2 (en) | Resource management system, resource management method, and computer product | |
| CN110248156B (en) | Video analysis method, platform equipment, intelligent camera and intelligent monitoring system | |
| CN105450679A (en) | Method and system for performing data cloud storage | |
| CN109542841B (en) | Method for creating data snapshot in cluster and terminal equipment | |
| CN109302319B (en) | Message pool distributed cluster and management method thereof | |
| CN104079431A (en) | Instant-messaging session management method and session management device | |
| KR101883671B1 (en) | Method and management server for dtitributing node | |
| CN103677994B (en) | Distributed data processing system, device and method | |
| CN109376509A (en) | A kind of KVM task distributes system and automatic distributing method automatically | |
| US10353365B2 (en) | Electric power facilities identification number generation apparatus and method | |
| CN114567519A (en) | Method and device for multithread parallel management of instruction messages of multiple intelligent devices | |
| CN114243914A (en) | Electric power monitoring system | |
| CN103870499A (en) | Method and device for hidden danger information processing of power transmission network | |
| CN113535402A (en) | Load balancing processing method and device based on 5G MEC and electronic equipment | |
| CN111324513A (en) | Monitoring management method and system for artificial intelligence development platform | |
| CN111158899A (en) | Data acquisition method, data acquisition device, task management center and task management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |