CN117591496B

CN117591496B - High-reliability time sequence data transmission and storage system based on cloud edge cooperation

Info

Publication number: CN117591496B
Application number: CN202410070799.5A
Authority: CN
Inventors: 程敏敏; 刘宇航; 兰洋; 何栓; 方华建; 李敏
Original assignee: Research Institute of Nuclear Power Operation; China Nuclear Power Operation Technology Corp Ltd
Current assignee: Research Institute of Nuclear Power Operation; China Nuclear Power Operation Technology Corp Ltd
Priority date: 2024-01-18
Filing date: 2024-01-18
Publication date: 2024-05-03
Anticipated expiration: 2044-01-18
Also published as: CN117591496A

Abstract

A cloud edge cooperation-based high-reliability time sequence data transmission and storage system comprises: the system comprises a real-time data transmission module, a historical data transmission module, a metadata storage and management module and a time sequence data processing platform, wherein the real-time data transmission module is used for acquiring real-time data of an edge side and then writing the corresponding real-time data into a target database for storage; the historical data transmission module is used for generating a historical supplementary acquisition task and transmitting the historical data which is supplementary acquired based on the historical supplementary acquisition task to the target database for storage; the time sequence data processing platform comprises a target database, wherein the target database is used for receiving and storing real-time data and supplementary collected historical data; the metadata storage and management module is used for recording and managing metadata records generated by the operation of the system; through the cooperation of the modules, complete and reliable transmission and storage of time sequence data from the edge side to the center side can be realized.

Description

High-reliability time sequence data transmission and storage system based on cloud edge cooperation

Technical Field

The invention relates to the technical field of nuclear power, in particular to a cloud edge cooperation-based high-reliability time sequence data transmission and storage system.

Background

The edge side of the nuclear power industrial internet platform is connected with and stores massive time sequence data of a power plant, the time sequence data can be divided into near real-time data and historical data according to the service characteristics of nuclear power data, and the problems of incomplete and unreliable data are often encountered in the process of transmitting the real-time data and the historical data from the edge side to the center side.

The near real-time data has the characteristics of real-time performance and short-term caching. The process of data acquisition, data caching, data distribution, data storage and the like is involved in a full data link for implementing real-time data access based on a nuclear power industrial interconnection platform, and the situation of real-time data access interruption caused by uncertain factors such as program error reporting, network interruption, system upgrading and the like can occur in the process of data real-time transmission, and in addition, the problem that the system cannot automatically trace back errors and the problem of data compensation also exist after data transmission fails. The historical data has the characteristics of large data volume and large time span, and due to the fact that a historical transmission link is long, data transfer links are more, and the like, the data finally stored in the nuclear power industrial Internet platform has incomplete problems, and due to the fact that various applications and analysis are needed to be carried out on the basis of the historical data in the nuclear power industrial Internet platform, the quality requirements on the historical data in a continuous interval are high, otherwise the reliability of the applications and the analysis can be affected, and furthermore, due to the fact that the historical transmission link is long, the data transfer links are more, the effective positioning and the analysis of the problem links cannot be timely realized.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a cloud-edge collaboration-based high-reliability time-series data transmission and storage system for the integrity and reliability of edge-to-center data acquisition and transmission, especially the complete and reliable transmission of near-real-time data and the complete and reliable migration of historical data.

The invention provides a cloud edge cooperation-based high-reliability time sequence data transmission and storage system, which is used for transmitting time sequence data from an edge side to a center side and comprises the following steps: the system comprises a real-time data transmission module, a historical data transmission module, a metadata storage and management module and a time-ordered data processing platform;

the time sequence data processing platform comprises a target database, wherein the target database is used for receiving and storing real-time data and supplementary collected historical data which are respectively sent by the real-time data transmission module and the historical data transmission module;

The metadata storage and management module is used for recording and managing metadata records generated by the system operation, and comprises a measuring point metadata management sub-module, wherein the measuring point metadata management sub-module is used for managing measuring point metadata in a source database and a target database at the edge side, and the measuring point metadata at least comprises a measuring point ID and a value type;

The real-time data transmission module is used for acquiring real-time data of an edge side, calling the measuring point metadata management submodule to acquire the value type of a measuring point to be written in the real-time data, and then writing the corresponding real-time data into the target database for storage;

The historical data transmission module is used for generating a historical supplementary acquisition task and sending the historical data which is supplementary acquired based on the historical supplementary acquisition task to the target database for storage, and comprises the following steps: the system comprises a historical data acquisition sub-module and a historical data transmission sub-module which are arranged on the edge side, a historical data acquisition task generation sub-module arranged on the center side, a rotor module in historical data and an importing sub-module, wherein:

the historical data acquisition task generation submodule comprises a historical data comparison unit, wherein the historical data comparison unit is used for comparing the historical data of the target database with the historical data of the appointed database in an appointed time period, generating a difference record according to a comparison result, and generating a historical acquisition task based on the difference record;

The historical data acquisition sub-module is in communication connection with the historical data acquisition task generation sub-module and is used for acquiring corresponding historical data from the source database according to the historical acquisition task;

The historical data transmission submodule is respectively in communication connection with the historical data acquisition submodule and the historical data middle rotor submodule, and is used for receiving the historical data sent by the historical data acquisition submodule and calling the metadata measurement point metadata management submodule to acquire the value type of a measurement point to be written in the historical data, and then sending the corresponding historical data to the historical data middle rotor module:

A transfer database is arranged in the rotor module in the historical data, and the transfer database is used for receiving, storing and forwarding the historical data sent by the historical data transmission submodule;

The importing sub-module is in communication connection with the historical data transfer sub-module and is used for converting the historical data in the transfer database into a database format file and importing the database format file into the target database.

Preferably, in the historical data comparison submodule, the specified database is a source database, and the historical data comparison unit is used for comparing a first time sequence data file of a corresponding time period with the source database after inquiring and deriving the first time sequence data file from the target database according to a specified time period, wherein the first time sequence data file at least comprises a measuring point ID, a collection total time period and a data total number corresponding to the collection total time period;

Or in the historical data comparison sub-module, the specified database is a transit database, and the historical data comparison unit is used for comparing the second time sequence data file with the transit database after inquiring and deriving the second time sequence data file of the corresponding time period from the target database according to the specified time period, wherein the second time sequence data file at least comprises a measuring point ID, a collecting total time period, actual sampling time included in each total time period and sampling values corresponding to each actual sampling time.

Preferably, the historical data acquisition task generating sub-module further comprises a task management unit, wherein the task management unit is in communication connection with the historical data comparison unit and is used for carrying out arrangement management on the historical acquisition tasks and carrying out statistics and analysis on the number, information and running state of the final historical acquisition tasks generated after arrangement management.

Preferably, the rotor module in the historical data further comprises a transmission metadata generation unit, wherein the transmission metadata generation unit is used for generating transmission metadata based on a transmission record of each time of historical data transmitted from the edge side to the center side, the transmission metadata is used for representing data information of a transmission process record, and the transmission metadata at least comprises a task name, a transmission period, and the total number of folders and the total number of files transmitted in the transmission period; the metadata storage and management module is correspondingly provided with a transmission metadata management sub-module, and the transmission metadata management sub-module is in communication connection with the rotor module in the historical data and is used for receiving and storing the transmission metadata.

Further, the time sequence data processing platform further comprises a storage metadata generation submodule, wherein the storage metadata generation submodule is used for generating storage metadata based on each write record of the real-time data and the historical data written into the target database, each storage metadata at least comprises a measuring point name, a write time period and a write data volume corresponding to the write time period, and the write data volume is used for representing the total write data volume of the measuring point stored at the center side on the appointed date; the metadata storage and management module is correspondingly provided with a storage metadata management sub-module, and the storage metadata management sub-module is in communication connection with the time sequence data processing platform and is used for receiving and storing the storage metadata.

Further, each piece of storage metadata stores the written data quantity of a specified measuring point for one year, wherein the written data quantity corresponding to the specified year is converted into the data of a specified system and then sequentially stored in the storage metadata according to the time sequence.

Preferably, the history transmission module further comprises a history data grading sub-module, wherein the history data grading sub-module is provided with an archiving database, and the archiving database is in communication connection with the transfer database and is used for archiving and storing history data of the transfer database.

The history data profiling sub-module further comprises an archiving metadata generation unit, wherein the archiving metadata generation unit is used for generating corresponding archiving metadata based on archiving records of each piece of history data in the transit database, and each piece of archiving metadata at least comprises an archiving file name, an archiving file size, archiving time and a task to which the archiving metadata belongs; the metadata storage and management module is correspondingly provided with an archiving metadata management sub-module, and the archiving metadata management sub-module is in communication connection with the historical data archiving sub-module and is used for receiving and storing each piece of archiving metadata.

Preferably, the historical data transmission module further comprises a historical data preprocessing sub-module arranged on the edge side, and the historical data preprocessing sub-module is in communication connection with the historical data transmission sub-module and is used for compressing and/or encrypting the historical data in the historical data transmission sub-module.

And/or the historical data transmission module further comprises a data link reliability monitoring submodule arranged at the center side, wherein the data link reliability monitoring submodule is used for carrying out link monitoring on the historical data acquisition submodule, the historical transmission submodule, the rotor module in the historical data, the importing submodule and the historical data grading submodule and carrying out abnormal condition alarm processing.

Preferably, the real-time transmission function module includes:

the real-time data acquisition sub-module is arranged at the edge side and is used for acquiring real-time data from a real-time data source at the edge side;

the real-time data transmission sub-module is arranged at the edge side and used for forwarding the acquired real-time data to the center side and/or other applications respectively;

The real-time data access sub-module is arranged on the center side and is in communication connection with the real-time data transmission sub-module, and is used for accessing the real-time data forwarded by the real-time data transmission sub-module to the center side;

the real-time data calculation and forwarding module is arranged at the center side and is used for screening, filtering and additionally calculating the edge business logic of the real-time data flow accessed by the real-time data access module and forwarding the real-time data flow to the target database;

An interrupt retry sub-module, configured to retry transmission for multiple times for network fluctuation or abnormal conditions;

and the missing data compensation sub-module is used for retransmitting missing near-real-time data.

Preferably, the historical data acquisition sub-module further comprises a first acquisition metadata generation unit, wherein the first acquisition metadata generation unit is used for generating acquisition metadata based on each historical data acquisition record; the real-time data acquisition sub-module further comprises a second acquisition metadata generation unit, wherein the second acquisition metadata generation unit is used for generating acquisition metadata based on each real-time data acquisition record, each acquisition metadata at least comprises a measuring point name, an acquisition time period and corresponding acquisition data quantity in the acquisition time period, and each acquisition data quantity is used for representing the total amount of acquisition data of a specified date of a measuring point; the metadata storage and management module is correspondingly provided with an acquisition metadata management submodule, and the acquisition metadata management submodule is respectively in communication connection with the historical data transmission submodule and the real-time data transmission submodule and is used for receiving and storing the acquisition metadata.

Further, each piece of acquisition metadata stores the acquisition data quantity of a specified measuring point for one year, wherein the specified year corresponds to all the acquisition data quantity and is sequentially stored in the acquisition metadata according to the time sequence after being converted into the data of a specified system.

Preferably, the metadata storage and management module further comprises a metadata processing sub-module, and the metadata processing sub-module is respectively in communication connection with the acquisition metadata management sub-module, the transmission metadata management sub-module, the storage metadata management sub-module and the archiving metadata management sub-module, so as to acquire the acquisition metadata, the transmission metadata, the storage metadata and the archiving metadata and then process various metadata.

Further, the steps of acquiring the acquisition metadata, transmitting the metadata, storing the metadata and archiving the metadata, and then processing the metadata of various types include:

a, acquiring a time range and a measuring point range of data to be positioned;

B, for each measuring point in the measuring point range, finding a corresponding archive file path from the archive metadata based on a time range, and downloading corresponding archive data from an archive database based on the archive file path; meanwhile, the corresponding written data quantity is found from the storage metadata based on the time range;

C, counting the amount of the archived data based on the archived data, and comparing the amount of the archived data with the amount of the written data;

If the two are equal, judging that the acquisition link and the transmission link possibly have problems; then

C11, respectively finding out corresponding acquired data quantity and file quantity from the acquired metadata and the transmission metadata based on the time range for each measuring point in the measuring point range;

C12 judges whether a corresponding file exists or not based on the value of each acquired data volume and written data volume, wherein the value is not 0, judges that the file exists, and respectively acquires the total amount of the corresponding acquired file and the total amount of the stored file after summarizing;

And C13, comparing the total amount of the acquired files, the total amount of the stored files and the amount of the files, and judging that the corresponding links have problems if one group of the files is different in number.

Or C12' respectively finding out the corresponding measuring point names and the number of folders from the acquisition metadata, the transmission metadata and the storage metadata for each measuring point in the measuring point range;

C13' counting the number of the measuring point names found in the acquisition metadata and the storage metadata to obtain the total amount of the corresponding acquisition folders and the total amount of the storage folders, wherein if the corresponding measuring point names are found, the existence of the corresponding folders is judged;

And C14' comparing the total amount of the acquired folders, the total amount of the stored folders and the file amount, and judging that the corresponding link has a problem if one group of the total amount of the stored folders and the file amount are different.

And C2, if the volume of the archived data is larger than the volume of the written data, judging that the writing link has a problem.

Further, the steps of acquiring the acquisition metadata, transmitting the metadata, storing the metadata and archiving the metadata, and then processing the metadata of various types further comprise periodic analysis: the periodic analysis includes a combination of one or more of the following:

Acquiring and displaying the acquisition metadata, namely acquiring the acquisition record of the corresponding acquisition metadata based on the measurement point name and the inquiry time period after acquiring the measurement point name and the inquiry time period, wherein the acquisition record comprises the acquisition condition of the measurement point every day or the acquisition state of the acquired measurement point according to years; or, using the collected metadata to count the data source distribution condition of the collected data;

Or, for the storage metadata, acquiring the name of the measuring point and the inquiry time period, acquiring the corresponding storage metadata based on the name of the measuring point and the inquiry time period, and displaying the record of the measuring point stored in the target database;

Or, for the transmission metadata, acquiring the task name, and then acquiring the corresponding transmission metadata based on the task name to display the total number of folders and the total number of files stored in the transmission metadata.

Preferably, the real-time transmission function module further comprises a monitoring alarm sub-module, and the monitoring alarm sub-module is used for recording abnormal logs and displacement of data, and alarming based on the abnormality.

Preferably, the cloud-edge cooperation-based high-reliability time sequence data transmission and storage system is used for transmission and storage of nuclear power time sequence data.

Compared with the prior art, the invention has the following beneficial effects: the high-reliability time sequence data transmission and storage system based on cloud edge cooperation can effectively ensure the availability, integrity and reliability of time sequence data access and historical data migration from the edge side to the center side, and particularly the time sequence data access and the historical data migration of the edge layer of the nuclear power industry Internet. Specifically, the real-time data transmission module realizes fault recovery and data recovery through monitoring of software service and hardware related to a transmission link, so that the problem of near-real-time sequence data complete and reliable transmission of the edge layer of the nuclear power industry Internet is guaranteed, the problem of complete and reliable migration of historical data is solved through data comparison and link monitoring, and the metadata storage and management module is responsible for recording and managing metadata records generated by all operations and providing inquiry interfaces for comparison and analysis for unified management of data standards and tracing of problem links in the system transmission process.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

FIG. 1 is a schematic diagram of an embodiment of a cloud-edge collaboration-based high-reliability time-series data transmission and storage system;

FIG. 2 is a schematic diagram of a workflow of an embodiment of a cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to the present invention;

In the figure, a real-time data transmission module 10, a real-time data acquisition sub-module 11, a real-time data transmission sub-module 12, a real-time data access sub-module 13, a real-time data calculation and forwarding module 14, an interrupt retry sub-module 15, a missing data compensation sub-module 16 and a monitoring alarm sub-module 17; the system comprises a historical data transmission module 20, a historical data acquisition sub-module 21, a historical data transmission sub-module 22, a historical data acquisition task generation sub-module 23, a historical data middle rotor module 24, an importing sub-module 25, a historical data preprocessing sub-module 26, a historical data grading sub-module 27 and a data link reliability monitoring sub-module 28; the system comprises a metadata storage and management module 30, a measurement point metadata management sub-module 31, a collection metadata management sub-module 32, a transmission metadata management sub-module 33, an archiving metadata management sub-module 34, a storage metadata management sub-module 35 and a metadata processing sub-module 36; a time series data processing platform 40, a target database 41, a storage metadata generation sub-module 42.

Detailed Description

The following detailed description of preferred embodiments of the application is made in connection with the accompanying drawings, which form a part hereof, and together with the description of the embodiments of the application, are used to explain the principles of the application and are not intended to limit the scope of the application.

Referring to fig. 1, a cloud-edge collaboration-based high-reliability time-series data transmission and storage system in the present embodiment is configured to transmit time-series data from an edge side to a center side, and includes: a real-time data transmission module 10, a history data transmission module 20, a metadata storage and management module 30, and a time-ordered data processing platform 40.

The metadata storage and management module 30 is used for recording and managing metadata records generated by all operations of the system, the metadata storage and management module 30 comprises a measuring point metadata management sub-module 31, and the measuring point metadata management sub-module 31 is used for managing measuring point metadata in a source database and a target database 41 at the edge side, wherein the measuring point metadata at least comprises a measuring point ID and a value type of a measuring point; for example, the site metadata may be set to FQ_01_0GEW004MU_XQ01, FLOAT, where FQ_01_0GEW004MU_XQ01 is the site ID and FLOAT indicates that the value corresponding to the site is floating point.

The time sequence data processing platform 40 comprises a target database 41, wherein the target database 41 is used for receiving and storing real-time data and supplementary collected historical data respectively sent by the real-time data transmission module 10 and the historical data transmission module 20;

The real-time data transmission module 10 is configured to, after acquiring real-time data of an edge side, invoke the measurement point metadata management sub-module 31 to acquire a value type of a measurement point to be written in the real-time data, so as to write the corresponding real-time data into the target database 41 for storage; in this embodiment, the target database 41 is IotDB, and when real-time data is written into the target database 41, the metadata management submodule acquires the value type of the input measurement point, and then invokes the writing interface of IotDB to execute the data writing operation.

The historical data transmission module 20 is configured to generate a historical supplementary acquisition task and send the historical data based on the supplementary acquisition task to the target database 41 for storage, including: the historical data acquisition sub-module 21 and the historical data transmission sub-module 22 are arranged on the edge side, the historical data acquisition task generation sub-module 23 is arranged on the center side, the rotor module 24 in the historical data and the importing sub-module 25. The historical data acquisition task generating sub-module 23 comprises a historical data comparing unit, wherein the historical data comparing unit is used for comparing the historical data of the target database 41 and the historical data of the appointed database in the appointed time period, generating a difference record according to the comparison result of the historical data, and calling the metadata management module to acquire the ID of the point to be acquired based on the difference record, so as to generate a historical acquisition task; the historical data acquisition sub-module 21 is in communication connection with the historical acquisition task generation sub-module 23 and is used for acquiring corresponding historical data from a source database at the edge side according to the historical acquisition task issued by the historical acquisition task generation sub-module 23; the historical data transmission sub-module 22 is respectively in communication connection with the historical data acquisition sub-module 21 and the historical data middle rotor module 24, and is configured to receive the historical data sent by the historical data acquisition sub-module 21, and send the corresponding historical data to the historical data middle rotor module 24 after calling the metadata management sub-module 31 to obtain the value type of the measurement point to be written in the historical data, so as to transmit the historical data from the edge side to the center side: the rotor module 24 in the history data is provided with a transfer database, and the transfer database is used for receiving, storing and transferring the history data sent by the history data transmission sub-module 22; the importing sub-module 25 is communicatively connected to the rotor module 24 in the history data, and is configured to convert the history data in the transit database into a database format file and import the database format file into the target database 41.

According to the cloud-edge cooperation-based high-reliability time sequence data transmission and storage system, the problem of integrity of transmission and storage of near-real-time data and historical data of nuclear power can be effectively solved, wherein a historical data transmission module 20 is designed aiming at the problem of incomplete historical data stored on a center side, and the historical data transmission module 20 is responsible for supplementing and collecting historical data of a corresponding interval section from an edge side to replace original incomplete data on the center side after confirming the interval section with incomplete data on the center side; aiming at the characteristics of real-time performance and short-term caching of near-real-time data, a real-time data transmission module 10 is designed, and the probability of packet loss is reduced by improving the integrity guarantee of real-time data transmission; meanwhile, the metadata management module is designed for unified management of data standards in the data acquisition and sensing processes in the historical data transmission module 20 and the real-time transmission module.

The functions and the working flows of each module in the high-reliability time sequence data transmission and storage system based on cloud edge cooperation are respectively described below

Historical data transfer Module 20

In this embodiment, as shown in fig. 2, the history data transmission module 20 is configured to obtain the missing record and generate a task of complementary acquisition, so as to complementary acquire the missing data in the central side target database 41 from the edge side history database. The workflow of the historic data transmission module 20 is described below in connection with the historic transmission business process.

1. And the historical data comparison unit is used for comparing the integrity of the data at the central side and correspondingly generating a historical acquisition task.

The historical data collection task generating sub-module 23 mainly confirms whether the data in the continuous intervals of each measuring point in the target database 41 has the characteristics of large data volume and large time span, as described above, in order to improve the comparison efficiency, in this embodiment, when the historical data is collected and compared in a complementary manner, the historical data comparing unit divides the continuous collection time period to be longer into a plurality of relatively shorter time periods for integrity comparison respectively, and when each time period is compared in integrity, two modes of detailed comparison and simple comparison are further designed, so that a user can alternatively generate the corresponding historical data collection task according to actual needs.

When simple comparison is performed, the historical data comparison sub-module designates the database as the source database, and the historical data comparison unit is used for comparing the first time sequence data file with the source database after inquiring and deriving the first time sequence data file of the corresponding time period from the target database 41 according to the designated time period, wherein the first time sequence data file at least comprises a measuring point ID, a total collection time period and a total data number corresponding to the total collection time period.

Since the source database is disposed on the edge side, the target database 41 is disposed on the center side, the spatial geographic distance between the two databases is usually far, for example, the center side is located in one nuclear power station, the edge side is located in another nuclear power station, and the interaction efficiency between the center side and the edge side is relatively low due to factors such as the spatial distance between the two nuclear power stations and the network, so that when the data integrity is judged, the time granularity is compared relatively greatly, that is, for any measuring point in a designated time period, the first time data file in the corresponding time period is queried and derived from the target database 41, and when the total amount of source data corresponding to any acquired total time period in the first time data file is judged to be different from that in the source database, the historical data corresponding to the central side is judged to be incomplete.

As shown in table 1 below, in this embodiment, each total acquisition time period is in days, when data of a specified period from 11 months in 2019 to 1 to 30 days needs to be compared, the first time sequence data files are grouped according to "measurement point ID-time" (fq_01_0gew004mu_xq01-2019-11-01), where "FQ" represents a name of a nuclear power plant, "01" represents a number 1 nuclear power unit, and "0gew004mu_xq01" represents a serial number of a measurement point; for a measurement point, there are 30 sets of data in the first time data file, and each set includes the data amount of the measurement point in the day, for example: FQ_01_xxx012019-11-01 100, this period of comparison is completed when the 30 sets of data are compared. When comparing, the total amount of source data of each day is compared, if the source data amount of the corresponding date in the source database is the same as the value of the first time data file, the data of the corresponding time period in the target database 41 is judged to be complete, if the value of one day is different or the data of one day is missing, the data of the corresponding time period in the target database 41 is judged to be incomplete, a historical acquisition task is required to be generated, namely, a database difference record is obtained according to the result of data comparison, and a supplementary acquisition task of the difference record is generated. In addition, the acquisition task can be generated manually according to time and measuring points.

TABLE 1

When the detailed comparison is performed, in the historical data comparison submodule, the designated database is a transit database, and the historical data comparison unit is used for inquiring and deriving a second time sequence data file of a corresponding time period from the target database 41 according to the designated time period, and comparing the second time sequence data file with data of the designated time period in the transit database, wherein the second time sequence data file at least comprises a measuring point ID, a total acquisition time period, actual sampling time included in each total acquisition time period and sampling values corresponding to each actual sampling time;

The transfer database and the second time sequence data file are both arranged at the center side and are positioned at the same geographic position, and interaction efficiency between the two is high, so that when data integrity judgment is carried out, any measuring point in a designated time period can be compared in detail, namely, data comparison of actual sampling time is carried out on any measuring point in the designated time period, and when the fact that a value corresponding to each actual sampling time in the second time sequence data file is different from data corresponding to the transfer database is judged, the fact that the corresponding historical data is stored at the center side is incomplete is judged.

For example, as shown in the following table 2, assuming that the derived second time-series data file is a section of fq_01_0gew004mu_xq01 measured point 19 years and 10 months, when comparing, for any day of 10 months, all actual sampling times and corresponding values are compared, if the corresponding time and values in the transit database are all the same as the data stored in the transit database, the data of the corresponding time period in the target database 41 is determined to be complete, if there is a difference in one day, if the value of a certain actual sampling time is missing, the value of a certain sampling time is different, or the value of a certain day is missing, the data of the corresponding time period in the target database 41 is determined to be incomplete, and a history acquisition task needs to be generated, that is, a database difference record is obtained according to the result of data comparison, and a supplementary acquisition task of the difference record is generated. In addition, the acquisition task can be generated manually according to time and measuring points.

TABLE 2

2. And (3) task management:

the invention discloses a cloud edge collaboration-based high-reliability time sequence data transmission and storage system, which is provided with a task management unit, wherein the task management unit is in communication connection with the historical data comparison unit and is used for carrying out task arrangement management on the historical acquisition tasks and carrying out statistics and analysis on the number, information and running states of the final historical acquisition tasks generated after arrangement management.

3. Edge side historical dataset data:

in this embodiment, the edge-side historical data collection sub-module 21 uses the SDK or API interface of the data source system to obtain the latest data record of each measurement point, packages the collected data records according to a standard format, and then gives the data records to the historical transmission module.

Furthermore, as a preferred aspect, the history data transmission sub-module 22 further includes a first acquisition metadata generating unit for generating acquisition metadata based on each history data acquisition record; the acquisition metadata is used for representing metadata logs of the current supplementary acquisition, and at least comprises a measuring point ID, acquisition management time and the daily acquisition data quantity in the acquisition management time range; the first acquisition metadata generation unit correspondingly generates acquisition metadata based on each supplementary acquisition and sends the acquisition metadata to the metadata management module.

4. Edge side data compression, encryption, data file forwarding:

Considering that the nuclear power field has a scenario with a high data transmission security requirement, in this embodiment, the history data transmission module 20 further includes a history data preprocessing sub-module 26 disposed at an edge side, where the history data preprocessing sub-module 26 is communicatively connected to the history data transmission sub-module 22, and is used for compressing and/or encrypting the collected history data in the history data transmission sub-module 22. When meeting a scene with higher requirements on data transmission safety, the encryption option of the packed file can be started, and meanwhile, the packed file is compressed and then transmitted, so that the disk space is saved.

5. Center side data file transfer:

The rotor module 24 in the historical data forwards the data transmitted to the center side by the historical data transmission sub-module 22, and can be forwarded to a corresponding time sequence database for importing, and can also be forwarded to other servers for data analysis and comparison.

In addition, in this embodiment, the rotor module 24 in the history data further includes a transmission metadata generating unit, where the transmission metadata generating unit is configured to generate transmission metadata based on a transmission record of each time the history data is transmitted from the edge side to the center side, as shown in table 3, where the transmission metadata includes at least a task name, a transmission period, and a total number of folders and a total number of files transmitted in the transmission period; the transmission metadata generation unit records and generates transmission metadata and sends the transmission metadata to the metadata management module when data forwarding is performed each time.

TABLE 3 Table 3

6. Center side data import: the importing sub-module 25 converts the file forwarded to the time series data processing platform 40 into a database format file and imports the database format file into the target database 41;

7. center side historical data archiving:

Considering that the collection rate of the edge side is affected by various aspects, which results in slower collection rate and insufficient data storage space of the edge side, the historical data transmission module 20 performs buffer processing and historical archiving on the collected data to prevent the situation that the data is lost due to data writing or transmission failure caused by abnormal reasons. Therefore, in the high-reliability time-series data transmission and storage system based on cloud edge cooperation, the history transmission module further comprises a history data grading sub-module 27, the history data grading sub-module 27 is provided with an archiving database, and the archiving database is in communication connection with the transit database and is used for archiving and storing the confirmed history data of the transit database so as to upload data files received by the central side to the archiving server according to the classification of the date format. The archived file data is sorted according to the measuring points and the time periods, and the inquiry service of the historical data is provided, so that when the historical data is required to be inquired, the corresponding file can be directly found from the archived database without exporting the data from the target database.

Furthermore, in a preferred embodiment, the history data profiling sub-module 27 further includes an archive metadata generating unit, where the archive metadata generating unit is configured to generate corresponding archive metadata based on an archive record of each piece of history data in the relay database, as shown in table 4, where each piece of archive metadata includes at least an archive file name, an archive file size, an archive file path, an archive time, and an affiliated task; each time the history data archiving is performed, the history data archiving sub-module 27 generates corresponding archiving metadata based on the archiving conditions and sends the corresponding archiving metadata to the metadata storage and management module 30 for storage.

TABLE 4 Table 4

8. Historical data monitoring and alarming:

Considering the characteristics of large data volume and large time span of the history number, the data quality requirement in the continuous interval is higher. Meanwhile, because the transmission link is longer and the data flow links are more, in this embodiment, the historical data transmission module 20 further includes a data link reliability monitoring sub-module 28 disposed at the center side, where the data link reliability monitoring sub-module 28 has a function of monitoring and logging the transmission of the historical data, and specifically, the data link reliability monitoring sub-module 28 is used for performing link monitoring on the historical data acquisition sub-module 21, the historical transmission sub-module, the rotor module 24 in the historical data, the importing sub-module 25 and the historical data grading sub-module 27, and performing an abnormal condition alarm processing.

(Two) real-time data Transmission Module 10

The real-time transmission function module is responsible for forwarding the acquired data to an edge side database and forwarding the acquired data to a center side for storage. The characteristics of real-time performance and short-term buffering of near-real-time data are considered, so that the real-time transmission function module is required to ensure the integrity of real-time data transmission to reduce the probability of packet loss when the near-real-time data are processed, and meanwhile, after a small amount of data are lost, the transmission flow can be read again from the acquisition buffer area in time and retried. When encountering network problems, a retry mechanism, a monitoring mechanism for input data and an alarm function under abnormal conditions are needed, so that the problems that the system cannot automatically trace back errors after data transmission fails and the problem of data compensation can be solved.

As a preferred solution, in this embodiment, the real-time transmission function module includes: the real-time acquisition sub-module is arranged on the edge side and is used for acquiring real-time data based on the edge side real-time acquisition device; the real-time data transmission sub-module 12 is arranged at the edge side and is used for forwarding the acquired real-time data to the edge side database and the center side respectively; the real-time data access sub-module 13 is arranged at the center side and is used for receiving the real-time data forwarded by the real-time data transmission sub-module 12 in real time; the real-time data calculation and forwarding sub-module 14 is configured to screen, filter and additionally calculate edge service logic for the real-time data flow accessed by the real-time data access module, and output the data to the target database 41; an interrupt retry sub-module 15 for retry transmission when an external abnormality causes an interrupt; the missing data compensation sub-module 16 is used for the retransmission of the missing near real-time data.

The real-time acquisition sub-module 11, the real-time data transmission sub-module 12, the real-time data access sub-module 13 and the real-time data calculation and forwarding sub-module 14 are matched for acquiring and transmitting real-time data from the edge side to the center side under normal conditions, and when special conditions of a small amount of data loss caused by external factors such as a network are met, the transmission flow can be read again from the acquisition buffer area and retried in time by interrupting the matching of the retry sub-module 15 and the missing data compensation sub-module 16.

Furthermore, the real-time transmission function module of the present invention further comprises a monitoring alarm sub-module 17 for recording abnormal logs and displacement of data and alarming based on the abnormality. The system is arranged in such a way, when a network problem is encountered, the cooperation of the sub-module and the monitoring alarm sub-module is checked through the program running state, and when retry transmission is carried out, the problem that the system cannot automatically trace back errors and the problem of data compensation after data transmission failure are solved through a monitoring mechanism of input data and an alarm function under abnormal conditions.

By matching the modules, the real-time transmission function module realizes fault recovery and data recovery through monitoring of software services and hardware related to the transmission link, so that the problem of complete and reliable transmission of near-real-time sequence data of the nuclear power industry Internet edge layer is solved.

The real-time transmission business process is shown in fig. 2, and specifically comprises the following steps:

a) The edge side real-time collector collects data: in this embodiment, the real-time data acquisition module 11 uses the SDK or API interface of the data source system to obtain the latest data record of each measurement point, and hands the acquired record to the real-time forwarding module.

In addition, as a preferred solution, in this embodiment, the real-time data acquisition sub-module 11 further includes a second acquisition metadata generating unit, where the second acquisition metadata generating unit is configured to generate, based on each real-time data acquisition record, acquisition metadata, where the acquisition metadata includes at least a measurement point ID, an acquisition management time, and an amount of acquired data per day within the acquisition management time range; the second acquisition metadata generation unit has the same function as the first acquisition metadata generation unit, and correspondingly generates acquisition metadata based on each real-time acquisition and sends the acquisition metadata to the metadata management module, wherein the acquisition metadata records the metadata log acquired in real time.

B) Edge side data forwarding: in this embodiment, the real-time data transmission sub-module 12 stores the collected data records in an edge side database, and forwards the data records from the edge side to the center side.

C) Center side data reception: in this embodiment, the real-time data access submodule 13 at the center side uses the internet of things proxy cluster to receive the real-time forwarded data in real time, and forwards the data flow according to the situation.

D) Center side calculation, forwarding and storage: in this embodiment, the central side real-time data calculation and forwarding sub-module 14 performs filtering, filtering and additional calculation of edge service logic on the central side real-time data stream, and outputs the result to the target database 41.

In the process, when a network has a problem, the interrupt retry sub-module 15 performs multiple retry transmission on network fluctuation or abnormal conditions, and the missing data compensation sub-module 16 retransmits missing near real-time data; meanwhile, the monitoring alarm sub-module 17 monitors the real-time data volume and sends an alarm mail when abnormal judgment is made.

(III) time-series data processing platform 40

The time sequence data processing platform 40 is used for storing the collected real-time data and the historical data, a target database 41 is arranged in the time sequence data processing platform 40, and the target database 41 is used for receiving and storing the real-time data and the supplementary collected historical data respectively sent by the real-time data transmission module 10 and the historical data transmission module 20;

Furthermore, as a preferable scheme, the time-series data processing platform 40 further includes a storage metadata generation sub-module 42, where the storage metadata generation sub-module 42 is configured to generate storage metadata based on each record of real-time data and history data written into the target database 41, each storage metadata includes at least a measurement point name, a writing time period, and a writing data amount corresponding to the writing time period, where the writing data amount is used to characterize a total writing data amount of a measurement point stored on a center side on a specified date.

(IV) metadata management Module

The metadata management module is responsible for recording and managing metadata records generated by all operations, so as to be used for tracing processing and state analysis management and data management when problems are found later. The high-reliability time sequence data transmission and storage system based on cloud edge cooperation is characterized by large data storage quantity, large time span and higher data quality requirement in continuous intervals besides large data quantity, and meanwhile, the metadata storage and management module 30 is independently arranged to monitor and log the transmission work of real-time data and historical data due to the fact that transmission links are longer and data transfer links are more. The metadata storage and management module 30 is used for recording and managing metadata records generated by all operations of the system, on one hand, links with problems can be rapidly located through means of link monitoring and log tracing, so that the problem of complete and reliable migration of historical data is solved. On the other hand, the working state and the process of each link in the current task can be obtained based on various metadata.

In this embodiment, the metadata record includes collecting metadata, transmitting metadata, storing metadata and archiving metadata, covering each important circulation link involved in the system by time series data, the metadata storage and management module 30 corresponds to the real-time data transmission module 10, the history data transmission module 20 and the time series data processing platform 40, and the collecting metadata management sub-module 32, the transmitting metadata management sub-module 33, the archiving metadata management sub-module 34 and the storing metadata management sub-module 35 are respectively set to realize that various metadata are respectively saved by category and provide a reading interface for data analysis. Specifically, the acquisition metadata management sub-module 32 is respectively in communication connection with the historical data transmission sub-module 22 and the real-time data transmission sub-module 12, and is configured to receive and store acquisition metadata, where each acquisition metadata at least includes a measurement point name, an acquisition time period, and a corresponding acquisition data amount in the acquisition time period, and each acquisition data amount is used to characterize an acquisition data total amount of a measurement point specified date; the transmission metadata management sub-module 33 is communicatively connected to the rotor module 24 in the history data, and is configured to receive and store transmission metadata, where the transmission metadata is used to characterize data information recorded in a transmission process, and the transmission metadata includes at least a task name, a transmission period, and a total number of folders and a total number of files transmitted in the transmission period; the storage metadata management sub-module 35 is communicatively connected with the time-series data processing platform 40, and is configured to receive and store storage metadata, where each storage metadata at least includes a measurement point name, a writing time period, and a writing data amount corresponding to the writing time period, where the writing data amount is used to characterize a total writing data amount of a measurement point stored on a center side on a specified date; the archive metadata management sub-module 34 is communicatively connected to the history data profiling sub-module 27 for receiving and storing archive metadata, wherein each of said archive metadata comprises at least an archive file name, an archive file size, an archive file path, an archive time and an associated task.

When it is found that the quality problem exists in the historical data or the real-time data received by the target database 41, the metadata collection, the metadata transmission, the metadata storage and the metadata archiving corresponding to the specified measurement point in the specified time period can be called from the metadata storage and management module 30, and the data quantity among the four can be compared; if the data quantity of some metadata is not matched with that of other metadata, the links corresponding to the metadata are indicated to have problems, so that the problems can be rapidly positioned.

Further, although the collected data volume and the stored data volume of each measurement point in a specified period of time can be obtained based on the collected metadata and the stored metadata for subsequent analysis, as described above, the historical data has the characteristic of large data volume, and the collected metadata and the stored metadata obtained based on the historical data also have the problem that the data volume is also large, and when the collected data volume in the specified time and the specified measurement point range needs to be obtained for analysis, a large amount of time is required to be spent to find all the information. Meanwhile, in consideration of single-thread traversal and circulation during historical data collection, in the embodiment, each piece of collected metadata stores the collected data quantity of a designated measurement point for one year, wherein the designated year corresponds to all the collected data quantity and is sequentially stored in the collected metadata according to time sequence after being converted into data of a designated system. As shown in table 5, in this embodiment, for the measurement points named ZS 02 20zas-EP-ET101A-26, all the collected data amounts in 2019 are sequentially stored according to the chronological order, where the collected data amounts in adjacent days are separated by commas, and X represents no data in the day, and meanwhile, for further reducing the storage space, each collected data amount corresponding to a specified year is converted into 16-system data and then sequentially stored in the collected metadata according to the chronological order.

TABLE 5

For the same measuring point, the collecting metadata management sub-module stores collecting metadata by taking the year as a unit, namely, the data of the same measuring point in a certain year is put in the same file data, the collecting data quantity of the corresponding date is sequentially written in according to the time sequence, and 365 collecting data quantities are recorded in the corresponding file by taking 2019 as an example; meanwhile, in the process of data writing, the original acquired data quantity is converted into 6-bit 16-system data for recording. By the arrangement, on one hand, space can be effectively saved; on the other hand, because the collected metadata is stored year by year, the method is also more beneficial to data query and acquisition in the subsequent overall analysis.

In this embodiment, each piece of storage metadata stores a write-in data amount of a specified measurement point for one year, where the storage metadata includes a measurement point name, a year, and a write-in data amount per day, and the write-in data amount corresponding to the specified year is sequentially stored in the storage metadata according to a chronological order after being converted into 10-system data. As shown in table 6, in this embodiment, for the measurement point named ZS 02 20zas-EP-ET101A-26, the total written data amount in 2019 was also sequentially stored in the storage metadata according to time sequence after being converted into 10-system data, wherein the written data amounts in each day are separated by commas, and 0 represents no data.

TABLE 6

The metadata storage and management module 30 further includes a metadata processing sub-module 36, where the metadata processing sub-module 36 is in communication with the collection metadata management sub-module 32, the transmission metadata management sub-module 33, the storage metadata management sub-module 35, and the archive metadata management sub-module 34, respectively, to obtain various types of metadata and process the various types of metadata.

In the embodiment, obtaining various metadata and processing the various metadata comprises (1) problem link positioning analysis processing; (2) Periodically analyzing various metadata to obtain various states in the nuclear power data transmission operation process, and generating an acquisition report and a comparison report for display; (3) performing asset analysis based on the archive metadata.

Wherein (1) processing the metadata of each category to perform problem link localization comprises the following steps:

a, acquiring a time range and a measuring point range of data to be positioned;

(2) Processing the metadata of each type further includes performing a periodic analysis based on the metadata, the periodic analysis including a combination of one or more of:

(3) And performing asset analysis based on the archive metadata, wherein the asset analysis comprises obtaining a task name, obtaining an archive file path from the corresponding archive metadata based on the task name, and downloading the corresponding data from an archive database based on the archive file path to perform asset analysis.

According to the cloud-edge collaboration-based high-reliability time sequence data transmission and storage system, acquisition metadata, transmission metadata, storage metadata and archiving metadata are respectively set for the whole flow from acquisition of mass nuclear power data at the edge side to storage at the center side, different data attribute information and storage structures are defined for different metadata, and various metadata are processed, so that on one hand, a link with problems can be rapidly positioned, and the problem of complete and reliable migration of historical data is solved; meanwhile, each state in the nuclear power data transmission operation process can be acquired and displayed through periodical analysis of various metadata. Specifically, for each link in the high reliability transmission process, when it is required to determine that the historical data of a certain section is incomplete, acquiring the acquisition metadata, the transmission metadata, the storage metadata and the archiving metadata of the corresponding section, and then comparing the records of each metadata, if one record is different from the other metadata, it can be determined that the possibility of occurrence of a problem of the link is the greatest. Meanwhile, the process of each link of the system can be acquired by further counting the metadata of different types according to the needs, and each state in the nuclear power data transmission operation process can be mastered in time.

The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be encompassed by the present invention.

Claims

1. The high-reliability time sequence data transmission and storage system based on cloud edge cooperation is characterized by comprising the following components: the system comprises a real-time data transmission module, a historical data transmission module, a metadata storage and management module and a time-ordered data processing platform;

The historical data transmission submodule is respectively in communication connection with the historical data acquisition submodule and the historical data middle rotor submodule, and is used for receiving the historical data sent by the historical data acquisition submodule and calling the measuring point metadata management submodule to acquire the value type of a measuring point to be written in the historical data, and then sending the corresponding historical data to the historical data middle rotor module:

the importing sub-module is in communication connection with the historical data transfer sub-module and is used for converting the historical data in the transfer database into a database format file and importing the database format file into a target database;

the historical data comparison unit is used for comparing a first time sequence data file of a corresponding time period with the source database after inquiring and deriving the first time sequence data file of the corresponding time period from the target database according to a specified time period, wherein the first time sequence data file at least comprises a measuring point ID, a total collection time period and a total data number corresponding to the total collection time period;

Or, in the historical data collection task generating sub-module, the specified database is a transit database, and the historical data comparing unit is used for comparing a second time sequence data file of a corresponding time period with the transit database after inquiring and deriving the second time sequence data file from the target database according to a specified time period, wherein the second time sequence data file at least comprises a measuring point ID, a collection total time period, actual sampling time included in each collection total time period and a sampling value corresponding to each actual sampling time;

The historical data transmission module further comprises a data link reliability monitoring submodule arranged on the center side, and the data link reliability monitoring submodule is used for carrying out link monitoring on the historical data acquisition submodule, the historical data transmission submodule, the historical data middle rotor module and the guiding submodule and carrying out abnormal condition alarm processing.

2. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 1, wherein the historical data acquisition task generation sub-module further comprises a task management unit, wherein the task management unit is in communication connection with the historical data comparison unit and is used for carrying out arrangement management on the historical acquisition tasks and carrying out statistics and analysis on the number, information and running states of the final historical acquisition tasks generated after arrangement management.

3. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 2, wherein the rotor module in the historical data further comprises a transmission metadata generation unit, the transmission metadata generation unit is used for generating transmission metadata based on a transmission record of each time the historical data is transmitted from an edge side to a center side, the transmission metadata is used for representing data information of a transmission process record, and the transmission metadata at least comprises task names, transmission periods, and total number of folders and total number of files transmitted in the transmission periods;

The metadata storage and management module is correspondingly provided with a transmission metadata management sub-module, and the transmission metadata management sub-module is in communication connection with the rotor module in the historical data and is used for receiving and storing the transmission metadata.

4. The cloud edge collaboration-based high-reliability time-series data transmission and storage system according to claim 3, wherein the time-series data processing platform further comprises a storage metadata generation submodule, wherein the storage metadata generation submodule is used for generating storage metadata based on each write record of real-time data and historical data written into a target database, each storage metadata at least comprises a measuring point name, a writing time period and a writing data volume corresponding to the writing time period, and the writing data volume is used for representing the total writing data volume of a measuring point stored on a center side on a specified date; the metadata storage and management module is correspondingly provided with a storage metadata management sub-module, and the storage metadata management sub-module is in communication connection with the time sequence data processing platform and is used for receiving and storing the storage metadata.

5. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system is characterized in that each piece of storage metadata stores one-year written data volume of a designated measuring point, wherein the written data volume corresponding to the designated year is sequentially stored in the storage metadata according to time sequence after being converted into data of a designated system.

6. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 4, wherein the history transmission module further comprises a history data grading sub-module, the history data grading sub-module is provided with an archiving database, and the archiving database is in communication connection with the transit database and is used for archiving and storing history data of the transit database.

7. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 6, wherein the history data profiling sub-module further comprises an archiving metadata generation unit, wherein the archiving metadata generation unit is used for generating corresponding archiving metadata based on archiving records of each piece of history data in the transit database, and each piece of archiving metadata at least comprises an archiving file name, an archiving file size, an archiving storage path, archiving time and a task to which the archiving metadata belongs;

the metadata storage and management module is correspondingly provided with an archiving metadata management sub-module, and the archiving metadata management sub-module is in communication connection with the historical data archiving sub-module and is used for receiving and storing each piece of archiving metadata.

8. The cloud edge collaboration-based high-reliability time-series data transmission and storage system according to claim 7, wherein the historical data transmission module further comprises a historical data preprocessing sub-module arranged on the edge side, and the historical data preprocessing sub-module is in communication connection with the historical data transmission sub-module and is used for compressing and/or encrypting historical data in the historical data transmission sub-module.

9. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system as claimed in claim 7, wherein the real-time transmission function module comprises:

10. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 9, wherein the historical data acquisition sub-module further comprises a first acquisition metadata generation unit for generating acquisition metadata based on each historical data acquisition record; the real-time data acquisition sub-module further comprises a second acquisition metadata generation unit, wherein the second acquisition metadata generation unit is used for generating acquisition metadata based on each real-time data acquisition record, each acquisition metadata at least comprises a measuring point name, an acquisition time period and corresponding acquisition data quantity in the acquisition time period, and each acquisition data quantity is used for representing the total amount of acquisition data of a specified date of a measuring point;

The metadata storage and management module is correspondingly provided with an acquisition metadata management submodule, and the acquisition metadata management submodule is respectively in communication connection with the historical data transmission submodule and the real-time data transmission submodule and is used for receiving and storing the acquisition metadata.

11. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 10, wherein each piece of acquisition metadata stores the acquired data quantity of a designated measurement point for one year, and all the acquired data quantity corresponding to the designated year is sequentially stored in the acquisition metadata according to time sequence after being converted into data of a designated system.

12. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 10, wherein the metadata storage and management module further comprises a metadata processing sub-module, and the metadata processing sub-module is respectively in communication connection with the acquisition metadata management sub-module, the transmission metadata management sub-module, the storage metadata management sub-module and the archiving metadata management sub-module, so as to acquire the acquisition metadata, the transmission metadata, the storage metadata and the archiving metadata, and then process various metadata.

13. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system of claim 12, wherein the processing of the various metadata after acquiring the acquisition metadata, transmitting the metadata, storing the metadata and archiving the metadata comprises:

a, acquiring a time range and a measuring point range of data to be positioned;

C1 if they are equal to each other

c13, comparing the total amount of the collected files, the total amount of the stored files and the amount of the files, and judging that the corresponding links have problems if one group of the files is different in number;

or (b)

C12' respectively finding out the corresponding measuring point names and the number of folders from the acquisition metadata, the transmission metadata and the storage metadata for each measuring point in the measuring point range;

c14' comparing the total amount of the collected folders, the total amount of the stored folders and the file amount, and judging that the corresponding link has a problem if one group of the total amount of the stored folders and the file amount are different;

14. The cloud-edge collaboration-based high-reliability time-series data transmission and storage system according to claim 9, wherein the real-time data transmission module further comprises a monitoring alarm sub-module, and the monitoring alarm sub-module is used for recording abnormal logs and displacement of data and alarming based on the abnormalities.

15. The cloud edge collaboration-based high-reliability time sequence data transmission and storage system according to claim 1, wherein the system is used for transmission and storage of nuclear power time sequence data.