CN112910606B - Data sending system adopting sectional type data structure - Google Patents
Data sending system adopting sectional type data structure Download PDFInfo
- Publication number
- CN112910606B CN112910606B CN202110287268.8A CN202110287268A CN112910606B CN 112910606 B CN112910606 B CN 112910606B CN 202110287268 A CN202110287268 A CN 202110287268A CN 112910606 B CN112910606 B CN 112910606B
- Authority
- CN
- China
- Prior art keywords
- data
- segment
- sending
- different
- sent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0006—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission format
- H04L1/0007—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission format by modifying the frame length
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0015—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the adaptation strategy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0023—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the signalling
- H04L1/0028—Formatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0033—Systems modifying transmission characteristics according to link quality, e.g. power backoff arrangements specific to the transmitter
Abstract
The invention discloses a data transmission system adopting a sectional data structure, which comprises a data real-time scanning module to be transmitted, different data stream segment dividing modules, a data configuration object transmission process object query module, a sectional data transmission speed synchronous acquisition module and a data integration module, wherein the data real-time scanning module to be transmitted is used for scanning a data stream to be transmitted, the different data stream segment dividing modules are used for determining a sectional mode for the data according to basic information, the data configuration object transmission process object query module is used for querying the content of the data positioned to each section, the sectional data transmission speed synchronous acquisition module is used for monitoring the transmission of the data of each section, the data integration module is used for integrating the data of the section in real time after the data transmission is finished and aiming at segmenting the data with larger data quantity when the data is transmitted, the user can conveniently check the downloaded data segments in real time.
Description
Technical Field
The invention relates to the field of data transmission, in particular to a data transmission system adopting a sectional data structure.
Background
The data transmission system is widely applied to various fields such as industrial control and the like, and how to improve the stability of acquisition software and the data transmission efficiency becomes a technical difficulty. The Transmission Control Protocol (TCP) is the most widely used transport layer protocol in the Internet. Common applications such as HTTP, FTP, BitTorrent, etc. use it to transfer data. During transmission, data transmission is limited by more than one factor, such as the sender, receiver, and network. The data transmission timer counts the transmitted data in a timing manner, and is a down counter, when the count value becomes 0, the data still does not receive a corresponding response, which indicates that the transmitted data exceeds the timing range and needs to be retransmitted or discarded.
During transmission, data transmission is limited by more than one factor, such as the sender, receiver, and network. When users are not fast in data transmission rate, they want to know the main factors that limit data transmission during transmission. Once a limiting factor in the data transmission can be diagnosed, the user can increase the transmission rate of the connection by removing such a limiting factor. For example, when the limiting factor is the receiving party, the user can improve the transmission rate by modifying the configuration of the host; and when the limiting factor is the sender or the network, the user can solve the problem by reflecting to the service provider and the ISP.
At present, when data is sent, data is often packed and then sent in a centralized mode, so that the data can be sent slowly under the condition of data volume, the data sent in the centralized mode needs to be consulted uniformly after the data is sent, waiting time is long, the data sending method and the device aim to segment the data with large data volume when the data is sent, the sending speed of the data is accelerated, and a user can conveniently check the downloaded data fragments in real time.
Disclosure of Invention
The present invention is directed to a data transmission system and method using a segmented data structure, so as to solve the problems in the prior art.
In order to achieve the purpose, the invention provides the following technical scheme:
a data transmission system adopting a sectional data structure comprises a data to be transmitted real-time scanning module, different data stream segment dividing modules, a data configuration object transmission process object query module, a segmented data transmission speed synchronous acquisition module and a data integration module, wherein the data to be transmitted real-time scanning module, the different data stream segment dividing modules and the data integration module are sequentially connected through an intranet;
the real-time scanning module of the data to be sent is used for scanning the data stream to be sent and judging different data formats of the sent data, the different data stream segment dividing module is used for determining the segmentation mode of the data according to basic information, the data configuration object sending process object query module is used for querying the content of each segment positioned by the data, the segment data sending speed synchronous acquisition module is used for monitoring the sending of the data of each segment, and the data integration module is used for integrating the data after the sending of the segment data is completed in real time.
By adopting the technical scheme: the real-time scanning module for the data to be sent comprises a data sorting set submodule and different data stream format analysis matching submodules, the data sorting set submodule is used for integrating the data to be sent and sending the integrated data to the different data stream format analysis matching submodules, and the different data stream format analysis matching submodules are used for analyzing the data format of the data to be sent and judging the data format of the different data and the data quantity in the different data formats.
By adopting the technical scheme: the different data stream segment dividing module comprises a data basic information analysis submodule and a data segmented scheme confirmation submodule, the data basic information analysis submodule is used for obtaining data formats of different data stream format analysis matching submodule detection data and counting formats of different data, the data segmented scheme confirmation submodule is used for confirming a data segmented scheme according to data format types analyzed by the data basic information analysis submodule, when the format types of data to be sent are smaller than or equal to a set type, the data format types to be sent are sent to the data segmented scheme confirmation submodule for data segmentation, the data segmented total data quantity of the data is counted by the data segmented scheme confirmation submodule, and the data to be sent are sent in a segmented mode according to the same data quantity of each single segment.
By adopting the technical scheme: the data basic information analysis submodule acquires data formats of data detected by different data stream format analysis matching submodules, acquires data quantity in different data formats detected by the different data stream format analysis matching submodules when the format type of the data to be transmitted is larger than a set type, analyzes the data quantity in the current data format, and sets the data format to be segmented into L sections according to different data formats 1 、L 2 、L 3 、…、L n-1 、L n Wherein the data amount of a single segment in the different segments is R 1 、R 2 、R 3 、…、R n-1 、R n Setting the data amount of the single segment not to be lower than R0, and setting the data amount in the single segment to satisfy the following formula:
when the data quantity in the data single section to be sent does not meet the formula, the data sectional type scheme confirmation submodule counts the total data quantity of the data to be sent, and carries out sectional sending on the data to be sent according to different data format types.
By adopting the technical scheme: the data configuration object sending process query module comprises a residual sending data subsection query submodule and a data sending completion information presentation submodule, wherein the residual sending data subsection query submodule is used for querying the sending process of subsection data in real time in different subsection data sending processes and accurately mastering different subsection sending contents, and the data sending completion information presentation submodule is used for presenting sending content information of the data of different subsections after the sending of the data of different subsections is completed and viewing the content of the data of different subsections which are sent in real time.
By adopting the technical scheme: the synchronous acquisition module for the sending speed of the segmented data comprises a single-segment data sending speed statistic submodule and a different segmented data sending speed information analysis submodule, wherein the single-segment data sending speed statistic submodule is used for counting the sending speed of different segmented data, the counted sending speed is matched with the single-segment data to be made into a list and sent to the different segmented data sending speed information analysis submodule, the different segmented data sending speed information analysis submodule is used for analyzing the sending speed of the single-segment data, when the sending speed of the single-segment is smaller than a set threshold value, and the data segmented scheme confirmation submodule sends the data to be sent in a segmented mode according to the same data quantity of each single-segment, the single-segment data of which the sending speed is smaller than the set threshold value is re-segmented according to the same data quantity of each segment, and when the sending speed of the single-segment is smaller than the set threshold value, and the data segmented scheme confirmation submodule divides the data to be sent according to different data format types And (5) segment sending, namely extracting the data format of the single segment with the current sending speed less than the set threshold value, and carrying out segment-by-segment sending again according to the same data quantity of each segment.
By adopting the technical scheme: the data integration module comprises a multi-segment data integration sub-module and a data release sub-module, the multi-segment data integration sub-module is used for integrating the transmitted multi-segment data, and the data release sub-module is used for integrating and releasing the data which is successfully transmitted.
A data transmission method using a segmented data structure:
s1: scanning a data stream to be transmitted by using a data real-time scanning module to be transmitted, judging different data formats of the transmitted data, integrating the data to be transmitted by using a data sorting set submodule, transmitting the integrated data to different data stream format analysis matching submodules, analyzing the data formats of the data to be transmitted by using the different data stream format analysis matching submodules, and judging the data formats of the different data and the data quantity in the different data formats;
s2: determining a segmentation mode of the data according to basic information by using different data stream segment dividing modules, acquiring data formats of different data stream format analysis matching sub-module detection data by using a data basic information analysis sub-module, counting the formats of different data, and determining a data segmentation scheme by using a data segmentation scheme determining sub-module according to the data format types analyzed by a data basic information analysis sub-module;
s3: the data configuration object sending process object query module is used for querying the content of each subsection positioned by the data, the remaining sent data subsection query submodule queries the sending process of the subsection data in real time in the sending process of different subsection data, the sending content of different subsections is accurately mastered, the data sending completion information presentation submodule presents the sending content information of the data of different subsections in time after the sending of the data of different subsections is completed, and the sent content of the data of different subsections is checked in real time;
s4: monitoring the sending of each segmented data by utilizing a segmented data sending speed synchronous acquisition module, counting the sending speeds of different segmented data by a single-segment data sending speed counting submodule, matching the counted sending speeds with the single-segment data to form a list and sending the list to different segmented data sending speed information analysis submodules, analyzing the sending speeds of the single-segment data by the different segmented data sending speed information analysis submodules, when the single-segment sending speed is less than a set threshold value, and a data segmented scheme confirmation submodule sends the data to be sent in a segmented mode according to the same data quantity of each single segment, re-segmenting the single segment of which the sending speed is less than the set threshold value according to the same data quantity of each segment, and when the single-segment sending speed is less than the set threshold value, and sending the data to be sent in a segmented mode according to different data format types by the data segmented scheme confirmation submodule, extracting the data format of a single segment with the current sending speed less than a set threshold value, and sending the data in segments again according to the same data quantity of each segment;
s5: and the data integration module is used for integrating the sent data in real time, the multi-segment data integration submodule is used for integrating the sent data, and the data release submodule is used for integrating the successfully sent data and then releasing the successfully sent data.
By adopting the technical scheme: in step S2, the method for determining the segmentation of the data according to the basic information by using different data stream segment segmentation modules, the data basic information analysis submodule obtaining the data formats of the data detected by the different data stream format analysis matching submodule, and counting the formats of the different data, and the data segmentation scheme confirmation submodule confirming the data segmentation scheme according to the data format types analyzed by the data basic information analysis submodule, further includes the following steps:
a1: acquiring data formats of detection data of different data stream format analysis matching sub-modules by using a data basic information analysis sub-module, counting the formats of different data, sending the counted data format types to a data sectional type scheme confirmation sub-module, and monitoring the format types of the data to be sent by using the data sectional type scheme confirmation sub-module;
a2: when the format type of the data to be sent is less than or equal to the set type, sending the format type of the data to be sent to a data sectional type scheme confirmation submodule for data segmentation, counting the total data volume of the data by the data sectional type scheme confirmation submodule, and sending the data to be sent in a sectional manner according to the same data volume of each single section;
a3: when the format type of the data to be sent is larger than the set type, acquiring the data quantity in different data formats detected by different data stream format analysis matching sub-modules, analyzing the data quantity in the current data format, and setting the data quantity to be L in a segmented mode according to different data formats 1 、L 2 、L 3 、…、L n-1 、L n Wherein the data amount of a single segment in the different segments is R 1 、R 2 、R 3 、…、R n-1 、R n Setting the data amount of the single segment not to be lower than R0, and setting the data amount in the single segment to satisfy the following formula:
By adopting the technical scheme: in step S4, the single segment data transmission speed statistics submodule counts the transmission speeds of different segment data, matches the counted transmission speed with the single segment data, and then makes a list to be transmitted to the different segment data transmission speed information analysis submodule, and the different segment data transmission speed information analysis submodule analyzes the transmission speed of the single segment data, and the method further includes the following steps:
the differently set segment data is L 1 、L 2 、L 3 、…、L n-1 、L n Extracting some segment data L n Monitoring current segmented data real-time transmissionSending speed, wherein the current segment data L is set n Has a real-time transmission speed of V 1 、V 2 、V 3 、…、V n-1 、V n Monitoring the sending speed of the sent segmented data to be V0 by using a single-segment data sending speed statistic submodule 1 、V0 2 、V0 3 、…、V0 n-1 、V0 n Wherein a certain segment data L is set n The real-time transmission speed of (2) satisfies the following formula:
when the data L is segmented n The real-time sending speed of the data transmission sub-module meets the formula, the different segment data sending speed information analysis sub-modules judge that the real-time sending speed of the current segment is normal, the current segment data is not processed, and when the segment data L is n Does not satisfy the above formula, segment the data L n The real-time sending speed is compared with the set threshold value when the data L is segmented n The real-time sending speed is less than the set threshold, and the data sectional scheme confirmation submodule sends the data to be sent in a sectional way according to the same data quantity of each single section, and transmits sectional data L n Re-segmenting according to the same data amount of each segment, and when the data L is segmented, re-segmenting n The real-time sending speed is less than the set threshold, and the data sectional type scheme confirmation submodule sends the data to be sent in a sectional way according to different data format types and extracts the current sectional data L n After the data format is marked, the data format is sent again in a subsection sending mode according to the same data quantity of each section, and the marked data format is sent to a data subsection scheme confirming submodule to be referred.
Compared with the prior art, the invention has the beneficial effects that: the invention aims to segment the data with larger data volume when sending, so as to accelerate the sending speed of the data and facilitate the user to check the downloaded data segments in real time;
the data transmission method comprises the steps that a data to be transmitted real-time scanning module is used for scanning data streams to be transmitted and judging different data formats of the transmitted data, different data stream segment dividing modules are used for determining segmentation modes of the data according to basic information, a data configuration object transmission process object query module is used for querying the content of each segment positioned by the data, a segment data transmission speed synchronous acquisition module is used for monitoring the transmission of the data of each segment, and a data integration module is used for integrating the data after the data transmission is completed in real time.
Drawings
In order that the present invention may be more readily and clearly understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings.
FIG. 1 is a block diagram of a data transmission system using a segmented data structure according to the present invention;
FIG. 2 is a schematic diagram illustrating steps of a data transmission method using a segmented data structure according to the present invention;
FIG. 3 is a diagram illustrating specific steps S2 of a data transmission system using a segmented data structure according to the present invention;
FIG. 4 is a schematic diagram of an embodiment of a data transmission method using a segmented data structure according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 to 4, in the embodiment of the present invention, a data transmission system and method using a segmented data structure includes a to-be-transmitted data real-time scanning module, a different data stream segment dividing module, a data configuration object transmission process object query module, a segmented data transmission speed synchronization acquisition module, and a data integration module, where the to-be-transmitted data real-time scanning module, the different data stream segment dividing module, and the data integration module are sequentially connected through an intranet, and the data configuration object transmission process object query module, the segmented data transmission speed synchronization acquisition module are respectively connected with the different data stream segment dividing module through the intranet;
the real-time scanning module of the data to be sent is used for scanning the data stream to be sent and judging different data formats of the sent data, the different data stream segment dividing module is used for determining a segmentation mode of the data according to basic information, the data configuration object sending process object query module is used for querying the content of each segment positioned by the data, the segment data sending speed synchronous acquisition module is used for monitoring the sending of the data of each segment, and the data integration module is used for integrating the data of the segments in real time after the sending of the data is completed.
By adopting the technical scheme: the real-time scanning module for the data to be sent comprises a data sorting set submodule and different data stream format analysis matching submodules, the data sorting set submodule is used for integrating the data to be sent and sending the integrated data to the different data stream format analysis matching submodules, and the different data stream format analysis matching submodules are used for analyzing the data format of the data to be sent and judging the data format of the different data and the data quantity in the different data formats.
By adopting the technical scheme: the different data stream segment dividing module comprises a data basic information analysis submodule and a data segmented scheme confirmation submodule, the data basic information analysis submodule is used for obtaining data formats of different data stream format analysis matching submodule detection data and counting formats of different data, the data segmented scheme confirmation submodule is used for confirming a data segmented scheme according to data format types analyzed by the data basic information analysis submodule, when the format types of data to be sent are smaller than or equal to a set type, the data format types to be sent are sent to the data segmented scheme confirmation submodule for data segmentation, the data segmented total data quantity of the data is counted by the data segmented scheme confirmation submodule, and the data to be sent are sent in a segmented mode according to the same data quantity of each single segment.
By adopting the technical scheme: the data basic information analysis submodule acquires data formats of data detected by different data stream format analysis matching submodules, acquires data quantity in different data formats detected by the different data stream format analysis matching submodules when the format type of the data to be transmitted is larger than a set type, analyzes the data quantity in the current data format, and sets the data format to be segmented into L sections according to different data formats 1 、L 2 、L 3 、…、L n-1 、L n Wherein the data amount of a single segment in the different segments is R 1 、R 2 、R 3 、…、R n-1 、R n Setting the data amount of the single segment not to be lower than R0, and setting the data amount in the single segment to satisfy the following formula:
when the data quantity in the data single section to be sent does not meet the formula, the data sectional type scheme confirmation submodule counts the total data quantity of the data to be sent, and carries out sectional sending on the data to be sent according to different data format types.
By adopting the technical scheme: the data configuration object sending process query module comprises a residual sending data subsection query submodule and a data sending completion information presentation submodule, wherein the residual sending data subsection query submodule is used for querying the sending process of subsection data in real time in different subsection data sending processes and accurately mastering different subsection sending contents, and the data sending completion information presentation submodule is used for presenting sending content information of the data of different subsections after the sending of the data of different subsections is completed and viewing the content of the data of different subsections which are sent in real time.
By adopting the technical scheme: the synchronous acquisition module for the sending speed of the segmented data comprises a single-segment data sending speed statistic submodule and a different segmented data sending speed information analysis submodule, wherein the single-segment data sending speed statistic submodule is used for counting the sending speed of different segmented data, the counted sending speed is matched with the single-segment data to be made into a list and sent to the different segmented data sending speed information analysis submodule, the different segmented data sending speed information analysis submodule is used for analyzing the sending speed of the single-segment data, when the sending speed of the single-segment is smaller than a set threshold value, and the data segmented scheme confirmation submodule sends the data to be sent in a segmented mode according to the same data quantity of each single-segment, the single-segment data of which the sending speed is smaller than the set threshold value is re-segmented according to the same data quantity of each segment, and when the sending speed of the single-segment is smaller than the set threshold value, and the data segmented scheme confirmation submodule divides the data to be sent according to different data format types And (5) segment sending, namely extracting the data format of the single segment with the current sending speed less than the set threshold value, and carrying out segment-by-segment sending again according to the same data quantity of each segment.
By adopting the technical scheme: the data integration module comprises a multi-segment data integration sub-module and a data release sub-module, the multi-segment data integration sub-module is used for integrating the transmitted multi-segment data, and the data release sub-module is used for integrating and releasing the data which is successfully transmitted.
A data transmission method using a segmented data structure:
s1: the real-time scanning module of the data to be sent is utilized to scan the data stream to be sent, different data formats of the sent data are judged, the data sorting set submodule integrates the data to be sent, the integrated data are sent to the different data stream format analysis matching submodule, the different data stream format analysis matching submodule analyzes the data format of the data to be sent, and the data formats of the different data and the data quantity in the different data formats are judged;
s2: determining a segmentation mode of the data according to basic information by using different data stream segment dividing modules, acquiring data formats of different data stream format analysis matching sub-module detection data by using a data basic information analysis sub-module, counting the formats of different data, and determining a data segmentation scheme by using a data segmentation scheme determining sub-module according to the data format types analyzed by a data basic information analysis sub-module;
s3: the data configuration object sending process object query module is used for querying the content of each subsection positioned by the data, the remaining sent data subsection query submodule queries the sending process of the subsection data in real time in the sending process of different subsection data, the sending content of different subsections is accurately mastered, the data sending completion information presentation submodule presents the sending content information of the data of different subsections in time after the sending of the data of different subsections is completed, and the sent content of the data of different subsections is checked in real time;
s4: monitoring the sending of each segmented data by utilizing a segmented data sending speed synchronous acquisition module, counting the sending speeds of different segmented data by a single-segment data sending speed counting submodule, matching the counted sending speeds with the single-segment data to form a list and sending the list to different segmented data sending speed information analysis submodules, analyzing the sending speeds of the single-segment data by the different segmented data sending speed information analysis submodules, when the single-segment sending speed is less than a set threshold value, and a data segmented scheme confirmation submodule sends the data to be sent in a segmented mode according to the same data quantity of each single segment, re-segmenting the single segment of which the sending speed is less than the set threshold value according to the same data quantity of each segment, and when the single-segment sending speed is less than the set threshold value, and sending the data to be sent in a segmented mode according to different data format types by the data segmented scheme confirmation submodule, extracting the data format of a single segment with the current sending speed less than a set threshold value, and sending the data in segments again according to the same data quantity of each segment;
s5: and the data integration module is used for integrating the sent data in real time, the multi-segment data integration submodule is used for integrating the sent data, and the data release submodule is used for integrating the successfully sent data and then releasing the successfully sent data.
By adopting the technical scheme: in step S2, the method for determining the segmentation of the data according to the basic information by using different data stream segment segmentation modules, the data basic information analysis submodule obtaining the data formats of the data detected by the different data stream format analysis matching submodule, and counting the formats of the different data, and the data segmentation scheme confirmation submodule confirming the data segmentation scheme according to the data format types analyzed by the data basic information analysis submodule, further includes the following steps:
a1: acquiring data formats of detection data of different data stream format analysis matching sub-modules by using a data basic information analysis sub-module, counting the formats of different data, sending the counted data format types to a data sectional type scheme confirmation sub-module, and monitoring the format types of the data to be sent by using the data sectional type scheme confirmation sub-module;
a2: when the format type of the data to be sent is less than or equal to the set type, sending the format type of the data to be sent to a data sectional type scheme confirmation submodule for data segmentation, counting the total data volume of the data by the data sectional type scheme confirmation submodule, and sending the data to be sent in a sectional manner according to the same data volume of each single section;
a3: when the format type of the data to be sent is larger than the set type, acquiring the data quantity in different data formats detected by different data stream format analysis matching sub-modules, analyzing the data quantity in the current data format, and setting the data quantity to be L in a segmented mode according to different data formats 1 、L 2 、L 3 、…、L n-1 、L n Wherein the data amount of a single segment in the different segments is R 1 、R 2 、R 3 、…、R n-1 、R n Setting the data amount of the single segment not to be lower than R0, and setting the data amount in the single segment to satisfy the following formula:
By adopting the technical scheme: in step S4, the single segment data transmission speed statistics submodule counts the transmission speeds of different segment data, matches the counted transmission speed with the single segment data, and then makes a list to be transmitted to the different segment data transmission speed information analysis submodule, and the different segment data transmission speed information analysis submodule analyzes the transmission speed of the single segment data, and the method further includes the following steps:
the differently set segment data is L 1 、L 2 、L 3 、…、L n-1 、L n Extracting some segment data L n Monitoring the real-time sending speed of the current segment data, wherein the current segment data L is set n Has a real-time transmission speed of V 1 、V 2 、V 3 、…、V n-1 、V n Monitoring the sending speed of the sent segmented data to be V0 by using a single-segment data sending speed statistic submodule 1 、V0 2 、V0 3 、…、V0 n-1 、V0 n Wherein a certain segment data L is set n The real-time transmission speed of (2) satisfies the following formula:
when the data L is segmented n The real-time sending speed of the data transmission sub-module meets the formula, the different segment data sending speed information analysis sub-modules judge that the real-time sending speed of the current segment is normal, the current segment data is not processed, and when the segment data L is n Does not satisfy the above formula, segment the data L n The real-time sending speed is compared with the set threshold value when the data L is segmented n Has low real-time transmission speedSetting a threshold value, and sending the data to be sent by the data sectional scheme confirmation submodule in a sectional mode according to the same data quantity of each single section, and sending sectional data L n Re-segmenting according to the same data amount of each segment, and when the data L is segmented, segmenting the data L n The real-time sending speed is less than the set threshold, and the data sectional type scheme confirmation submodule sends the data to be sent in a sectional way according to different data format types and extracts the current sectional data L n After the data format is marked, the data format is sent in a subsection sending mode again according to the same data quantity of each section, and the marked data format is sent to a data subsection scheme confirming submodule to be referred to.
Example 1: the basic data information analysis submodule acquires data formats of data detected by different data stream format analysis matching submodules under the limited condition, acquires data quantity in different data formats detected by the different data stream format analysis matching submodules when the format type of the data to be transmitted is larger than a set type, analyzes the data quantity in the current data format, and sets the data format to be segmented into L sections according to different data formats 1 、L 2 、L 3 、L 4 Wherein, the data amount of the single segment in the different segments is 120M, 98M, 131M, 117M, the data amount of the single segment is not less than 60M, and the data amount in the single segment is set to satisfy the following formula:
and when the data quantity in the single segment of the data to be sent meets the formula, sending the analysis data to a data sectional type scheme confirmation submodule, counting the data format type of the data to be sent by the data sectional type scheme confirmation submodule, and sending the data to be sent in a sectional mode according to different data format types.
Example 2: the basic data information analysis submodule acquires the data formats of the detection data of different data stream format analysis matching submodules under the limiting condition, and when the format type of the data to be sent is larger than the set type, the basic data information analysis submodule acquires the detection data of the different data stream format analysis matching submodulesMeasuring data amount in different data formats, analyzing data amount in current data format, and setting L as segment according to different data formats 1 、L 2 、L 3 、L 4 Wherein, the data volume of the single segment in the different segments is 380M, 71M, 878M, 129M, the data volume of the single segment is not less than 60M, and the data volume in the single segment is set to satisfy the following formula:
and when the data quantity in the single segment of the data to be sent does not meet the formula, the data sectional scheme confirmation submodule counts the total data quantity of the data, and the data to be sent is sent in a sectional mode according to the same data quantity of each single segment.
Example 3: defining conditions, setting different segment data as L 1 、L 2 、L 3 、…、L n-1 、L n Extracting some segment data L n Monitoring the real-time sending speed of the current segment data, wherein the current segment data L is set n The real-time sending speed of the data transmission system is 7M/s, 11M/s, 9M/s and 8M/s, the sending speed of the sent segmented data is monitored to be 11M/s, 10M/s, 11M/s, 12M/s and 13M/s by using a single-segment data sending speed statistic submodule, wherein certain segmented data L is set n The real-time transmission speed of (2) satisfies the following formula:
when the data L is segmented n The real-time sending speed of the current subsection meets the formula, and the different subsection data sending speed information analysis submodule judges that the real-time sending speed of the current subsection is not processed under the normal condition.
Example 4: defining conditions, setting different segment data as L 1 、L 2 、L 3 、…、L n-1 、L n Extracting some segment data L n Monitoring current segment dataReal-time transmission speed, wherein the current segment data L is set n The real-time sending speed of the data transmission system is 12M/s, 15M/s, 14M/s and 11M/s, the sending speed of the sent segmented data is monitored to be 17M/s, 21M/s and 19M/s by using a single-segment data sending speed statistic submodule, wherein certain segmented data L is set n Satisfies the following formula:
segment data L n Does not satisfy the above formula, segment the data L n The real-time sending speed of the data is compared with a set threshold value, the set threshold value is 12M/s, and the data L is segmented n The real-time sending speed of (1) is 11M/s, and the data L is segmented n The real-time transmission speed 11M/s is less than the set threshold value 12M/s, and the data sectional scheme confirmation submodule transmits the data to be transmitted in sections according to the same data quantity of each single section, and transmits the sectional data L n Re-segmenting according to the same data volume of each segment;
segment data L n The real-time sending speed 11M/s is less than the set threshold value 12M/s, and the data sectional scheme confirming submodule sends the data to be sent in sections according to different data format types and extracts the current sectional data L n After the data format is marked, the data format is sent again in a subsection sending mode according to the same data quantity of each section, and the marked data format is sent to a data subsection scheme confirming submodule to be referred.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (3)
1. A data transmission system employing a segmented data structure, characterized by: the system comprises a data to be sent real-time scanning module, different data stream segment dividing modules, a data configuration object sending process object query module, a segmented data sending speed synchronous acquisition module and a data integration module, wherein the data to be sent real-time scanning module, the different data stream segment dividing modules and the data integration module are sequentially connected through an intranet;
the system comprises a data to be transmitted, a real-time scanning module, a data configuration object transmitting process object query module, a segment data transmitting speed synchronous acquisition module, a data integration module and a data transmission module, wherein the data to be transmitted is used for scanning a data stream to be transmitted and judging different data formats of the transmitted data;
the sending method of the data sending system adopting the segmented data structure comprises the following steps:
s1: scanning a data stream to be transmitted by using a data real-time scanning module to be transmitted, judging different data formats of the transmitted data, integrating the data to be transmitted by using a data sorting set submodule, transmitting the integrated data to different data stream format analysis matching submodules, analyzing the data formats of the data to be transmitted by using the different data stream format analysis matching submodules, and judging the data formats of the different data and the data quantity in the different data formats;
s2: determining a segmentation mode of the data according to basic information by using different data stream segment dividing modules, acquiring data formats of different data stream format analysis matching sub-module detection data by using a data basic information analysis sub-module, counting the formats of different data, and determining a data segmentation scheme by using a data segmentation scheme determining sub-module according to the data format types analyzed by a data basic information analysis sub-module;
s3: the data configuration object sending process object query module is used for querying the content of each subsection positioned by the data, the remaining sent data subsection query submodule queries the sending process of the subsection data in real time in the sending process of different subsection data, the sending content of different subsections is accurately mastered, the data sending completion information presentation submodule presents the sending content information of the data of different subsections in time after the sending of the data of different subsections is completed, and the sent content of the data of different subsections is checked in real time;
s4: monitoring the sending of each segmented data by utilizing a segmented data sending speed synchronous acquisition module, counting the sending speeds of different segmented data by a single-segment data sending speed counting submodule, matching the counted sending speeds with the single-segment data to form a list and sending the list to different segmented data sending speed information analysis submodules, analyzing the sending speeds of the single-segment data by the different segmented data sending speed information analysis submodules, when the single-segment sending speed is less than a set threshold value, and a data segmented scheme confirmation submodule sends the data to be sent in a segmented mode according to the same data quantity of each single segment, re-segmenting the single segment of which the sending speed is less than the set threshold value according to the same data quantity of each segment, and when the single-segment sending speed is less than the set threshold value, and sending the data to be sent in a segmented mode according to different data format types by the data segmented scheme confirmation submodule, extracting the data format of a single section with the current sending speed less than a set threshold value, and sending the data in a subsection mode again according to the same data quantity of each section;
s5: and the data integration module is used for integrating the sent data in real time, the multi-segment data integration submodule is used for integrating the sent data, and the data release submodule is used for integrating and releasing the data after the data are successfully sent.
2. The data transmission system according to claim 1, wherein: in step S2, the method for determining the segmentation of the data according to the basic information by using different data stream segment segmentation modules, the data basic information analysis submodule obtaining the data formats of the data detected by the different data stream format analysis matching submodule, and counting the formats of the different data, and the data segmentation scheme confirmation submodule confirming the data segmentation scheme according to the data format types analyzed by the data basic information analysis submodule, further includes the following steps:
a1: acquiring data formats of detection data of different data stream format analysis matching sub-modules by using a data basic information analysis sub-module, counting the formats of different data, sending the counted data format types to a data sectional type scheme confirmation sub-module, and monitoring the format types of the data to be sent by using the data sectional type scheme confirmation sub-module;
a2: when the format type of the data to be sent is less than or equal to the set type, sending the format type of the data to be sent to a data sectional type scheme confirmation submodule for data segmentation, counting the total data volume of the data by the data sectional type scheme confirmation submodule, and sending the data to be sent in a sectional manner according to the same data volume of each single section;
a3: when the format type of the data to be sent is larger than the set type, acquiring the data quantity in different data formats detected by different data stream format analysis matching sub-modules, analyzing the data quantity in the current data format, and setting the data format to be segmented into L according to different data formats 1 、L 2 、L 3 、…、L n-1 、L n Wherein the data amount of a single segment in the different segments is R 1 、R 2 、R 3 、…、R n-1 、R n Setting the data amount of the single segment not to be lower than R0, and setting the data amount in the single segment to satisfy the following formula:
when the data quantity in the data to be sent single section does not meet the formula, the data sectional type scheme confirming submodule counts the total data quantity of the data to be sent and sends the data to be sent in a sectional mode according to different data format types, and when the data quantity in the data to be sent single section does not meet the formula, the data sectional type scheme confirming submodule counts the total data quantity of the data to be sent and sends the data to be sent in a sectional mode according to the same data quantity of each single section.
3. The data transmission system according to claim 2, wherein: in step S4, the single segment data transmission speed statistics submodule counts the transmission speeds of different segment data, matches the counted transmission speed with the single segment data, and then makes a list to be transmitted to the different segment data transmission speed information analysis submodule, and the different segment data transmission speed information analysis submodule analyzes the transmission speed of the single segment data, and the method further includes the following steps:
the differently set segment data is L 1 、L 2 、L 3 、…、L n-1 、L n Extracting some segment data L n Monitoring the real-time sending speed of the current segment data, wherein the current segment data L is set n Has a real-time transmission speed of V 1 、V 2 、V 3 、…、V n-1 、V n Monitoring the sending speed of the sent segmented data to be V0 by using a single-segment data sending speed statistic submodule 1 、V0 2 、V0 3 、…、V0 n-1 、V0 n Wherein a certain piece of data L is set n The real-time transmission speed of (2) satisfies the following formula:
when the data L is segmented n The real-time sending speed of the data processing system meets the formula, and the different-segment data sending speed information analysis submodule judges the current segmentThe real-time transmission speed is not processed under normal condition, and when the data L is segmented n Does not satisfy the above formula, segment the data L n The real-time sending speed is compared with a set threshold value when the data L is segmented n The real-time sending speed is less than the set threshold, and the data sectional scheme confirmation submodule sends the data to be sent in a sectional way according to the same data quantity of each single section, and transmits sectional data L n Re-segmenting according to the same data amount of each segment, and when the data L is segmented, re-segmenting n The real-time sending speed is less than the set threshold, and the data sectional type scheme confirmation submodule sends the data to be sent in a sectional way according to different data format types and extracts the current sectional data L n After the data format is marked, the data format is sent again in a subsection mode according to the same data quantity of each section, and the marked data format is sent to a data subsection scheme confirming submodule to be referred.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110287268.8A CN112910606B (en) | 2020-05-22 | 2020-05-22 | Data sending system adopting sectional type data structure |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110287268.8A CN112910606B (en) | 2020-05-22 | 2020-05-22 | Data sending system adopting sectional type data structure |
| CN202010439189.XA CN111614440B (en) | 2020-05-22 | 2020-05-22 | Data sending system and method adopting sectional type data structure |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010439189.XA Division CN111614440B (en) | 2020-05-22 | 2020-05-22 | Data sending system and method adopting sectional type data structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112910606A CN112910606A (en) | 2021-06-04 |
| CN112910606B true CN112910606B (en) | 2022-08-02 |
Family
ID=72205177
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110287268.8A Active CN112910606B (en) | 2020-05-22 | 2020-05-22 | Data sending system adopting sectional type data structure |
| CN202010439189.XA Active CN111614440B (en) | 2020-05-22 | 2020-05-22 | Data sending system and method adopting sectional type data structure |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010439189.XA Active CN111614440B (en) | 2020-05-22 | 2020-05-22 | Data sending system and method adopting sectional type data structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN112910606B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117349322B (en) * | 2023-12-05 | 2024-03-08 | 摩尔元数(福建)科技有限公司 | SPC real-time analysis method and system based on analysis control chart |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649691A (en) * | 2016-12-19 | 2017-05-10 | 雷桦松 | Stream data storage method and device |
| CN107229872A (en) * | 2016-03-26 | 2017-10-03 | 肖哲 | It is a kind of to separate storage query logic and the private data guard method of segment data |
| CN110035069A (en) * | 2019-03-15 | 2019-07-19 | 河南高通物联网有限公司 | A kind of Internet of Things information safety protecting method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10713596B2 (en) * | 2007-05-15 | 2020-07-14 | Intellireal, Llc | Transforming property data into sufficiently sized, relatively homogeneous data segments for configuring automated modeling systems |
| CN109995876B (en) * | 2019-04-11 | 2022-04-19 | 深圳市永兴元科技股份有限公司 | File transmission method, device, system and computer storage medium |
| CN110232136A (en) * | 2019-06-19 | 2019-09-13 | 浙江中普科技咨询有限公司 | A kind of big data processing system based on cloud computing |
| CN111090618B (en) * | 2019-10-29 | 2023-08-18 | 厦门网宿有限公司 | Data reading method, system and equipment |
-
2020
- 2020-05-22 CN CN202110287268.8A patent/CN112910606B/en active Active
- 2020-05-22 CN CN202010439189.XA patent/CN111614440B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107229872A (en) * | 2016-03-26 | 2017-10-03 | 肖哲 | It is a kind of to separate storage query logic and the private data guard method of segment data |
| CN106649691A (en) * | 2016-12-19 | 2017-05-10 | 雷桦松 | Stream data storage method and device |
| CN110035069A (en) * | 2019-03-15 | 2019-07-19 | 河南高通物联网有限公司 | A kind of Internet of Things information safety protecting method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111614440A (en) | 2020-09-01 |
| CN112910606A (en) | 2021-06-04 |
| CN111614440B (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108337652B (en) | Method and device for detecting flow fraud | |
| CN101119246B (en) | Data packet sampling statistic method and apparatus | |
| US20090185503A1 (en) | Network traffic analyzing device, network traffic analyzing method and network traffic analyzing system | |
| CN111970353A (en) | Data processing method and device for heterogeneous equipment in Internet of things of cloud computing platform | |
| CN112910606B (en) | Data sending system adopting sectional type data structure | |
| CN106610987A (en) | Video image retrieval method, device and system | |
| CN110536324B (en) | WiFi signal data acquisition and processing method and system | |
| CN110956123B (en) | Method, device, server and storage medium for auditing rich media content | |
| CN111611280A (en) | Encrypted traffic identification method based on CNN and SAE | |
| CN106506557B (en) | A kind of Portscan Detection Method and device | |
| CN105871585A (en) | Terminal association method and device | |
| CN116366497A (en) | Big data-based Internet of things card time sequence anomaly detection system | |
| CN104021348B (en) | Real-time detection method and system of dormant P2P (Peer to Peer) programs | |
| CN116501780B (en) | Enterprise audit data analysis processing system and method | |
| CN108881306B (en) | Encrypted flow analysis defense method based on data packet size sequence | |
| CN116647389A (en) | Network access security early warning system and method for industrial control system | |
| CN110636077A (en) | Network security protection system and method based on unified platform | |
| CN109618139A (en) | A kind of intelligent video monitoring system and method for view-based access control model routing | |
| CN111404832B (en) | Service classification method and device based on continuous TCP link | |
| CN111614786A (en) | System and method for processing data at high speed by remote server based on block chain | |
| CN114363944B (en) | Equipment communication performance test system based on C-V2X and test method thereof | |
| CN106612241A (en) | Service control method and service control device | |
| CN117313004B (en) | QoS flow classification method based on deep learning in Internet of things | |
| CN117714213B (en) | Evidence chain automatic verification method and system based on commercial password evaluation requirements | |
| CN117097578B (en) | Network traffic safety monitoring method, system, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220718 Address after: 430000 No.2, Donghu West Road, Wuchang District, Wuhan City, Hubei Province Applicant after: Hubei complete bidding Co.,Ltd. Address before: 215000 88 beixieyu street, Wuzhong District, Suzhou City, Jiangsu Province Applicant before: Chen Jie |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |