CN112350807A - Fault-tolerant method for avoiding network storm for terminal big data acquisition system - Google Patents
Fault-tolerant method for avoiding network storm for terminal big data acquisition system Download PDFInfo
- Publication number
- CN112350807A CN112350807A CN201910757295.XA CN201910757295A CN112350807A CN 112350807 A CN112350807 A CN 112350807A CN 201910757295 A CN201910757295 A CN 201910757295A CN 112350807 A CN112350807 A CN 112350807A
- Authority
- CN
- China
- Prior art keywords
- terminal
- connection
- retry
- time
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1867—Arrangements specially adapted for the transmitter end
- H04L1/188—Time-out mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/32—Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/143—Termination or inactivation of sessions, e.g. event-controlled end of session
- H04L67/145—Termination or inactivation of sessions, e.g. event-controlled end of session avoiding end of session, e.g. keep-alive, heartbeats, resumption message or wake-up for inactive or interrupted session
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a fault-tolerant method for a terminal big data acquisition system to avoid network storms, which avoids the network storm phenomenon of a server side by three mechanisms of nonlinear increase of overtime retransmission time length, random jitter of the overtime retransmission time length and suppression of retransmission times under the condition that a terminal fails to report data to the server side; each retry time point of the terminals is distributed discretely and randomly, so that the probability that a large number of terminals initiate retry requests at the same time can be reduced; the interval time of the terminal initiating the retry request is increased along with the retry times, so that a vicious circle caused by continuous retry-failure-retry of a large number of terminals in unit time can be avoided; and flexibly setting the maximum retry times of the terminal according to the data uploading reliability requirement. The fault-tolerant method for avoiding the network storm of the terminal big data acquisition system can avoid the network storm generated by the server side due to repeated data retransmission of the terminal.
Description
Technical Field
The invention belongs to the technical field of network communication, relates to a fault tolerance method, and particularly relates to a fault tolerance method for avoiding a server from generating a network storm due to repeated data retransmission of a terminal under the scenes of network abnormality, server abnormality and client abnormality when a data acquisition terminal in a terminal big data acquisition system reports data to the server.
Background
With the rapid development of IP networks and applications and the increasing competition of telecommunication markets, the operators of large networks around the world are turning off the trend of the development of the convergence of three networks. Network operators are developing broadband services and video services, and user side terminal equipment (including intelligent set top boxes, intelligent gateways and the like) shows explosive growth and is accompanied by increasingly intense market competition. In order to improve the service quality of products and market competitiveness, operators need to acquire the service quality perception of users on the internet and videos, and usually adopt a means of implanting a soft probe into user side terminal equipment to collect service quality data of user terminals, and then mine and analyze a large amount of terminal data, so as to realize network optimization, user behavior analysis and refined service operation.
The terminal big data acquisition system is generally divided into a terminal soft probe and a data acquisition analysis platform, taking a set-top box soft probe as an example, the number of the soft probe is often millions, or even billions, and the service data acquired by the terminal soft probe comprises: login authentication, video playing, authentication ordering, set top box performance data and the like. Therefore, reporting of a large amount of terminal data is a very serious challenge to both network bandwidth and processing performance of the acquisition platform. How to reasonably set a terminal reporting strategy to ensure that an acquisition and analysis platform cannot generate overload or even downtime serious faults due to the influence of simultaneously sending massive requests and data reports by a terminal in abnormal scenes such as network abnormality, platform faults and the like is a challenge faced by a terminal big data acquisition system.
In a terminal big data acquisition system, a terminal soft probe generally reports data to an acquisition analysis platform periodically. If the network or the platform fails within a period of time, the soft probe cannot report data successfully; to ensure that data is not lost, soft probes typically attempt to reconnect the server immediately; at this time, because a large number of terminals all initiate reconnection requests in the same time period, a large number of requests and data reports are generated instantaneously, which is a network data storm for the acquisition and analysis platform.
In a terminal big data acquisition system, when a network data storm occurs, the network load on a platform side is greatly increased, the performance of an acquisition platform is sharply reduced, a terminal request cannot be responded in time and terminal data are processed, and then a terminal initiates a reconnection retry request more times, so that vicious circle is caused, data acquisition is seriously influenced, and the system fault duration is increased. If the problem cannot be effectively solved, the stability of the terminal big data acquisition system is inevitably reduced, data is inevitably lost, and the system cannot exert the due efficiency.
In order to solve the network storm problem, the current common implementation methods in the industry mainly include two types:
the method comprises the following steps: the number of retries of the terminal in an abnormal situation is reduced.
The method has the advantages that the terminal can be prevented from generating excessive or even endless retry requests, and therefore system paralysis caused by a large number of endless retry requests of the terminal on the platform side can be avoided.
The method has the defects that the problem of overlarge load of a platform caused by simultaneous reporting of data by a large number of terminals at the time of fault recovery cannot be solved; in addition, the terminal gives up data reporting after repeated retries, which also causes part of terminal data to be lost.
The second method comprises the following steps: the terminal retry request interval is increased.
The method has the advantages that the retry and data reporting times initiated by the terminal to the server in unit time can be reduced by increasing the retry request interval of the terminal, so that the pressure of a platform is relieved.
The disadvantage is that the terminal retry reporting requests cannot be evenly distributed over a period of time, and there is still a possibility of unbalanced load stress on the server.
In view of the above, there is an urgent need to design a terminal big data acquisition method so as to overcome the above-mentioned defects of the existing terminal big data acquisition methods.
Disclosure of Invention
The invention provides a fault-tolerant method for a terminal big data acquisition system to avoid network storms, which can avoid the network storms generated by a server side due to repeated data retransmission of a terminal.
In order to solve the technical problem, according to one aspect of the present invention, the following technical solutions are adopted:
a fault-tolerant method for a terminal big data acquisition system to avoid network storms is characterized in that the fault-tolerant method avoids the network storm phenomenon at a server side through three mechanisms of 'nonlinear increase of timeout retransmission time length', 'random jitter of timeout retransmission time length' and 'inhibition of retransmission times' under the condition that a terminal fails to report data to the server side, and specifically comprises the following steps:
step A1: after the terminal collects the data, reading the retry initialization variable, and preparing data to report for subsequent calculation and judgment;
the number of retries: retries is 1;
fixed time delay (seconds): interval is 5;
maximum waiting period: max _ retries ═ 3;
the connection state is as follows: connection _ state ═ success; wherein success represents connection success;
step A2: the terminal tries to establish connection with the server, if the connection is established successfully, data is reported, if the connection is established unsuccessfully, a connection state connection _ state is marked as fail, and retransmission is prepared to be started;
step A3: when the connection between the terminal and the server is successfully established, data reporting is tried, if the data reporting is successful, the connection quitting process is closed, and if the data reporting is failed, the terminal prepares to start retransmission;
step A4: in preparation for starting retransmission, the following variables are calculated:
random time delay: range _ interval is set to a random value of 0 to interval;
retry waiting time: wait _ time ═ (interval + rand _ interval) × (res);
step A5: the terminal waits for a certain delay wait _ time, because the wait _ time is a random value, even if a large number of terminals are simultaneously ready for retry, the actual retry time point will discretely fall within a period of time;
step A6: judging whether the retry times retries exceed a preset maximum retry time max _ retries, if so, quitting the process, giving up the data report, closing the connection quitting process, and if not, setting the retry times retries as retries + 1;
step A7: judging whether the connection _ state is fail, if so, re-initiating a new connection request to the server, otherwise, actively closing the connection, and then re-initiating the new connection request to the server, so as to ensure that the terminal side does not have connection accumulation; wherein fail indicates a connection failure.
The invention comprehensively analyzes the defects of the common network storm avoiding mechanism in the industry, fully considers the balance point of data reporting timeliness and network storm avoiding when designing a new scheme strategy, and provides the following optimization scheme aiming at the mechanism of the terminal big data acquisition system for avoiding the network storm:
each retry time point of the terminals is distributed discretely and randomly, so that the probability that a large number of terminals initiate retry requests at the same time can be reduced;
the interval time of the terminal initiating the retry request is increased along with the retry times, so that a vicious circle caused by continuous retry-failure-retry of a large number of terminals in unit time can be avoided, the pressure on the platform side can be gradually reduced along with the increase of the retry interval, and the request of the terminal can be better processed;
and flexibly setting the maximum retry times of the terminal according to the data uploading reliability requirement.
The invention has the beneficial effects that: the fault-tolerant method for avoiding the network storm of the terminal big data acquisition system can avoid the network storm generated by the server side due to repeated data retransmission of the terminal.
Drawings
FIG. 1 is a flowchart illustrating a fault tolerant method for avoiding network storms according to an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
For a further understanding of the invention, reference will now be made to the preferred embodiments of the invention by way of example, and it is to be understood that the description is intended to further illustrate features and advantages of the invention, and not to limit the scope of the claims.
The description in this section is for several exemplary embodiments only, and the present invention is not limited only to the scope of the embodiments described. It is within the scope of the present disclosure and protection that the same or similar prior art means and some features of the embodiments may be interchanged.
Example one
Referring to fig. 1, the present invention discloses a fault-tolerant method for a terminal big data acquisition system to avoid a network storm, which is characterized in that the fault-tolerant method avoids the network storm phenomenon at a server through three mechanisms of "nonlinear increase of timeout retransmission time length", "random jitter of timeout retransmission time length", and "suppression of retransmission times" when a terminal fails to report data to the server, and specifically includes the following steps:
step A1: after the terminal collects the data, reading the retry initialization variable, and preparing data to report for subsequent calculation and judgment;
the number of retries: retries is 1;
fixed time delay (seconds): interval is 5;
maximum waiting period: max _ retries ═ 3;
the connection state is as follows: connection _ state ═ success; wherein success represents connection success;
step A2: the terminal tries to establish connection with the server, if the connection is established successfully, data is reported, if the connection is established unsuccessfully, a connection state connection _ state is marked as fail, and retransmission is prepared to be started;
step A3: when the connection between the terminal and the server is successfully established, data reporting is tried, if the data reporting is successful, the connection quitting process is closed, and if the data reporting is failed, the terminal prepares to start retransmission;
step A4: in preparation for starting retransmission, the following variables are calculated:
the random delay rand _ interval is set to a random value from 0 to interval: range _ interval ═ range (0, interval);
retry waiting time: wait _ time ═ (interval + rand _ interval) × (res);
step A5: the terminal waits for a certain delay wait _ time, because the wait _ time is a random value, even if a large number of terminals are simultaneously ready for retry, the actual retry time point will discretely fall within a period of time;
step A6: judging whether the retry times retries exceed a preset maximum retry time max _ retries, if so, quitting the process, giving up the data report, closing the connection quitting process, and if not, setting the retry times retries as retries + 1;
step A7: judging whether the connection _ state is fail, if so, re-initiating a new connection request to the server, otherwise, actively closing the connection, and then re-initiating the new connection request to the server, so as to ensure that the terminal side does not have connection accumulation; wherein fail indicates a connection failure.
In summary, the fault-tolerant method for avoiding the network storm by the terminal big data acquisition system provided by the invention can avoid the network storm generated by the server due to the repeated retransmission of the data by the terminal.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The description and applications of the invention herein are illustrative and are not intended to limit the scope of the invention to the embodiments described above. Variations and modifications of the embodiments disclosed herein are possible, and alternative and equivalent various components of the embodiments will be apparent to those skilled in the art. It will be clear to those skilled in the art that the present invention may be embodied in other forms, structures, arrangements, proportions, and with other components, materials, and parts, without departing from the spirit or essential characteristics thereof. Other variations and modifications of the embodiments disclosed herein may be made without departing from the scope and spirit of the invention.
Claims (1)
1. A fault-tolerant method for a terminal big data acquisition system to avoid network storms is characterized in that the fault-tolerant method avoids the network storm phenomenon at a server side through three mechanisms of 'nonlinear increase of timeout retransmission time length', 'random jitter of timeout retransmission time length' and 'inhibition of retransmission times' under the condition that a terminal fails to report data to the server side, and specifically comprises the following steps:
step A1: after the terminal collects the data, reading the retry initialization variable, and preparing data to report for subsequent calculation and judgment;
the number of retries: retries is 1;
fixing time delay: interval is 5;
maximum waiting period: max _ retries ═ 3;
the connection state is as follows: connection _ state ═ success; wherein success represents connection success;
step A2: the terminal tries to establish connection with the server, if the connection is established successfully, data is reported, if the connection is established unsuccessfully, a connection state connection _ state is marked as fail, and retransmission is prepared to be started;
step A3: when the connection between the terminal and the server is successfully established, data reporting is tried, if the data reporting is successful, the connection quitting process is closed, and if the data reporting is failed, the terminal prepares to start retransmission;
step A4: in preparation for starting retransmission, the following variables are calculated:
random time delay: range _ interval is set to a random value of 0 to interval;
retry waiting time: wait _ time ═ (interval + rand _ interval) × (res);
step A5: the terminal waits for the set delay wait _ time, because the wait _ time is a random value, even if a large number of terminals are simultaneously ready for retry, the actual retry time point will discretely fall within a period of time;
step A6: judging whether the retry times retries exceed a preset maximum retry time max _ retries, if so, quitting the process, giving up the data report, closing the connection quitting process, and if not, setting the retry times retries as retries + 1;
step A7: judging whether the connection _ state is fail, if so, re-initiating a new connection request to the server, otherwise, actively closing the connection, and then re-initiating the new connection request to the server, so as to ensure that the terminal side does not have connection accumulation; wherein fail indicates a connection failure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910757295.XA CN112350807A (en) | 2019-08-07 | 2019-08-07 | Fault-tolerant method for avoiding network storm for terminal big data acquisition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910757295.XA CN112350807A (en) | 2019-08-07 | 2019-08-07 | Fault-tolerant method for avoiding network storm for terminal big data acquisition system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112350807A true CN112350807A (en) | 2021-02-09 |
Family
ID=74367803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910757295.XA Withdrawn CN112350807A (en) | 2019-08-07 | 2019-08-07 | Fault-tolerant method for avoiding network storm for terminal big data acquisition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112350807A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20080067451A (en) * | 2007-01-16 | 2008-07-21 | 삼성전자주식회사 | Method and apparatus for retransmission packet data |
| CN101730136A (en) * | 2009-11-26 | 2010-06-09 | 中国电信股份有限公司 | Processing method and system for M2M terminal call |
| CN102255709A (en) * | 2010-05-17 | 2011-11-23 | 中兴通讯股份有限公司 | Data retransmission method, base station, terminal and wireless communication system |
| CN102291846A (en) * | 2010-06-21 | 2011-12-21 | 中兴通讯股份有限公司 | Method for random access and system for realizing same |
| CN102958101A (en) * | 2012-12-03 | 2013-03-06 | 李俊 | Method for avoiding wireless channel congestion on wireless communication terminal |
| CN106547635A (en) * | 2015-09-18 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of operation of operation retries method and apparatus |
| CN107690200A (en) * | 2016-08-05 | 2018-02-13 | 电信科学技术研究院 | A kind of accidental access method and terminal |
| CN108307439A (en) * | 2017-08-17 | 2018-07-20 | 上海大学 | A kind of self-adoptive retreating method under a large amount of multi-hop wireless nodes |
| CN109118344A (en) * | 2018-07-06 | 2019-01-01 | 阿里巴巴集团控股有限公司 | A kind of business retries method and apparatus |
| CN109391593A (en) * | 2017-08-08 | 2019-02-26 | 展讯通信（上海）有限公司 | The time that retries for media session determines method and device, storage medium, terminal |
-
2019
- 2019-08-07 CN CN201910757295.XA patent/CN112350807A/en not_active Withdrawn
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20080067451A (en) * | 2007-01-16 | 2008-07-21 | 삼성전자주식회사 | Method and apparatus for retransmission packet data |
| CN101730136A (en) * | 2009-11-26 | 2010-06-09 | 中国电信股份有限公司 | Processing method and system for M2M terminal call |
| CN102255709A (en) * | 2010-05-17 | 2011-11-23 | 中兴通讯股份有限公司 | Data retransmission method, base station, terminal and wireless communication system |
| CN102291846A (en) * | 2010-06-21 | 2011-12-21 | 中兴通讯股份有限公司 | Method for random access and system for realizing same |
| WO2011160579A1 (en) * | 2010-06-21 | 2011-12-29 | 中兴通讯股份有限公司 | Random access method and system for realizing random access |
| CN102958101A (en) * | 2012-12-03 | 2013-03-06 | 李俊 | Method for avoiding wireless channel congestion on wireless communication terminal |
| CN106547635A (en) * | 2015-09-18 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of operation of operation retries method and apparatus |
| CN107690200A (en) * | 2016-08-05 | 2018-02-13 | 电信科学技术研究院 | A kind of accidental access method and terminal |
| CN109391593A (en) * | 2017-08-08 | 2019-02-26 | 展讯通信（上海）有限公司 | The time that retries for media session determines method and device, storage medium, terminal |
| CN108307439A (en) * | 2017-08-17 | 2018-07-20 | 上海大学 | A kind of self-adoptive retreating method under a large amount of multi-hop wireless nodes |
| CN109118344A (en) * | 2018-07-06 | 2019-01-01 | 阿里巴巴集团控股有限公司 | A kind of business retries method and apparatus |
Non-Patent Citations (1)
| Title |
|---|
| 吴克军等: "Ad Hoc网络媒体接入控制中一种新的退避算法", 《北京邮电大学学报》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10133644B2 (en) | Method and apparatus for indirectly assessing a status of an active entity | |
| CN101123588B (en) | Method, media gateway and system for controlling redundant data packet transmission | |
| EP2008114B1 (en) | Method and system for alert throttling in media quality monitoring | |
| US9088494B2 (en) | Packet fragmentation prevention | |
| CN100553213C (en) | A kind of method and apparatus of WLAN abnormal quick restoring | |
| KR100812374B1 (en) | System and method for managing protocol network failures in a cluster system | |
| US20120124431A1 (en) | Method and system for client recovery strategy in a redundant server configuration | |
| CN101820499B (en) | Method and system for realizing automatic interaction between STB (set top box) and home gateway | |
| US11297158B1 (en) | Proxy selection by monitoring quality and available capacity | |
| Seufert et al. | Quicker or not?-an empirical analysis of quic vs tcp for video streaming qoe provisioning | |
| CN111092900B (en) | Method and device for monitoring abnormal connection and scanning behavior of server | |
| CN114422571A (en) | Quantum communication client disconnection reconnection system and method | |
| WO2014110911A1 (en) | Fault processing method and apparatus in iptv system | |
| CN107682442A (en) | A kind of Web connection methods and device | |
| US11082309B2 (en) | Dynamic and interactive control of a residential gateway connected to a communication network | |
| CN112350807A (en) | Fault-tolerant method for avoiding network storm for terminal big data acquisition system | |
| CN110557381B (en) | Media high-availability system based on media stream hot migration mechanism | |
| EP2727286B1 (en) | Remote management of devices | |
| WO2016176910A1 (en) | Tr-069 message processing method and apparatus | |
| CN101562563A (en) | Fast re-dial method after user is off line improperly | |
| JP6470199B2 (en) | Communication device, resetting method, and resetting program | |
| US20140282617A1 (en) | Density-based event handling | |
| JP2005011267A (en) | Real-time data communication system, real-time data communication device and method for real-time communication | |
| US20100312848A1 (en) | Method and System for Parallel Call Setup | |
| EP2827535A1 (en) | Method for isolated anomaly detection in large-scale audio/video/data processing systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210209 |
|
| WW01 | Invention patent application withdrawn after publication |