WO2025227697A1 - 一种保障广域云网管控面稳定的方法和系统 - Google Patents
一种保障广域云网管控面稳定的方法和系统Info
- Publication number
- WO2025227697A1 WO2025227697A1 PCT/CN2024/135840 CN2024135840W WO2025227697A1 WO 2025227697 A1 WO2025227697 A1 WO 2025227697A1 CN 2024135840 W CN2024135840 W CN 2024135840W WO 2025227697 A1 WO2025227697 A1 WO 2025227697A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- inspection
- business
- data
- control plane
- fault
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
Definitions
- This invention relates to the field of network communication technology, and in particular to a method and system for ensuring the stability of a wide-area cloud network control plane.
- the present invention provides the following technical solution: a method for ensuring the stability of the wide-area cloud network control plane, comprising the following specific steps:
- the priority and weight of inspection tasks can be set automatically or manually.
- Manual setting involves setting the weight and number of times of the task, while automatic setting involves calculating the unstable parameters and channel coefficients of the interface based on monitoring and historical data. The two interact to comprehensively determine the inspection priority of each interface.
- step S1 the weight classification is divided into multiple dimensions, and the order and number of times a single service runs during inspection are determined according to the priority and weight of the dimensions and the influence of the environment.
- step S2 subsequent tasks are first issued in sequence, and after the set time has passed, they are forwarded to the queue of tasks to be issued in the order of their sequence.
- step S3 the weight is automatically increased based on the impact of the business; an optimized inspection task is automatically generated periodically based on the current monitoring situation, and relevant personnel determine whether to update the new task.
- the types of faults include downlink faults, data anomaly faults, system faults, etc., and the system handles them by combining different fault types with the characteristics of the system.
- step S5 when a fault occurs during inspection and cannot be resolved immediately, temporary changes can be made through emergency avoidance measures to prevent online faults.
- this interface will automatically send the request three times to avoid faults caused by network jitter. If the request is successful, it will be recorded as an unstable interface and the fault will not be recorded.
- the system will determine whether the fault is a link failure, a data anomaly, or a system defect based on the error message.
- the main manifestation of link failure is timeout. If a timeout occurs, the first priority is retry. While retrying, the system performs a dial test on the peer end and judges whether the peer end service is normal by using heartbeat information, peer machine monitoring and other multi-dimensional data. If the retry fails after three attempts, the system's judgment and comprehensive analysis indicate that the failure is caused by the link being unavailable. At this time, the system switches to the backup link for delivery and actively initiates a link alarm to notify the relevant operation and maintenance personnel to repair it.
- the backup link fails to be deployed, it indicates that the region controller may be faulty. In this case, the configuration is deployed through the backup region controller corresponding to the region controller. At the same time, an alarm is actively issued to notify the relevant maintenance personnel to repair it.
- the aforementioned data issues mainly refer to situations where the uploaded data passes the control plane verification but fails the verification of the underlying network element layer. This type of problem is primarily caused by inconsistencies between the upper and lower layers. In this case, the system re-triggers the automatic reconciliation function. If the reconciliation data is inconsistent, the system will send the reconciliation results to the operations and maintenance personnel to verify whether data synchronization is required. If data synchronization is required, the system will retry after data synchronization to see if the problem is resolved.
- the present invention also provides a system for ensuring the stability of the wide area cloud network control plane, characterized in that: the system includes an inspection use case platform, a queue to be issued, a policy setting platform, a log collection and monitoring system, an inspection policy configuration module, a fault handling module, and a data reconciliation system;
- the inspection use case platform is used to manage inspection use cases. Users can manage the inspection use cases for each task through the inspection use case platform, and modify and add related use cases to adapt to changes in the platform.
- the pending queue is used when the control and control channel is too busy and inspection cases cannot be executed in a timely manner.
- the inspection cases are first put into the pending queue so that the system can reprocess the tasks in the pending queue at intervals.
- the strategy setting platform is used by users to set relevant business priorities, thereby affecting the settings of the entire inspection use case platform;
- Log collection and monitoring are mainly used to collect the configurations issued by the global controller and regional controller for each service, thereby affecting the entire issuance task;
- the inspection strategy configuration module allows users to actively set strategies through the strategy setting platform or calculate optimized solutions based on the current situation through the monitoring system.
- the fault handling module is mainly used to handle problems promptly when they are discovered during inspections by immediately activating the fault handling module. These problems mainly include issues such as line outages and service failures.
- Data reconciliation is primarily used to address the impact of data inconsistencies, dirty data, or missing data.
- the user calculates the priority of the test case by setting weights, times and other factors.
- the formula for the score of the inspection test case is ((business weight/times) + channel coefficient) * instability coefficient.
- the inspection test case with the higher score calculated by this formula is executed first.
- business weight and frequency refer to classifying businesses into core businesses, key businesses, general businesses, and test businesses;
- core business refers to business covering more than 60% of customers and more than 90% of VIP customers, with a weight of 1000 and an occurrence of 10 times; key business refers to business coverage beyond core business, including more than 80% of business and all VIP business, with a weight of 400 and an occurrence of 5 times; general business includes all online businesses currently in use, with a weight of 120 and an occurrence of 2 times; and test business refers to new businesses in the testing process, with a weight of 50 and an occurrence of 1 time.
- the channel coefficient represents the importance of a link, determined by the traffic volume of the link.
- the coefficient for particularly important links is 100, for generally important links it is 50, and for links used less frequently it is 10.
- the instability coefficient is determined by the key monitoring items, the number of failures, and the mean time between failures. Generally, if a key monitoring item alarms once, the initial value is 1 and cannot be less than 1, and the coefficient increases by 0.01; if a request fails once, the coefficient increases by 0.02; if there are no failures for an average of 6 hours, the coefficient decreases by 0.1.
- the inspection strategy configuration module is the core module of the entire system.
- the optimized scheme needs to be confirmed by the operation and maintenance personnel before it can be switched.
- This solution optimizes the priority of inspection test cases.
- the weight and number of test cases in each round of inspection can be set according to the importance of the function.
- the priority of relevant test cases can be automatically adjusted within a certain range through monitoring data, so as to maximize the operation of inspection resources.
- the inspection system in this solution prioritizes the real-time detection of faults in critical services. Through the combined effect of weighted and monitoring modifications, important services, services recovering from faults, and services with unstable operation are prioritized for inspection and multiple inspections. This allows for the timely detection of system problems and their prioritization for avoidance and resolution.
- Figure 1 is a flowchart of the cloud network management and control plane distribution process in the existing technology.
- Figure 2 is a system architecture diagram of the present invention.
- FIG. 3 is a system flowchart of the present invention.
- FIG. 4 is a flowchart of the fault handling process of the present invention.
- one embodiment or “embodiment” as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention.
- the phrase "in one embodiment” appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.
- This solution focuses on addressing the stability of the cloud network control plane. Due to the increasing complexity of networks and the rapid increase in business capacity, the workload and difficulty of online inspections are growing. If inspections are conducted frequently, they will consume a large amount of control channel resources, thereby affecting the distribution of control planes. At the same time, if the number of inspections is too low, the effectiveness of inspections will be reduced. Therefore, designing a reasonable inspection method is very important.
- the purpose of inspection is to proactively identify system problems. In a large system, it is valuable to discover problems in a timely manner with limited resources. Discovering problems before customers can effectively reduce network failures and improve system stability. After identifying related problems, rapid solutions are needed. Targeted measures should be taken to quickly resolve potential related issues in order to reduce failures. Inspections should not affect online business. The system also needs to be strictly designed with relevant mechanisms to ensure that inspection tasks do not affect the delivery of normal business.
- an embodiment of the present invention is provided to solve the above-mentioned technical problems by ensuring the stability of the wide-area cloud network control plane, as shown in Figure 2.
- the specific details are as follows:
- Inspection Case Platform This platform is for managing inspection case platforms. Users can manage inspection case platforms for each task here, and modify and add related case platforms to adapt to changes in the platform.
- the pending queue refers to tasks that are not executed in a timely manner because the control and control channel is constantly busy due to high water levels. Therefore, these tasks are placed in the pending queue and the system will reprocess them at intervals.
- Strategy setting platform Users can use this platform to set relevant business priorities, thereby affecting the settings of the entire inspection use case platform;
- Log collection and monitoring mainly collects the configurations issued by the global controller and regional controller for each service, thereby affecting the entire issuance task;
- the inspection strategy configuration module is the core module of the entire system. It can be actively configured through the strategy setting platform, or the monitoring system can calculate the optimized solution based on the current situation. However, the optimized solution needs to be confirmed by the operation and maintenance personnel before it can be switched.
- Fault handling The main function is to immediately activate the fault handling module if a problem is found during inspection, so as to deal with problems such as line failure and service failure in a timely manner.
- Data reconciliation This mainly deals with the impact of data inconsistencies, dirty data, or missing data.
- the existing technology initiates a task through a cloud platform, then distributes it to the global controller, and then the global controller finds the corresponding regional controller through the network element device that needs to be distributed the configuration, and then distributes the configuration to the corresponding network element through the regional controller.
- the business is set up. Users calculate the priority of the use case by setting weights, frequency, and other factors.
- the formula for the score of the inspection use case is ((business weight/frequency) + channel coefficient) * instability coefficient.
- Businesses are divided into core businesses, key businesses, general businesses, and test businesses. Core businesses generally cover more than 60% of customers and more than 90% of VIP customers. Core businesses have a weight of 1000 and a frequency of 10. Key businesses refer to further business coverage beyond core businesses, including more than 80% of businesses and all VIP businesses. Key businesses have a weight of 400 and a frequency of 5. General businesses include all businesses currently in use online. General businesses have a weight of 120 and a frequency of 2. Test businesses refer to new businesses in the testing process. Test businesses have a weight of 50 and a frequency of 1.
- the channel coefficient represents the importance of the link, which is determined by the traffic volume of the link.
- the coefficient for particularly important links is 100, for generally important links it is 50, and for links used less frequently it is 10.
- the instability coefficient is determined by the key monitoring items, the number of failures, and the average number of faults. Generally, if a key monitoring item alarms once, the initial value is 1 and cannot be less than 1. The coefficient increases by 0.01. If a request fails once, the coefficient increases by 0.02. If there are no faults for an average of 6 hours, the coefficient decreases by 0.1.
- Task A is a priority task, so we can set its weight to 400 and the number of runs to 5.
- Task B is a general task, so we can set its weight to 120 and the number of runs to 2.
- the channel coefficient is 0 and the instability coefficient is 1.
- the scores of task A after 5 runs are 400, 320, 240, 160, and 80, and the scores of task B are 120 and 60.
- important tasks can be prioritized and run multiple times within a cycle to ensure performance.
- the channel coefficient refers to the weighting of more important channels. When a channel is more important, its weight will increase, and the execution priority of the related function will rise.
- the instability coefficient is fed back in real time based on the system's condition. Administrators can enable the system to automatically optimize in real time based on network conditions, and focus the verification of inspection test cases more on interfaces with unstable interfaces and poor channel coefficients, thereby making it more likely to discover problems.
- service A has two distribution links, which distribute to network elements a1 and b1 respectively.
- the path to network element b1 is worse. This will increase the instability of network element b1, and the priority of testing network element b1 will increase, so that problems can be detected earlier.
- the instability coefficient will automatically decrease, so that the link returns to the normal predetermined priority.
- the distribution process begins. However, the distribution of inspection tasks should not affect the distribution of normal business tasks. Therefore, before each inspection task is distributed, the load level of the relevant distribution channel is checked. When the relevant load level exceeds a certain threshold, the task will enter the waiting queue for distribution. After a period of time, a judgment will be made. If the task is not distributed for a certain number of times, an alarm will be triggered. If the load level of the control channel is consistently high, there may be a problem.
- the global controller can locate the region controller to which the network element belongs, and then the region controller issues the task to the specific network element.
- the log and monitoring system records the success rate, performance and other parameters of each issued task, and generates relevant reports for output. For important tasks and interfaces, relevant alarm information will also be generated to facilitate timely detection of problems.
- test information will also be combined with online data to evaluate each interface and related control channels, recalculate the channel parameters and interface instability parameters, and thus affect the test priority of related interfaces through the above formula.
- the tasks are divided into two types: manual setting and automatic setting.
- Manual setting involves setting the weight and number of times the task is executed, while automatic setting involves calculating the unstable parameters of the interface based on monitoring and historical data.
- the two types of settings work together to comprehensively determine the inspection priority of each interface.
- the solution is as follows: When a fault occurs, the interface will automatically send three requests to avoid faults caused by network jitter. If any request is successful, it can be recorded as an unstable interface, and the fault will not be recorded.
- the system will determine whether it is a link failure, data anomaly, or system defect based on the error message.
- the main manifestation of link failure is timeout. If a timeout occurs, the first priority is to retry. While retrying, the system will perform dial tests on the peer end and use heartbeat information, peer machine monitoring and other dimensions to determine whether the peer end service is normal. If the retry fails after 3 attempts and the system's judgment and comprehensive analysis determine that the link is not working, then switch to the backup link for delivery and actively initiate link alarm to notify relevant operation and maintenance personnel to repair it.
- the data problem here mainly refers to the uploaded data passing the control plane data verification but failing the underlying network element layer verification. This is mainly caused by inconsistencies between the upper and lower layers. In this case, the system will re-trigger automatic reconciliation. If the reconciliation data is inconsistent, the system will send the reconciliation results to the operations and maintenance personnel to verify whether data synchronization is required. If so, after synchronizing the data, retry to see if the problem is resolved.
- the system can set corresponding weight values and frequency based on the importance of the business.
- the system can determine which test cases to execute first by comparing the final calculated priorities. Different delivery channels for the same test case also vary depending on the number of users and the importance of each user. Therefore, it is also necessary to set channel parameters to affect the priority of test business delivery.
- monitoring and logging can also change the relevant results based on the success rate of channels and interfaces, allowing those interfaces and channels with poor stability to be tested first within a certain range, and to discover related problems in a timely manner.
- the system also provides emergency solutions for common problems, promptly resolves and corrects system errors, resolves 100% of non-system defects during operation and maintenance, and avoids system defects as much as possible.
- this system can set corresponding weight values and frequency based on the importance of the business.
- the system can determine which test cases to execute first by comparing the final calculated priorities. Different delivery channels for the same test case also vary depending on the number of users and their importance. Therefore, it is also necessary to set channel parameters to affect the priority of test business delivery.
- monitoring and logging can also change the relevant results based on the success rate of channels and interfaces, allowing those interfaces and channels with poor stability to be tested first within a certain range, and related problems to be discovered in a timely manner.
- the system provides emergency solutions for common problems, promptly resolving and correcting system errors, ensuring 100% resolution of non-system defects during operation and maintenance, and minimizing the occurrence of system defects.
- Wide Area Network This refers to the network between local devices and remote cloud VPCs, including switches, network access devices, gateways, firewalls, and the network between VPCs.
- Control plane refers to the control channel for the configuration distribution process when business changes occur, from the platform side to the controller, and then to the network element device control distribution channel.
- Inspection refers to performing test case checks on online systems to discover problems in real time.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
本发明公开了一种保障广域云网管控面稳定的方法和系统,本方法对巡检用例的优先级进行了优化,可以按照功能的重要程度来设置每轮巡检中用例的权重和次数,同时,还可以通过监控数据自动的去对相关用例的优先级去进行一定范围内的调整,能最大范围内运行巡检资源;巡检系统下发巡检用例时,会严格按照监控系统中关于各管控通道下各机器的水位来执行,如果相关通道水位过高,则用例进入待下发队列,过段时间重新校验,避免影响线上正常业务的下发;巡检系统注重重要业务的故障优先发现的实时性,通过加权和监控修改共同作用,让重要的业务、故障恢复的业务、运行不稳定的业务优先巡检、多次巡检,这样就可以及时发现系统问题,优先规避和解决。
Description
本发明涉及网络通信技术领域,尤其涉及一种保障广域云网管控面稳定的方法和系统。
随着云计算技术的高度发展,越来越多的业务选择了上云管理,随着上云业务的复杂度的提升,以及业务规模的增大,对业务的稳定性,业务变更的稳定性,以及网络的稳定性的要求也越来越高,为了保证管控下发通道的稳定性,现通常采用两种方案,一种是线上巡检的方式,主动的去通过测试接口去验证通道的稳定性;一种是通过监控去统计各类指标,从而推断出通道是否稳定,但是上述两种方法存在以下几个缺陷:
1)线上巡检的工作量会随着业务的发展,工作量越来越大,由于网络的不断扩充以及业务模式的越来越复杂,全量验证的工作量呈指数型增长,而全量验证工作量的增多也给线上巡检工作带来了很大的难度,如果巡检次数过于频繁,就会占用大量的管控通道的资源,如果次数过低,又不能实时反应出问题;
2)监控存在一定的滞后性,如果仅通过监控下发请求的错误来推导、定位出哪块业务存在问题,那么这个是问题已经发生后的弥补,如果是只是监控机器的状态,水位容量之类的,则很多业务上的问题没法发现;
3)管控通道的稳定性有着一定的不确定性,无法和数据面一样通过流量去实时反馈,很多场景要需要线上触发特殊的场景才能验证,因此巡检的时效性就显得特别重要;
4)很多巡检的系统只是去发现问题,却没有实时的去解决,这样出了故障再到运维人员那,对于一些重要的系统有可能在中途因为这些原因发生更大的故障,所有还需要巡检系统在发现问题先进行一些初步的判断和恢复,对非系统缺陷的问题进行解决。
本部分的目的在于概述本发明的实施例的一些方面以及简要介绍一些较佳实施例。在本部分以及本申请的说明书摘要和发明名称中可能会做些简化或省略以避免使本部分、说明书摘要和发明名称的目的模糊,而这种简化或省略不能用于限制本发明的范围。
因此,为解决上述技术问题,本发明提供如下技术方案:一种保障广域云网管控面稳定的方法,包括以下几个具体的步骤:
S1:对整体的业务进行权重分级;
S2:巡检任务开始后,任务根据管控通道的水位阈值决定是否下发,当水位超过了设置的阈值,则任务进入了待下发队列;
S3:巡检任务的优先级和权重有自动设置和手动设置两种方式;手动设置就是设置任务的权值和和次数,自动设置就是根据监控和历史数据来计算出接口的不稳定参数以及通道系数,两者互相作用,全面的决定每个接口的巡检优先级;
S4:当巡检出故障后,按照故障的类型进行处理;
S5:故障处理完成后,根据监控参数影响权重优先级的调整,该用例优先与该业务的其他业务优先执行,第一时间内验证修改效果,并在一定时间内,提升该类问题的优先级。
作为本发明所述保障广域云网管控面稳定的方法的一种优选方案,其中:在步骤S1中,权重分级分为多个维度,按照维度的优先级和权重、以及环境的影响来确定巡检时单个业务运行的顺序和次数。
作为本发明所述保障广域云网管控面稳定的方法的一种优选方案,其中:在步骤S2中,先按照顺序进行后续任务的下发,待过了设置时间,再去待下发队列里按照任务的先后顺序进行转发。
作为本发明所述保障广域云网管控面稳定的方法的一种优选方案,其中:在步骤S3中,手动设置根据业务的影响自动提升权重;自动设置根据当前监控的情况去周期性的生成一个优化后的巡检任务,待相关人员确定是否更新新任务。
作为本发明所述保障广域云网管控面稳定的方法的一种优选方案,其中:在步骤S4中,故障的类型包括下发链路故障、数据异常故障、系统故障等,系统通过不同的故障类型结合系统的特点进行处理。
作为本发明所述保障广域云网管控面稳定的方法的一种优选方案,其中:在步骤S5中,当巡检发生故障时,若一时解决不了,还可通过紧急规避手段来临时变更,从而达到避免线上故障的情况,具体步骤如下:
S51、当出现故障情况时,该接口先自动下发三次,避免存在因网络抖动所造成的故障;若下发成功,则作为不稳定接口进行记录,不记录故障;
S52、若下发不成功,判断出现故障,系统根据报错依次判断是否为链路故障、数据异常故障、系统缺陷等;
S53、判断、处理链路故障报警;
S531、链路故障主要表现的是超时;如果发生超时,第一优先级是重试,在重试的同时,系统对对端进行拨测,通过心跳信息对端机器监控等多维度数据判断对端服务是否正常;如果重试了三次还不成功,结合系统的判断综合分析得出链路不通引起的故障,此时,系统切换到备用链路进行下发,并且主动发起链路告警,通知相关的运维人员进行修复;
S532、如果备用链路下发失败,则说明该地域控制器可能出现故障,此时,通过该地域控制器所对应的备用地域控制器进行配置下发;同时,主动发出告警,通知相关的运维人员进行修复;
S533、如果备用地域控制器也下发失败,则判断控制面网络大范围故障或者系统缺陷,此时,紧急告警运维人员,进行重点修复;
S54、数据异常故障与解决;
若接口报错,则优先判断是否是数据异常引起的故障;
上述数据问题主要是指上传通过了控制面数据的校验,但是没有通过底层网元层的校验的情况,此类问题主要是由于上下层数据不一致引起的;此时,系统重新触发自动对账功能,若对账数据不一致,系统将对账结果发给运维人员,验证是否需要同步数据;若需要同步数据,则在数据同步后重试,查看问题是否解决;
S55、系统缺陷;
若接口报错,切数据和链路都无异常,且已经进行了参数校验和业务校验操作,此时判断系统缺陷,进行告警,并通知相关的运维值班人员进行定位解决。
本发明还提供了上述一种保障广域云网管控面稳定的系统,其特征在于:该系统包括巡检用例平台、待下发队列、策略设置平台、日志收集与监控、巡检策略配置模块、故障处理模块和数据对账;
其中,巡检用例平台用于管理巡检用例,用户通过巡检用例平台管理每个任务的巡检用例,并对相关的用例进行修改和增加,以适应平台的变动;
待下发队列用于在管控下发通道水位过高一直处于繁忙状态,导致巡检用例得不到及时执行时,巡检用例先进入待下发队列,使得系统隔一段时间对待下发队列中的任务进行重新处理;
策略设置平台用于用户通过该平台来设置相关业务优先级,从而影响整个巡检用例平台的设置;
日志收集与监控主要用于收集全局控制器和地域控制器针对各条业务下发的配置,从而影响整个下发任务;
巡检策略配置模块,通过策略设置平台去主动设置或通过监控系统去根据当前情况计算出优化后的方案;
故障处理模块主要用于在巡检中发现问题时,通过立刻启动故障处理模块及时处理,主要包括线路不通、业务故障等问题;
数据对账主要用于处理数据不一致、或者脏数据或数据缺失所造成的影响。
作为本发明所述保障广域云网管控面稳定的系统的一种优选方案,其中:在策略设置平台中,用户通过设置权重和次数以及其他因素来算出该用例的优先级,巡检用例得分的公式为((业务权重/次数)+通道系数)*不稳定系数,通过该公式算出的得分越高的巡检用例越先执行。
作为本发明所述保障广域云网管控面稳定的系统的一种优选方案,其中:在策略设置平台中,业务权重和次数是指将业务分为核心业务、重点业务、一般业务和测试业务;
其中,核心业务是指覆盖了60%以上客户的业务和90%以上的VIP客户的业务,核心业务权重为1000,次数为10次;重点业务是指除核心业务的进一步业务覆盖,包含了80%以上的业务和所有VIP业务,重点业务权重为400,次数为5次;一般业务是包括所有线上正在使用的业务,权重为120,次数为2次;测试业务指的是测试过程中的新业务,权重为50,次数为1次;
通道系数为链路的重要程度,按照链路的业务量来决定的,特别重要的系数为100,一般重要的为50,用的次数比较少的为10;
不稳定系数由监控重点项、失败次数和平均无故障次数决定的,一般监控重点项告警一次,初始值为1,且不能小于1,系数增加0.01;请求失败一次,系数增加0.02,平均6小时无故障,系数降低0.1。
作为本发明所述保障广域云网管控面稳定的系统的一种优选方案,其中:巡检策略配置模块是整个系统的核心模块,在巡检策略配置模块中,优化后的方案需要运维人员确认后才能切换。
本发明的有益效果:
1、本方案对巡检用例的优先级进行了优化,可以按照功能的重要程度来设置每轮巡检中用例的权重和次数,同时,还可以通过监控数据自动的去对相关用例的优先级去进行一定范围内的调整,能最大范围内运行巡检资源。
2、本方案中的巡检系统下发巡检用例时,会严格按照监控系统中关于各管控通道下各机器的水位来执行,如果相关通道水位过高,则用例进入待下发队列,过段时间重新校验,避免影响线上正常业务的下发。
3、本方案中的巡检系统注重重要业务的故障优先发现的实时性,通过加权和监控修改共同作用,让重要的业务、故障恢复的业务、运行不稳定的业务优先巡检、多次巡检,这样就可以及时发现系统问题,优先规避和解决。
为了更清楚地说明本发明实施例的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其它的附图。其中:
图1为现有技术中云网络管控面下发流程图。
图2为本发明的系统架构图。
图3为本发明的系统流程图。
图4为本发明的故障处理流程图。
为使本发明的上述目的、特征和优点能够更加明显易懂,下面结合说明书附图对本发明的具体实施方式做详细的说明。
在下面的描述中阐述了很多具体细节以便于充分理解本发明,但是本发明还可以采用其他不同于在此描述的其它方式来实施,本领域技术人员可以在不违背本发明内涵的情况下做类似推广,因此本发明不受下面公开的具体实施例的限制。
其次,此处所称的“一个实施例”或“实施例”是指可包含于本发明至少一个实现方式中的特定特征、结构或特性。在本说明书中不同地方出现的“在一个实施例中”并非均指同一个实施例,也不是单独的或选择性的与其他实施例互相排斥的实施例。
本方案重点解决云网络管控面稳定的问题,由于网络的日益复杂和业务容量的急剧增加,线上巡检的工作量和难度越来越大,如果频繁的进行巡检,则会占用大量的管控通道资源,从而对管控面下发进行影响,同时,如果巡检次数过少,则巡检的效用就会降低,因此,设计一种合理的巡检方法就显得非常重要了;
巡检的作用是主动发现系统的问题,而在大系统中利用有限的资源去及时发现问题是一个很有价值的问题,先于客户发现问题,可以有效的减少现网故障,提高系统的稳定性;巡检出相关的问题后,还需要快速的解决方案,针对可能出现的相关问题进行针对性的手段快速解决,这样才能达到减少故障;巡检不能影响线上业务,系统还需要严格设计相关的机制,不能让巡检任务影响正常业务的下发。
参照图2~4,为本发明的实施例,提供了一种保障广域云网管控面稳定的系统来解决上述技术问题,如图2所示,具体如下:
巡检用例平台:此平台为管理巡检用例的平台,用户可以在这里管理每个任务的巡检用例,并对相关的用例进行修改和增加,以适应平台的变动;
待下发队列:指的是由于管控下发通道由于水位过高一直繁忙,导致巡检用例得不到及时执行,因此先进入待下发队列,系统会隔一段时间对待下发队列种的任务进行重新处理;
策略设置平台:用户可以通过该平台来设置相关业务优先级,从而影响整个巡检用例平台的设置;
日志收集与监控:主要收集全局控制器和地域控制器针对各条业务下发的配置,从而影响整个下发任务;
巡检策略配置模块,是整个系统的核心模块,可以通过策略设置平台去主动设置,也可以通过监控系统去根据当前情况计算出优化后的方案,但是优化后的方案需要运维人员确认后才能切换;
故障处理:主要是巡检如果发现问题即会立刻启动故障处理模块,能够及时处理诸如线路不通、业务故障等问题;
数据对账:主要是处理数据不一致、或者脏数据或数据缺失所造成的影响。
以下以一个实施例来说明该系统的运行:
如图1所示,现有技术通过云平台发起任务,然后下发到全局控制器,再由全局控制器通过该配置需要下发的网元设备,去找到相对应的地域控制器,然后通过地域控制器去下发配置到对应的网元。
首先是业务的设置,用户通过设置权重和次数以及其他因素来算出该用例的优先级,巡检用例得分的公式为((业务权重/次数)+通道系数)*不稳定系数;
通过该公式算出的得分越高的巡检用例越先执行。
下面重点介绍该方案的细则:
1、业务权重和次数:将业务分为核心业务,重点业务,一般业务,测试业务,核心业务一般是覆盖了60%以上客户的业务和90%以上的VIP客户的业务,核心业务权重为1000,次数为10次,重点业务是指除核心业务的进一步业务覆盖,包含了80%以上的业务和所有VIP业务,重点业务权重为400,次数为5次,一般业务是包括所有线上正在使用的业务,权重为120,次数为2次,测试业务指的是测试过程中的新业务,权重为为50,次数为1次;
2、通道系数为链路的重要程度,按照链路的业务量来决定的,特别重要的系数为100,一般重要的为50,用的次数比较少的为10;
3、不稳定系数由监控重点项,失败次数和平均无故障次数决定的,一般监控重点项告警一次,初始值为1,且不能小于1,系数增加0.01,请求失败一次,系数增加0.02,平均6小时无故障,系数降低0.1;
以上示例只是为了更好的说明该方案的细节,具体参数可以根据实际情况自行调整。
具体实施例如下:
比如有a业务和b业务两个业务,a业务是重点业务,咱们可以将其设置权重为400,次数选5,而b业务是一般业务,设置权重为120,次数选2,假设两个业务的接口稳定性和通道都是一样的话,通道系数都为0,不稳定系数都为1,那么a业务5次的得分为400,320,240,160,80,b为120,60,这样就可以先进行a四次,再进行任务b一次,再进行a一次,最后再进行b一次,通过这种设置,可以让重要的业务优先执行,并且在一个周期内多次运行保障;
通道系数是指更重要的通道加权,当通道更重要时,相关的权重就会增加,相关功能的执行优先级就会上升;
不稳定系数根据系统的情况去实时反馈,管理员可以让系统自动根据网络情况去实时优化,将巡检用例的验证更多的倾向于接口不稳定和通道系数差的接口,从而更能发现问题。
比如业务a有两条下发链路,分别下发到a1和b1网元,但是根据监控,到b1网元的路径更差,这样b1网元的不稳定系统就会变高,测b1网元的优先级就会变高,从而更早测,更发现问题,同时,随着链路的完善,无故障时间的增长,不稳定系数又会自动降低,从而使得该链路回归到正常的预定优先级;
设置好任务下发的优先级后,就是下发的过程了,但是巡检的下发又不能影响正常业务的下发,所以每次巡检任务下发前都会检查相关下发通道的负载水位,当相关的负载水位超过了某个阀值后,该任务就会进入待下发队列,过一段时间再进行判断,如果该任务一直没有进行下发,超过一定的次数也会进行报警,管控通道一直水位偏高也有可能有问题;
当任务进行下发后,就可以通过全局控制器去找到需要下发到网元所属的地域控制器,再由地域控制器下发到具体的网元,同时,日志和监控系统会记录每一条下发任务的成功率,性能等参数,形成相关的报告进行输出,对于重要的任务和接口,也会形成相关的告警信息输出,利于及时发现问题;
同时,测试信息也会结合线上的数据,去对每个接口和相关管控通道进行评估,重新计算出通道的通道参数和接口的不稳定参数,从而通过上面的公式来影响相关接口的测试优先级;
任务分为手动设置和自动设置两种方式,手动设置就是设置任务的权值和和次数,自动设置就是根据监控和历史数据来计算出接口的不稳定参数,两者互相作用,全面的决定每个接口的巡检优先级。
如图4所示,当巡检发生故障时,要是一时解决不了,也可以通过紧急规避手段来临时变更,从而达到避免线上故障的情况:
1、由于前端有相关的参数校验和业务校验,所以真正到达下发业务流程的请求,都认为该请求是合法的,因此这时系统的原则都是尽量下发该请求,而系统在上线流程中一般都是经过了充分验证的,因此会认为真正的系统缺陷是非常少的,该系统重点会识别和自动规避非系统缺陷的问题,保障系统的正常业务下发。
2、解决思路如下:当出现故障情况时,首先该接口会自动下发三次,避免因网络抖动所造成的故障,如果有成功的,则可作为不稳定接口的记录,不记录故障。
3、当系统真出现故障时,系统会根据报错依次判断是否为链路故障、数据异常故障、系统缺陷等。
4、链路故障报警判断与处理:
1)链路故障主要表现的是超时,如果发生超时,第一优先级是重试,在重试的同时,系统会对对端进行拨测,以及通过心跳信息,对端机器监控等多维度来判断对端服务是否正常,如果重试了3次还不成功,并结合系统的判断综合分析得出是链路不通引起的,则切换到备用链路进行下发,并且主动发起链路告警,通知相关的运维人员进行修复;
2)还有一种情况就是如果备用链路也不行,则说明该地域控制器可能出现故障,这时可以通过该地域控制器所对应的备用地域控制器进行配置下发,同时,主动发出告警,通知相关的运维人员进行修复;
3)如果备用地域控制器也不行,那大概率是控制面网络大范围故障或者系统缺陷,这时会紧急告警运维人员,进行重点修复。
5、数据异常故障与解决:如果是接口报错,则优先可以判断是否是数据异常引起的;此处的数据问题主要是指上传通过了控制面数据的校验,但是没有通过底层网元层的校验,这种主要是上下层数据不一致引起的;这时,系统会重新触发自动对账,如果对账数据不一致,系统会将对账结果发给运维人员,验证是否需要同步数据,如果确定的话,同步数据后,再重试看问题是否解决。
6、系统缺陷:如果接口报错,数据和链路都无异常的话,且之前已经进行了参数校验和业务校验,此时应该就是真正的系统缺陷,会进行告警,通知相关的运维值班人员进行定位解决。
综上,该系统可以根据业务的重要性设置出相对应的权重值和次数,系统可以通过比较最终计算出的优先级来决定优先执行哪些用例,同一个用例的不同下发通道也是随着用户量的大小和用户的重要优先程度有所不同,因此,还需要设置通道参数来影响测试业务下发的优先级,同时,监控和日志也可以根据通道和接口的成功率来改变相关的结果,让那些稳定性差的接口和通道在一定范围内优先测试,及时发现相关的问题。
同时,系统还提供了常见问题的紧急解决方案,及时解决和更正系统的错误,将运维过程中非系统缺陷100%解决,系统缺陷也尽量得以规避。
综上,该系统可以根据业务的重要性设置出相对应的权重值和次数,系统可以通过比较最终计算出的优先级来决定优先执行哪些用例,同一个用例的不同下发通道也是随着用户量的大小和用户的重要优先程度有所不同,因此,还需要设置通道参数来影响测试业务下发的优先级,同时,监控和日志也可以根据通道和接口的成功率来改变相关的结果,让那些稳定性差的接口和通道在一定范围内优先测试,及时发现相关的问题
同时,系统还提供了常见问题的紧急解决方案,及时解决和更正系统的错误,将运维过程中非系统缺陷100%解决,系统缺陷也尽量得以规避
在本方案中:
广域云网:指的是本地设备到远端云VPC之间的网络,包括交换机、网络接入设备,网关、防火墙以及VPC和VPC之间的网络。
管控面:指的是业务变更时,下发配置过程的管控通道,从平台侧,到控制器,再到网元设备管控下发通道。
巡检:指的是对线上系统进行用例检测,实时的去发现线上的问题。
应说明的是,以上实施例仅用以说明本发明的技术方案而非限制,尽管参照较佳实施例对本发明进行了详细说明,本领域的普通技术人员应当理解,可以对本发明的技术方案进行修改或者等同替换,而不脱离本发明技术方案的精神和范围,其均应涵盖在本发明的权利要求范围当中。
Claims (10)
- 一种保障广域云网管控面稳定的方法,其特征在于:包括以下几个具体的步骤:S1:对整体的业务进行权重分级;S2:巡检任务开始后,任务根据管控通道的水位阈值决定是否下发,当水位超过了设置的阈值,则任务进入了待下发队列;S3:巡检任务的优先级和权重有自动设置和手动设置两种方式;手动设置就是设置任务的权值和和次数,自动设置就是根据监控和历史数据来计算出接口的不稳定参数以及通道系数,两者互相作用,全面的决定每个接口的巡检优先级;S4:当巡检出故障后,按照故障的类型进行处理;S5:故障处理完成后,根据监控参数影响权重优先级的调整,该用例优先与该业务的其他业务优先执行,第一时间内验证修改效果,并在一定时间内,提升该类问题的优先级。
- 如权利要求1所述的保障广域云网管控面稳定的方法,其特征在于:在步骤S1中,权重分级分为多个维度,按照维度的优先级和权重、以及环境的影响来确定巡检时单个业务运行的顺序和次数。
- 如权利要求2所述的保障广域云网管控面稳定的方法,其特征在于:在步骤S2中,先按照顺序进行后续任务的下发,待过了设置时间,再去待下发队列里按照任务的先后顺序进行转发。
- 如权利要求3所述的保障广域云网管控面稳定的方法,其特征在于:在步骤S3中,手动设置根据业务的影响自动提升权重;自动设置根据当前监控的情况去周期性的生成一个优化后的巡检任务,待相关人员确定是否更新新任务。
- 如权利要求4所述的保障广域云网管控面稳定的方法,其特征在于:在步骤S4中,故障的类型包括下发链路故障、数据异常故障、系统故障,系统通过不同的故障类型结合系统的特点进行处理。
- 如权利要求5所述的保障广域云网管控面稳定的方法,其特征在于:在步骤S5中,当巡检发生故障时,若一时解决不了,还可通过紧急规避手段来临时变更,从而达到避免线上故障的情况,具体步骤如下:S51、当出现故障情况时,该接口先自动下发三次,避免存在因网络抖动所造成的故障;若下发成功,则作为不稳定接口进行记录,不记录故障;S52、若下发不成功,判断出现故障,系统根据报错依次判断是否为链路故障、数据异常故障、系统缺陷;S53、判断、处理链路故障报警;S531、链路故障主要表现的是超时;如果发生超时,第一优先级是重试,在重试的同时,系统对对端进行拨测,通过心跳信息对端机器监控多维度数据判断对端服务是否正常;如果重试了三次还不成功,结合系统的判断综合分析得出链路不通引起的故障,此时,系统切换到备用链路进行下发,并且主动发起链路告警,通知相关的运维人员进行修复;S532、如果备用链路下发失败,则说明该地域控制器可能出现故障,此时,通过该地域控制器所对应的备用地域控制器进行配置下发;同时,主动发出告警,通知相关的运维人员进行修复;S533、如果备用地域控制器也下发失败,则判断控制面网络大范围故障或者系统缺陷,此时,紧急告警运维人员,进行重点修复;S54、数据异常故障与解决;若接口报错,则优先判断是否是数据异常引起的故障;上述数据问题主要是指上传通过了控制面数据的校验,但是没有通过底层网元层的校验的情况,此类问题主要是由于上下层数据不一致引起的;此时,系统重新触发自动对账功能,若对账数据不一致,系统将对账结果发给运维人员,验证是否需要同步数据;若需要同步数据,则在数据同步后重试,查看问题是否解决;S55、系统缺陷;若接口报错,切数据和链路都无异常,且已经进行了参数校验和业务校验操作,此时判断系统缺陷,进行告警,并通知相关的运维值班人员进行定位解决。
- 如权利要求6所述的保障广域云网管控面稳定的系统,其特征在于:包括巡检用例平台、待下发队列、策略设置平台、日志收集与监控、巡检策略配置模块、故障处理模块和数据对账;其中,巡检用例平台用于管理巡检用例,用户通过巡检用例平台管理每个任务的巡检用例,并对相关的用例进行修改和增加,以适应平台的变动;待下发队列用于在管控下发通道水位过高一直处于繁忙状态,导致巡检用例得不到及时执行时,巡检用例先进入待下发队列,使得系统隔一段时间对待下发队列中的任务进行重新处理;策略设置平台用于用户通过该平台来设置相关业务优先级,从而影响整个巡检用例平台的设置;日志收集与监控主要用于收集全局控制器和地域控制器针对各条业务下发的配置,从而影响整个下发任务;巡检策略配置模块,通过策略设置平台去主动设置或通过监控系统去根据当前情况计算出优化后的方案;故障处理模块主要用于在巡检中发现问题时,通过立刻启动故障处理模块及时处理,主要包括线路不通、业务故障问题;数据对账主要用于处理数据不一致、或者脏数据或数据缺失所造成的影响。
- 如权利要求7所述的保障广域云网管控面稳定的系统,其特征在于:在策略设置平台中,用户通过设置权重和次数以及其他因素来算出该用例的优先级,巡检用例得分的公式为((业务权重/次数)+通道系数)*不稳定系数,通过该公式算出的得分越高的巡检用例越先执行。
- 如权利要求8所述的保障广域云网管控面稳定的系统,其特征在于:在策略设置平台中,业务权重和次数是指将业务分为核心业务、重点业务、一般业务和测试业务;其中,核心业务是指覆盖了60%以上客户的业务和90%以上的VIP客户的业务,核心业务权重为1000,次数为10次;重点业务是指除核心业务的进一步业务覆盖,包含了80%以上的业务和所有VIP业务,重点业务权重为400,次数为5次;一般业务是包括所有线上正在使用的业务,权重为120,次数为2次;测试业务指的是测试过程中的新业务,权重为50,次数为1次;通道系数为链路的重要程度,按照链路的业务量来决定的,特别重要的系数为100,一般重要的为50,用的次数比较少的为10;不稳定系数由监控重点项、失败次数和平均无故障次数决定的,一般监控重点项告警一次,初始值为1,且不能小于1,系数增加0.01;请求失败一次,系数增加0.02,平均6小时无故障,系数降低0.1。
- 如权利要求9所述的保障广域云网管控面稳定的系统,其特征在于:巡检策略配置模块是整个系统的核心模块,在巡检策略配置模块中,优化后的方案需要运维人员确认后才能切换。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410536113.7A CN118488049A (zh) | 2024-04-30 | 2024-04-30 | 一种保障广域云网管控面稳定的方法和系统 |
| CN202410536113.7 | 2024-04-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025227697A1 true WO2025227697A1 (zh) | 2025-11-06 |
Family
ID=92190450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2024/135840 Pending WO2025227697A1 (zh) | 2024-04-30 | 2024-11-29 | 一种保障广域云网管控面稳定的方法和系统 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN118488049A (zh) |
| WO (1) | WO2025227697A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118488049A (zh) * | 2024-04-30 | 2024-08-13 | 天翼云科技有限公司 | 一种保障广域云网管控面稳定的方法和系统 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659135A (zh) * | 2019-09-12 | 2020-01-07 | 苏州浪潮智能科技有限公司 | 一种raid巡检方法、装置、电子设备及可读存储介质 |
| CN113472577A (zh) * | 2021-06-30 | 2021-10-01 | 济南浪潮数据技术有限公司 | 一种集群巡检方法、装置及系统 |
| WO2023103627A1 (zh) * | 2021-12-06 | 2023-06-15 | 中兴通讯股份有限公司 | 网络巡检的方法、装置、电子设备和存储介质 |
| CN117236669A (zh) * | 2023-11-15 | 2023-12-15 | 中铁四局集团有限公司 | 一种基于数据密集型批任务调度的自动巡检策略系统 |
| CN117827608A (zh) * | 2023-12-13 | 2024-04-05 | 天翼云科技有限公司 | 一种基于历史监控数据的智能化预警和处置方法 |
| CN118488049A (zh) * | 2024-04-30 | 2024-08-13 | 天翼云科技有限公司 | 一种保障广域云网管控面稳定的方法和系统 |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117319173A (zh) * | 2023-10-17 | 2023-12-29 | 烽火通信科技股份有限公司 | 一种云平台巡检任务调度方法及系统 |
-
2024
- 2024-04-30 CN CN202410536113.7A patent/CN118488049A/zh active Pending
- 2024-11-29 WO PCT/CN2024/135840 patent/WO2025227697A1/zh active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659135A (zh) * | 2019-09-12 | 2020-01-07 | 苏州浪潮智能科技有限公司 | 一种raid巡检方法、装置、电子设备及可读存储介质 |
| CN113472577A (zh) * | 2021-06-30 | 2021-10-01 | 济南浪潮数据技术有限公司 | 一种集群巡检方法、装置及系统 |
| WO2023103627A1 (zh) * | 2021-12-06 | 2023-06-15 | 中兴通讯股份有限公司 | 网络巡检的方法、装置、电子设备和存储介质 |
| CN117236669A (zh) * | 2023-11-15 | 2023-12-15 | 中铁四局集团有限公司 | 一种基于数据密集型批任务调度的自动巡检策略系统 |
| CN117827608A (zh) * | 2023-12-13 | 2024-04-05 | 天翼云科技有限公司 | 一种基于历史监控数据的智能化预警和处置方法 |
| CN118488049A (zh) * | 2024-04-30 | 2024-08-13 | 天翼云科技有限公司 | 一种保障广域云网管控面稳定的方法和系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118488049A (zh) | 2024-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11614943B2 (en) | Determining problem dependencies in application dependency discovery, reporting, and management tool | |
| US11868237B2 (en) | Intelligent services for application dependency discovery, reporting, and management tool | |
| US11675692B2 (en) | Testing agent for application dependency discovery, reporting, and management tool | |
| US11620211B2 (en) | Discovery crawler for application dependency discovery, reporting, and management tool | |
| US20250036488A1 (en) | Dependency analyzer in application dependency discovery, reporting, and management tool | |
| US20250103475A1 (en) | Intelligent Services and Training Agent for Application Dependency Discovery, Reporting and Manageme | |
| US11379292B2 (en) | Baseline modeling for application dependency discovery, reporting, and management tool | |
| US12181962B2 (en) | Intelligent network operation platform for network fault mitigation | |
| CN119011374B (zh) | 一种基于设备同步的主备切换方法及系统 | |
| CN118331779B (zh) | 分布式系统故障判断与恢复方法、应用该方法的云操作系统以及计算平台 | |
| CN110430071A (zh) | 业务节点故障自愈方法、装置、计算机设备及存储介质 | |
| EP3991030B1 (en) | Determining problem dependencies in application dependency discovery, reporting, and management tool | |
| CN116719664A (zh) | 基于微服务部署的应用和云平台跨层故障分析方法及系统 | |
| CN119071140B (zh) | 一种计算机网络的故障诊断方法 | |
| CN118488049A (zh) | 一种保障广域云网管控面稳定的方法和系统 | |
| CN120508515A (zh) | 一种数据传输方法、装置、设备及介质 | |
| CN120653626A (zh) | 一种无感知数据动态迁移方法 | |
| CN111277805B (zh) | 一种针对olt上联链路crc错误的处理方法及装置 | |
| CN120710873B (zh) | 网卡管理方法、系统、计算机设备、存储介质及程序产品 | |
| CN119011363B (zh) | 一种校园网认证计费系统故障自动处置方法及系统 | |
| CN121000629A (zh) | 网络设备的设备性能测试方法、装置、存储介质及设备 | |
| CN121807643A (zh) | 一种数据中台的主动式智能运维监控方法 | |
| CN120821644A (zh) | 一种系统故障自动检测方法及存储介质 | |
| CN120929370A (zh) | 一种自动化测试方法、装置、电子设备及存储介质 | |
| CN121984866A (zh) | 一种回滚方法、装置、设备和介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24937726 Country of ref document: EP Kind code of ref document: A1 |