CN112732999B - Static disaster recovery method, system, electronic equipment and storage medium - Google Patents
Static disaster recovery method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112732999B CN112732999B CN202110080886.5A CN202110080886A CN112732999B CN 112732999 B CN112732999 B CN 112732999B CN 202110080886 A CN202110080886 A CN 202110080886A CN 112732999 B CN112732999 B CN 112732999B
- Authority
- CN
- China
- Prior art keywords
- disaster recovery
- url
- static
- state
- user request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention provides a static disaster recovery method, a static disaster recovery system, electronic equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: after receiving a user request, judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state; the disaster recovery state of the URL is updated according to log monitoring data and disaster recovery triggering conditions, and the log monitoring data is generated according to access logs; if the user request URL service is configured with the disaster recovery state, reversely proxy the user request to a disaster recovery cache of the static disaster recovery system; the disaster recovery buffer of the static disaster recovery system is buffered with buffer data periodically crawled from the URL request history record in the access log; and if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to the back-end server. The static disaster recovery method can enable the static disaster recovery system to have dynamic configuration supporting capability, intelligent monitoring switching capability and unified intelligent checking capability.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a static disaster recovery method, a static disaster recovery system, an electronic device, and a storage medium.
Background
Aiming at the problems of system service stability reduction and even service breakdown, most enterprises can make precautionary measures from the aspects of system, process and system support, and common methods in the aspect of the system comprise service degradation and fault tolerance design. For example, systems can generally be classified into 3 classes, from simple to complex, depending on the different circumstances in which the system services use the cache. The first type uses a simple cached system for service, and the probability of occurrence of abnormality of the cached system is low. The second type is a system that uses complex caches for services, which are typically controlled by the business layer, subject to application layer code. If the application code has problems, the cache can not be correctly called, and the page is displayed abnormally. The third class is systems where the service does not use caching at all. The first type of system has low abnormal probability and can not be subjected to fault-tolerant design. The probability of anomalies occurring in the second and third class of systems is high. For the second type of system, the overall stability of the system can be improved by introducing an intelligent static disaster recovery system. For the third class of systems, the stability cannot be improved by introducing a static disaster recovery system due to the dynamic service characteristic.
At present, in a mode of improving the overall stability of a second type of system by introducing an intelligent static disaster recovery system, one scheme is to introduce a fault tolerance mechanism at an application code layer, for example, a response time threshold is set, and a timeout mechanism is triggered when service response is slow, so that fault tolerance degradation is performed. A common practice for fault tolerance degradation is to uniformly return a default data template, and all users see the same data, degrading from thousands of people to thousands of people. The implementation scheme of the static disaster recovery system mainly relates to developers, and can solve problems in most scenes, but the following hidden troubles exist in actual production and application: 1) Due to the instability of the application itself, an overall "hang-up" of the application may occur; 2) Each application service needs to introduce a fault-tolerant mechanism, fault-tolerant codes are dispersed in different applications, and unified upgrading and maintenance are difficult; 3) The scattered fault tolerant code is difficult to test and the overall static disaster recovery capability is difficult to evaluate.
Disclosure of Invention
Based on the problems existing in the prior art, the embodiment of the invention provides a static disaster recovery method, a system, electronic equipment and a storage medium.
In a first aspect, an embodiment of the present invention provides a static disaster recovery method, where the method is applied to a static disaster recovery system, and the method includes: after receiving a user request, judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state; the disaster recovery state of the URL is updated according to log monitoring data and disaster recovery triggering conditions, and the log monitoring data is generated according to access logs; if the user request URL service is configured with a disaster recovery state, reversely proxy the user request to a disaster recovery cache of the static disaster recovery system; the disaster recovery buffer of the static disaster recovery system is buffered with buffer data periodically crawled from the URL request history record in the access log; and if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to a back-end server.
Optionally, before the determining whether the user requests the URL to configure the disaster recovery status, the method further includes: judging the switching state of the static disaster recovery system, wherein the switching state of the static disaster recovery system comprises on or off; if the switch state of the static disaster recovery system is on, judging whether the URL service requested by the user is configured with a disaster recovery state or not; and if the switch state of the static disaster recovery system is closed, reversely proxy the user request to a back-end server.
Optionally, after reversely proxy the user request into the backend server, the method further comprises: judging whether the returned result of the back-end server is a server error or not; and if the returned result of the back-end server is a server error, reversely proxy the user request to the disaster recovery cache of the static disaster recovery system.
Optionally, the method further comprises: and if the returned result of the back-end server is a normal result, returning the normal result to the user.
Optionally, the method further comprises: and acquiring a URL list supporting static disaster recovery, periodically crawling cache data from a URL request history record in an access log, and storing the cache data into a disaster recovery cache.
Optionally, the method further comprises: reading a URL service list which needs disaster recovery; reading log monitoring data; and updating the disaster recovery state of the URL in the URL service list according to the log monitoring data and the disaster recovery triggering condition.
Optionally, updating the disaster recovery status of the URL according to the log monitoring data and the disaster recovery triggering condition includes: judging the state of the URL service in the URL service list according to the log monitoring data and the disaster recovery triggering condition; if the state of the URL service is abnormal, setting the URL service as a disaster recovery state, wherein the URL service is set as the configured disaster recovery state after the disaster recovery state; and if the state of the URL service is recovered abnormally, clearing the disaster recovery state of the URL service, wherein the cleared disaster recovery state of the URL service is an unconfigured disaster recovery state.
Optionally, the disaster recovery triggering condition includes: the response time is greater than or equal to the average response time threshold or tp90 response time threshold.
Optionally, before the log monitoring data is read, the method further includes: and collecting and analyzing the access log to generate log monitoring data.
Optionally, the reading the URL service list requiring disaster recovery includes: reading the URL service list needing disaster recovery from the pre-configured key configuration information; the URL service list requiring disaster recovery comprises resource names, URLs, disaster recovery triggering conditions and contacts corresponding to URL services requiring static disaster recovery; the URL service list requiring disaster recovery is configured into the static disaster recovery system after the URL service requiring disaster recovery is manually analyzed and identified.
Optionally, before the periodically crawling the cache data from the URL request history in the access log, the method further includes: and performing deduplication processing on the URL request history record in the access log.
Optionally, the key cached in the disaster recovery cache of the static disaster recovery system is a url+time version; the static disaster recovery system is used for crawling cache data after collecting the URL request history record data of the first time, and is used for generating different time versions according to crawling frequency corresponding to the resources.
The embodiment of the invention has at least the following beneficial effects:
1) Has dynamic configuration supporting capability. The dynamic configuration of the support disaster recovery is allowed to be carried out on the URL service, and the system can newly add the static disaster recovery of the support service without restarting.
2) The intelligent monitoring switching capability is provided. And judging the stability of the URL service according to the response state of the request, and automatically proxy the user request into a static disaster-tolerant cache system when an abnormal situation occurs.
3) The system has unified intelligent checking capability. The system can be uniformly managed and maintained, and relevant tests can be carried out to evaluate the static disaster recovery capability of the system.
In a second aspect, an embodiment of the present application further provides a static disaster recovery system, where the static disaster recovery system includes: the system comprises a reverse proxy module, a log analysis module, a disaster recovery configuration module, a crawler module and a disaster recovery cache module; the crawler module is used for acquiring a URL list supporting static disaster recovery from the reverse proxy module, and periodically crawling cache data from a URL request history record in an access log to the disaster recovery cache module; the disaster recovery cache module caches the cache data crawled by the crawler module; the log analysis module is used for generating log monitoring data according to the access log; the disaster recovery configuration module is used for updating the disaster recovery state of the URL service to the reverse proxy module according to the log monitoring data and the disaster recovery triggering condition; the reverse proxy module is used for judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state after receiving the user request; if the user request URL service is configured with a disaster recovery state, reversely proxy the user request to the disaster recovery cache module; and if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to a back-end server.
Optionally, the reverse proxy module is further configured to determine, before determining whether the user requests the URL to configure the disaster recovery state, a switch state of the static disaster recovery system, where the switch state of the static disaster recovery system includes on or off; if the switch state of the static disaster recovery system is on, judging whether the URL service requested by the user is configured with a disaster recovery state or not; and if the switch state of the static disaster recovery system is closed, reversely proxy the user request to a back-end server.
Optionally, the reverse proxy module is further configured to determine whether a returned result of the backend server is a server error after reversely proxy the user request into the backend server; and if the returned result of the back-end server is a server error, reversely proxy the user request to the disaster recovery cache of the static disaster recovery system.
Optionally, the reverse proxy module is further configured to return the normal result to the user if the returned result of the back-end server is the normal result.
Optionally, the disaster recovery configuration module is specifically configured to read a URL service list of a disaster to be recovered; reading log monitoring data; and updating the disaster recovery state of the URL in the URL service list to the reverse proxy module according to the log monitoring data and the disaster recovery triggering condition.
Optionally, the disaster recovery configuration module is specifically configured to determine a state of URL service in the URL service list according to the log monitoring data and the disaster recovery trigger condition; if the state of the URL service is abnormal, requesting the reverse proxy module to set the URL service into a disaster recovery state, wherein the URL service is set into the configured disaster recovery state after the disaster recovery state; and if the state of the URL service is recovered abnormally, requesting the reverse proxy module to empty the disaster recovery state of the URL service, wherein the disaster recovery state of the URL service is an unconfigured disaster recovery state after being emptied.
Optionally, the disaster recovery triggering condition includes: the response time is greater than or equal to the average response time threshold or tp90 response time threshold.
Optionally, the log analysis module is specifically configured to collect and analyze the access log, and generate log monitoring data.
Optionally, the disaster recovery configuration module is specifically configured to read the URL service list required to be disaster recovery from the preconfigured key configuration information; the URL service list requiring disaster recovery comprises resource names, URLs, disaster recovery triggering conditions and contacts corresponding to URL services requiring static disaster recovery; the URL service list requiring disaster recovery is configured into the disaster recovery configuration module after the URL service requiring disaster recovery is manually analyzed and identified.
Optionally, the crawler module is further configured to perform deduplication processing on the URL request history record in the access log.
Optionally, the key cached in the disaster recovery configuration module is a url+time version; the crawler module is used for crawling cache data after collecting URL request history record data of a first duration, and the disaster recovery configuration module is used for generating different time versions according to crawling frequency of the crawler module corresponding to resources.
In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor, a storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over a bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method as described in the first aspect when executed.
In a fourth aspect, embodiments of the present invention provide a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method according to the first aspect.
The advantages of the second to fourth aspects may be referred to in the first aspect, and are not described here.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a schematic structural diagram of a static disaster recovery system provided in an embodiment of the present application;
fig. 2 shows a schematic process flow diagram of a reverse proxy module provided in an embodiment of the present application;
fig. 3 is a schematic process flow diagram of a disaster recovery configuration module according to an embodiment of the present application;
fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention, and it should be understood that the drawings in the present invention are for the purpose of illustration and description only and are not intended to limit the scope of the present invention. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.
In addition, the described embodiments of the invention are only some, but not all, embodiments of the invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention.
It should be noted that the term "comprising" will be used in embodiments of the invention to indicate the presence of the features stated hereafter, but not to exclude the addition of other features. It should also be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. In the description of the present invention, it should also be noted that the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and should not be construed as indicating or implying relative importance.
The stability of the service operation of the system is always pursued by enterprises, but there are always some accidents that can cause the service stability of the system to be reduced or even the service to crash. For example, common factors that negatively impact system service stability include: 1) Lack of performance pressure measurement; 2) High availability designs lacking service; 3) Lack of fault tolerance mechanisms; 4) Capacity estimation is insufficient, and a large amount of slow queries appear under high concurrency requests; 5) The machine room is powered off, the network is disconnected, and the server hardware fails; 6) Frequent replacement of application versions, etc.
Aiming at the problems of system service stability reduction and even service breakdown caused by factors 1) to 6), most enterprises can make precautionary measures from the aspects of system, process and system support, and common methods in the aspect of the system comprise service degradation and fault tolerance design.
For example, systems can generally be classified into 3 classes, from simple to complex, depending on the different circumstances in which the system services use the cache. The cache refers to a data pool for storing hot spot data (frequently used data), and is generally used for accelerating data acquisition or reducing pressure on a system caused by directly reading original data.
The first type uses a system of simple caching for services, and can directly configure content delivery network (Content Delivery Network, CDN) caching, and the cached data generally has no business logic, such as style files, js library files, pictures, audios and videos, various documents, and the like. Overall, the probability of such a cache system abnormality is low. The second type is a system in which complex cache is used for services, for example, thousands of people and thousands of people who are popular at present, different users who access the system services can be identified through application of big data and related technologies, different response contents can be displayed for the same request of different users, namely, different results can be displayed for the same request according to different users, and the purpose of accurate marketing is achieved. These caches are typically controlled by the business layer and are affected by the application layer code. If the application code has problems, the cache can not be correctly called, and the page is displayed abnormally. The third category is systems where the service does not use a cache at all, such as the service of placing orders, subtracting stock, paying, returning, etc. that needs to be persisted into the database.
Among the three types of systems, the first type of system has low abnormal probability and does not need fault-tolerant design. The probability of anomalies occurring in the second and third class of systems is high. For the second type of system, the overall stability of the system can be improved by introducing an intelligent static disaster recovery system. For the third class of systems, the stability cannot be improved by introducing a static disaster recovery system due to the dynamic service characteristic. The static disaster recovery means that when the service is abnormal, static data is used for access.
Currently, in a manner of improving the overall stability of the second-class system by introducing an intelligent static disaster recovery system, the static disaster recovery system generally has the following two implementation schemes.
The first scheme is to introduce a fault-tolerant mechanism at the application code layer, for example, a response time threshold is set, and if the service response is slow, a timeout mechanism is triggered to perform fault-tolerant degradation. A common practice for fault tolerance degradation is to uniformly return a default data template, and all users see the same data, degrading from thousands of people to thousands of people.
The implementation scheme of the static disaster recovery system mainly relates to developers, and can solve problems in most scenes, but the following hidden troubles exist in actual production and application: 1) Due to the instability of the application itself, an overall "hang-up" of the application may occur; 2) Each application service needs to introduce a fault-tolerant mechanism, fault-tolerant codes are dispersed in different applications, and unified upgrading and maintenance are difficult; 3) The scattered fault tolerant code is difficult to test and the overall static disaster recovery capability is difficult to evaluate.
The second scheme is to adopt a certain screening and updating strategy, store the response data of the user request into an independent cache system, enable the cache area to have hot data, and switch the user request into the independent cache system when abnormality occurs.
The implementation scheme of the static disaster recovery system mainly relates to operation and maintenance personnel, the static disaster recovery system has no service logic, the stability is only related to hardware equipment, and the single-theory stability is greatly improved compared with the implementation scheme of the first static disaster recovery system. However, the disadvantage is that in extreme cases, the cached data is erroneous, and the return of the erroneous data to the user does not perform a static disaster recovery function.
Aiming at the problems existing in the implementation scheme of the existing static disaster recovery system, the embodiment of the invention provides the static disaster recovery system, which can provide dynamic configuration supporting capability for the system while improving the stability of the second class system service (using complex caches), and realize intelligent monitoring switching and unified intelligent checking of the system.
Wherein, providing dynamic configuration support capability for the system refers to allowing the system to dynamically configure a uniform resource locator (Uniform Resource Locator, URL) service supporting disaster recovery, and the system can add a static disaster recovery of the supporting service without restarting. The intelligent monitoring switching of the system is that the stability of the URL service can be judged according to the response state of the request, and when abnormal conditions occur, the user request can be automatically proxied into the static disaster-tolerant cache system. The unified intelligent checking means that the static disaster recovery system can be managed and maintained in a unified mode, and the static disaster recovery capability of the system is evaluated.
The static disaster recovery system provided by the embodiment of the application is exemplified below with reference to the accompanying drawings.
Fig. 1 shows a schematic structural diagram of a static disaster recovery system provided in an embodiment of the present application. As shown in fig. 1, the static disaster recovery system includes: the system comprises a reverse proxy module, a log analysis module, a disaster recovery configuration module, a crawler module and a disaster recovery cache module.
The reverse proxy module is used for distributing the user request and judging that the user request is sent to the back-end server or the static disaster recovery system.
The log analysis module is used for uniformly collecting, centrally managing and analyzing the access logs so as to monitor the real-time state of the URL service, such as the average value of response time, tp90, tp99 and the like. Illustratively, tp99 is generally used to embody the response capability of the service, i.e. 99% of the requests can get a response within this point in time.
The disaster recovery configuration module is configured with key configuration information, wherein the key configuration information comprises resource names, URLs, disaster recovery triggering conditions, contacts and the like corresponding to URL services needing static disaster recovery. The key configuration information can be configured into the disaster recovery configuration module after the URL service of the static disaster recovery is identified by manual analysis.
The crawler module is used for acquiring a URL list supporting static disaster recovery from the reverse proxy module, performing deduplication processing on a URL request history record (including parameters) in the access log, and crawling data regularly.
The disaster recovery buffer module is used for storing the buffer data, and the buffered key is URL+time version. Specifically, the crawler module performs crawling after collecting the URL request history data within a certain time, so that the cache module can generate different time versions for the resource according to the crawling frequency.
Introducing a temporal version in the disaster recovery cache module can solve the following problems: in an extreme case, the disaster recovery system data is erroneous (e.g. the back-end service returns 200 status code, but the response is empty or erroneous), at which time the URL cache may be switched to the previous temporal version data. The most recent temporal version data may be used when a request has no static disaster recovery cache within the most recent temporal version.
The specific implementation logic of the reverse proxy module and the disaster recovery configuration module are respectively described below.
Fig. 2 shows a schematic process flow diagram of a reverse proxy module provided in an embodiment of the present application.
As shown in fig. 2, the process flow of the reverse proxy module may include:
s201, receiving a user request.
S202, judging whether the static disaster recovery system is started.
If the switch state of the static disaster recovery system is on, executing S203; if the switch state of the static disaster recovery system is off, S205 is executed.
S203, judging whether the user requests the URL service to be configured with the disaster recovery state.
If the user requests the URL service to have configured the disaster recovery status (i.e., if yes), then S204 is performed; if the user requests the URL service to not configure the disaster recovery status (i.e., if not), S205 is performed.
S204, reversely proxy the user request to the disaster recovery cache module.
S205, reversely proxy the user request to the back-end server.
If S205 is executed, S206 is continued.
S206, judging whether the returned result of the back-end server is a server error.
If the return result of the back-end server is a server error, executing S204; if the returned result of the back-end server is a normal result, ending and returning the normal result to the user.
It can be appreciated that if the user request is reversely proxied into the disaster recovery cache module, the cache data in the disaster recovery cache module is returned to the user.
Fig. 3 is a schematic process flow diagram of a disaster recovery configuration module according to an embodiment of the present application.
As shown in fig. 3, the processing flow of the disaster recovery configuration module may include:
s301, reading a URL service list requiring disaster recovery.
S302, reading log monitoring data.
S303, judging the state of the URL service in the URL service list according to the log monitoring data and the disaster recovery triggering condition.
If the state of the URL service is changed from normal to abnormal (i.e., an abnormality occurs), S304 is performed; if the state of the URL service is changed from abnormal to normal (i.e., abnormal recovery), S305 is performed.
S304, requesting the reverse proxy module to set the URL service to be in a disaster recovery state.
S305, requesting the reverse proxy module to clear the disaster recovery state of the URL service.
Optionally, the common disaster recovery trigger conditions are related to response time, including: an average response time threshold; tp90 response time threshold.
The static disaster recovery system provided by the embodiment of the invention has a high-availability, high-stability and intelligent service system, can provide degradation and fault tolerance functions when the back-end service is abnormal, improves the stability of the whole service, and reduces the occurrence of major production accidents.
Optionally, in the embodiment of the present invention, an operating system used for implementing the static disaster recovery system may be CentOS 7.4, the web server may be nginnx 1.12.2, the lua just-in-time compiler may be LuaJIT-2.1.0-beta3, and the nginnx three-way module may be lua-rginx-module.
From the above, the static disaster recovery system provided by the embodiment of the invention has the capabilities of dynamic service configuration support, intelligent monitoring and switching, unified intelligent checking and the like. The invention can identify URL service resources needing static disaster recovery by adopting a manual configuration mode, and realizes a dynamic configurable, intelligent monitoring and automatic switching static disaster recovery solution through 5 system modules with clear responsibility, high cohesion and low coupling in a static disaster recovery system. In addition, the time version introduced by the disaster recovery buffer module of the static disaster recovery system solves the problems of error data buffer and missing buffer data buffer of the disaster recovery system under extreme conditions.
For example, the static disaster recovery system provided by the embodiment of the invention has at least the following beneficial effects:
1) Has dynamic configuration supporting capability. The dynamic configuration of the support disaster recovery is allowed to be carried out on the URL service, and the system can newly add the static disaster recovery of the support service without restarting.
2) The intelligent monitoring switching capability is provided. And judging the stability of the URL service according to the response state of the request, and automatically proxy the user request into a static disaster-tolerant cache system when an abnormal situation occurs.
3) The system has unified intelligent checking capability. The system can be uniformly managed and maintained, and relevant tests can be carried out to evaluate the static disaster recovery capability of the system.
Based on the static disaster recovery system provided in the foregoing embodiment, the embodiment of the present invention further provides a static disaster recovery method, where the method is applied to the static disaster recovery system. The method may include: after receiving the user request, judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state. The disaster recovery state of the URL is updated according to log monitoring data and disaster recovery triggering conditions, and the log monitoring data is generated according to access logs. And if the user request URL service is configured with the disaster recovery state, reversely proxy the user request to a disaster recovery cache of the static disaster recovery system. The disaster recovery buffer of the static disaster recovery system is buffered with buffer data periodically crawled from the URL request history record in the access log. And if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to the back-end server.
Optionally, before the determining whether the user requests the URL to configure the disaster recovery status, the method further includes: judging the switch state of the static disaster recovery system; the switch state of the static disaster recovery system comprises on or off. If the switch state of the static disaster recovery system is on, judging whether the URL service requested by the user is configured with the disaster recovery state. And if the switch state of the static disaster recovery system is closed, reversely proxy the user request to the back-end server.
Optionally, after the reversely brokering the user request into the backend server, the method further comprises: judging whether the returned result of the back-end server is a server error or not; if the return result of the back-end server is a server error, the user request is reversely proxied into the disaster recovery cache of the static disaster recovery system.
Optionally, the method further comprises: and if the returned result of the back-end server is a normal result, returning the normal result to the user.
Optionally, the method further comprises: and acquiring a URL list supporting static disaster recovery, periodically crawling cache data from a URL request history record in an access log, and storing the cache data into a disaster recovery cache.
Optionally, the method further comprises: reading a URL service list which needs disaster recovery; reading log monitoring data; and updating the disaster recovery state of the URL in the URL service list according to the log monitoring data and the disaster recovery triggering condition.
Optionally, updating the disaster recovery status of the URL according to the log monitoring data and the disaster recovery triggering condition includes: judging the state of the URL service in the URL service list according to the log monitoring data and the disaster recovery triggering condition; if the state of the URL service is abnormal, setting the URL service as a disaster recovery state, and setting the URL service as the configured disaster recovery state after the disaster recovery state; if the state of the URL service is recovered abnormally, the disaster recovery state of the URL service is cleared, and the disaster recovery state of the URL service is the unconfigured disaster recovery state after the disaster recovery state of the URL service is cleared.
Optionally, the disaster recovery triggering condition includes: the response time is greater than or equal to the average response time threshold or tp90 response time threshold.
Optionally, before the log monitoring data is read, the method further includes: and collecting and analyzing the access log to generate log monitoring data.
Optionally, the reading the URL service list requiring disaster recovery includes: reading a URL service list needing disaster recovery from the preconfigured key configuration information; the URL service list requiring disaster recovery comprises resource names, URLs, disaster recovery triggering conditions and contacts corresponding to the URL service requiring static disaster recovery; the URL service list requiring disaster recovery is configured into the static disaster recovery system after the URL service of the static disaster recovery is identified by manual analysis.
Optionally, before the periodically crawling the cache data from the URL request history in the access log, the method further includes: and performing deduplication processing on the URL request history record in the access log.
Optionally, the key cached in the disaster recovery cache of the static disaster recovery system is a url+time version; the static disaster recovery system is used for crawling cache data after collecting the URL request history record data of the first time, and the static disaster recovery system is used for generating different time versions according to the crawling frequency corresponding to the resources.
It can be understood that the static disaster recovery method can be implemented based on the module architecture of the static disaster recovery system described in the foregoing embodiment, or can be implemented based on the static disaster recovery system of other architectures. The embodiments of the present application are not limited in this regard. Each module in the static disaster recovery system for realizing the static disaster recovery method can form a static disaster recovery device.
The static disaster recovery device can be integrated in a server, a computer and other devices, and the invention is not limited herein. It will be clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the above apparatus may refer to the corresponding process of the method described in the foregoing method embodiment, which is not repeated in the present invention.
It should be understood that the above-described device embodiments are merely illustrative, and that the devices and methods disclosed in the embodiments of the present invention may be implemented in other manners. For example, the modules may be divided into only one logic function, and there may be another division manner when actually implemented, and for example, a plurality of modules or components may be combined or may be integrated into another system, or some features may be omitted or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with respect to each other may be through some communication interface, indirect coupling or communication connection of devices or modules, electrical, mechanical, or other form. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a processor to carry out all or part of the steps of the method according to the embodiments of the present invention.
That is, those skilled in the art will appreciate that embodiments of the invention may be implemented in any of the forms of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.
Based on this, the embodiment of the present invention further provides a program product, which may be a storage medium such as a usb disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, where a computer program may be stored, and when the computer program is executed by a processor, the steps of the method described in the foregoing method embodiments are performed. The specific implementation manner and the technical effect are similar, and are not repeated here.
Optionally, the embodiment of the present invention further provides an electronic device, where the electronic device may be a server, a computer, or other devices, and fig. 4 shows a schematic structural diagram of the electronic device provided by the embodiment of the present invention.
As shown in fig. 4, the electronic device may include: a processor 401, a storage medium 402 and a bus 403, the storage medium 402 storing machine-readable instructions executable by the processor 401, when the electronic device is running, communicating with the storage medium 402 via the bus 403, the processor 401 executing the machine-readable instructions to perform the steps of the method as described in the previous embodiments. The specific implementation manner and the technical effect are similar, and are not repeated here.
For ease of illustration, only one processor is depicted in the above-described electronic device. It should be noted, however, that in some embodiments, the electronic device of the present invention may also include multiple processors, and thus, steps performed by one processor described in the present invention may also be performed jointly by multiple processors or separately.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily appreciate variations or alternatives within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (12)
1. A static disaster recovery method, wherein the method is applied to a static disaster recovery system, the method comprising:
after receiving a user request, judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state; the disaster recovery state of the URL is updated according to log monitoring data and disaster recovery triggering conditions, and the log monitoring data is generated after acquisition and analysis of access logs;
if the user request URL service is configured with a disaster recovery state, reversely proxy the user request to a disaster recovery cache of the static disaster recovery system; the disaster recovery buffer of the static disaster recovery system is buffered with buffer data periodically crawled from the URL request history record in the access log; acquiring a URL list supporting static disaster recovery, periodically crawling cache data from a URL request history record in an access log, and storing the cache data into a disaster recovery cache;
if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to a back-end server;
reading a URL service list which needs disaster recovery;
reading log monitoring data;
updating the disaster recovery state of the URL in the URL service list according to the log monitoring data and the disaster recovery triggering condition;
the updating the disaster recovery state of the URL according to the log monitoring data and the disaster recovery triggering condition comprises the following steps:
judging the state of the URL service in the URL service list according to the log monitoring data and the disaster recovery triggering condition;
if the state of the URL service is abnormal, setting the URL service as a disaster recovery state, wherein the URL service is set as the configured disaster recovery state after the disaster recovery state;
and if the state of the URL service is recovered abnormally, clearing the disaster recovery state of the URL service, wherein the cleared disaster recovery state of the URL service is an unconfigured disaster recovery state.
2. The method of claim 1, wherein prior to determining whether the user request URL has configured a disaster recovery status, the method further comprises:
judging the switching state of the static disaster recovery system, wherein the switching state of the static disaster recovery system comprises on or off;
if the switch state of the static disaster recovery system is on, judging whether the URL service requested by the user is configured with a disaster recovery state or not;
and if the switch state of the static disaster recovery system is closed, reversely proxy the user request to a back-end server.
3. The method of claim 2, wherein after reverse-brokering the user request into a backend server, the method further comprises:
judging whether the returned result of the back-end server is a server error or not;
and if the returned result of the back-end server is a server error, reversely proxy the user request to the disaster recovery cache of the static disaster recovery system.
4. A method according to claim 3, characterized in that the method further comprises:
and if the returned result of the back-end server is a normal result, returning the normal result to the user.
5. The method of claim 1, wherein the disaster recovery trigger condition comprises:
the response time is greater than or equal to the average response time threshold or tp90 response time threshold.
6. The method of claim 1, wherein prior to the reading log monitor data, the method further comprises:
and collecting and analyzing the access log to generate log monitoring data.
7. The method of claim 1, wherein the reading the URL service list requiring disaster recovery comprises:
reading the URL service list needing disaster recovery from the pre-configured key configuration information;
the URL service list requiring disaster recovery comprises resource names, URLs, disaster recovery triggering conditions and contacts corresponding to URL services requiring static disaster recovery; the URL service list requiring disaster recovery is configured into the static disaster recovery system after the URL service requiring disaster recovery is manually analyzed and identified.
8. The method of claim 1, wherein prior to the periodically crawling the cached data from the URL request history in the access log, the method further comprises:
and performing deduplication processing on the URL request history record in the access log.
9. The method of claim 1, wherein the key cached in the disaster recovery cache of the static disaster recovery system is a url+temporal version;
the static disaster recovery system is used for crawling cache data after collecting the URL request history record data of the first time, and is used for generating different time versions according to crawling frequency corresponding to the resources.
10. A static disaster recovery system, the static disaster recovery system comprising: the system comprises a reverse proxy module, a log analysis module, a disaster recovery configuration module, a crawler module and a disaster recovery cache module;
the crawler module is used for acquiring a URL list supporting static disaster recovery from the reverse proxy module, and periodically crawling cache data from a URL request history record in an access log to the disaster recovery cache module; the disaster recovery cache module caches the cache data crawled by the crawler module;
the log analysis module is used for uniformly collecting, centrally managing and analyzing the access log to monitor the real-time state of the URL service so as to generate log monitoring data; the disaster recovery configuration module is used for updating the disaster recovery state of the URL service to the reverse proxy module according to the log monitoring data and the disaster recovery triggering condition;
the reverse proxy module is used for judging whether the URL of the user request uniform resource locator is configured with a disaster recovery state after receiving the user request; if the user request URL service is configured with a disaster recovery state, reversely proxy the user request to the disaster recovery cache module; and if the disaster recovery state is not configured for the user request URL service, reversely proxy the user request to a back-end server.
11. An electronic device, comprising: a processor, a storage medium, and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium in communication over the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the method of any of claims 1-9 when executed.
12. A storage medium having stored thereon a computer program which, when executed by a processor, performs the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110080886.5A CN112732999B (en) | 2021-01-21 | 2021-01-21 | Static disaster recovery method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110080886.5A CN112732999B (en) | 2021-01-21 | 2021-01-21 | Static disaster recovery method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112732999A CN112732999A (en) | 2021-04-30 |
| CN112732999B true CN112732999B (en) | 2023-06-09 |
Family
ID=75593540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110080886.5A Active CN112732999B (en) | 2021-01-21 | 2021-01-21 | Static disaster recovery method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112732999B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793538A (en) * | 2014-03-06 | 2014-05-14 | 赛特斯信息科技股份有限公司 | System and method for realizing restoration of web service in case of crash of database |
| CN105528422A (en) * | 2015-12-07 | 2016-04-27 | 中国建设银行股份有限公司 | Focused crawler processing method and apparatus |
| CN106156231A (en) * | 2015-04-24 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of website disaster recovery method, Apparatus and system |
| WO2017107812A1 (en) * | 2015-12-21 | 2017-06-29 | 阿里巴巴集团控股有限公司 | User log storage method and device |
| CN108170561A (en) * | 2018-01-03 | 2018-06-15 | 杭州时趣信息技术有限公司 | A kind of disaster-tolerant backup method, apparatus and system |
| CN108737470A (en) * | 2017-04-19 | 2018-11-02 | 贵州白山云科技有限公司 | A kind of access request time source method and apparatus |
| CN109842518A (en) * | 2018-12-13 | 2019-06-04 | 平安科技（深圳）有限公司 | Content distributing network disaster recovery method, device, computer equipment and storage medium |
| CN110113224A (en) * | 2019-03-19 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Capacity monitor method, apparatus, computer equipment and storage medium |
| CN110784498A (en) * | 2018-07-31 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Personalized data disaster tolerance method and device |
| CN111414523A (en) * | 2020-03-11 | 2020-07-14 | 中国建设银行股份有限公司 | Data acquisition method and device |
| CN111767495A (en) * | 2019-04-01 | 2020-10-13 | 北京沃东天骏信息技术有限公司 | Method and system for synthesizing webpage |
| CN111866205A (en) * | 2020-06-17 | 2020-10-30 | 新浪网技术(中国)有限公司 | Method and system for converting IP address into position information |
| CN112000394A (en) * | 2020-08-27 | 2020-11-27 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for accessing an applet |
| CN112181723A (en) * | 2020-09-22 | 2021-01-05 | 中国建设银行股份有限公司 | Financial disaster recovery method and device, storage medium and electronic equipment |
| CN112202631A (en) * | 2020-09-17 | 2021-01-08 | 北京金山云网络技术有限公司 | Resource access method, device and system, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101674495B (en) * | 2009-10-20 | 2015-06-03 | 中兴通讯股份有限公司 | Method and device for preprocessing data disaster tolerance |
| CN106254100B (en) * | 2016-07-27 | 2019-04-16 | 腾讯科技（深圳）有限公司 | A kind of data disaster tolerance methods, devices and systems |
-
2021
- 2021-01-21 CN CN202110080886.5A patent/CN112732999B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793538A (en) * | 2014-03-06 | 2014-05-14 | 赛特斯信息科技股份有限公司 | System and method for realizing restoration of web service in case of crash of database |
| CN106156231A (en) * | 2015-04-24 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of website disaster recovery method, Apparatus and system |
| CN105528422A (en) * | 2015-12-07 | 2016-04-27 | 中国建设银行股份有限公司 | Focused crawler processing method and apparatus |
| WO2017107812A1 (en) * | 2015-12-21 | 2017-06-29 | 阿里巴巴集团控股有限公司 | User log storage method and device |
| CN108737470A (en) * | 2017-04-19 | 2018-11-02 | 贵州白山云科技有限公司 | A kind of access request time source method and apparatus |
| CN108170561A (en) * | 2018-01-03 | 2018-06-15 | 杭州时趣信息技术有限公司 | A kind of disaster-tolerant backup method, apparatus and system |
| CN110784498A (en) * | 2018-07-31 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Personalized data disaster tolerance method and device |
| CN109842518A (en) * | 2018-12-13 | 2019-06-04 | 平安科技（深圳）有限公司 | Content distributing network disaster recovery method, device, computer equipment and storage medium |
| CN110113224A (en) * | 2019-03-19 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Capacity monitor method, apparatus, computer equipment and storage medium |
| CN111767495A (en) * | 2019-04-01 | 2020-10-13 | 北京沃东天骏信息技术有限公司 | Method and system for synthesizing webpage |
| CN111414523A (en) * | 2020-03-11 | 2020-07-14 | 中国建设银行股份有限公司 | Data acquisition method and device |
| CN111866205A (en) * | 2020-06-17 | 2020-10-30 | 新浪网技术(中国)有限公司 | Method and system for converting IP address into position information |
| CN112000394A (en) * | 2020-08-27 | 2020-11-27 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for accessing an applet |
| CN112202631A (en) * | 2020-09-17 | 2021-01-08 | 北京金山云网络技术有限公司 | Resource access method, device and system, electronic equipment and storage medium |
| CN112181723A (en) * | 2020-09-22 | 2021-01-05 | 中国建设银行股份有限公司 | Financial disaster recovery method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112732999A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11550630B2 (en) | Monitoring and automatic scaling of data volumes | |
| CN105357038B (en) | Monitor the method and system of cluster virtual machine | |
| CN107818431B (en) | Method and system for providing order track data | |
| US9590880B2 (en) | Dynamic collection analysis and reporting of telemetry data | |
| CN111522703B (en) | Method, apparatus and computer program product for monitoring access requests | |
| US20180032387A1 (en) | Predictive Analytics on Database Wait Events | |
| JP2012053903A (en) | Distributed retrieval method, architecture, system and software | |
| US20200341868A1 (en) | System and Method for Reactive Log Spooling | |
| CN111309550A (en) | Data acquisition method, system, equipment and storage medium of application program | |
| CN112506915B (en) | Application data management system, processing method and device and server | |
| CN107544832A (en) | A kind of monitoring method, the device and system of virtual machine process | |
| US20030014507A1 (en) | Method and system for providing performance analysis for clusters | |
| US20200057714A1 (en) | Testing data changes in production systems | |
| CN105574008A (en) | Task scheduling method and equipment applied to distributed file system | |
| CN112433888B (en) | Data processing method and device, storage medium and electronic equipment | |
| US10180914B2 (en) | Dynamic domain name service caching | |
| US20230376470A1 (en) | Moving Window Data Deduplication in Distributed Storage | |
| CN112732999B (en) | Static disaster recovery method, system, electronic equipment and storage medium | |
| CN116521639A (en) | Log data processing method, electronic equipment and computer readable medium | |
| CN109254880B (en) | Method and device for processing database downtime | |
| US20160085638A1 (en) | Computer system and method of identifying a failure | |
| JP5684640B2 (en) | Virtual environment management system | |
| CN114925283A (en) | Management method and system of push task, electronic device and medium | |
| CN114816914A (en) | Data processing method, equipment and medium based on Kubernetes | |
| CN110658989B (en) | System and method for backup storage garbage collection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |