CN112737962B - Storage service request processing method and device, computer equipment and storage medium - Google Patents
Storage service request processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112737962B CN112737962B CN202011555935.8A CN202011555935A CN112737962B CN 112737962 B CN112737962 B CN 112737962B CN 202011555935 A CN202011555935 A CN 202011555935A CN 112737962 B CN112737962 B CN 112737962B
- Authority
- CN
- China
- Prior art keywords
- request
- cluster
- storage service
- access link
- read request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/38—Flow based routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a processing method and device of a storage service request, computer equipment and a storage medium. The method provided by the invention comprises the following steps: receiving a storage service request, analyzing the storage service request, and classifying the storage service request according to an analysis result to obtain a read request and a write request; the reading request is filtered by the interceptor to obtain an effective reading request; acquiring all corresponding access links according to the write request and the effective read request; when the first bit cluster corresponding to the first access link in the access links does not have availability, selecting a second access link from the accessed links, sending a write request and an effective read request to the secondary bit cluster corresponding to the second access link, performing flow limiting control on the access link of the secondary bit cluster, and updating the filtering interceptor after the secondary bit cluster successfully responds to the write request and the effective read request. By the invention, the high availability of the storage service can be maintained.
Description
Technical Field
The present invention relates to the field of route optimization, and in particular, to a method and apparatus for processing a storage service request, a computer device, and a storage medium.
Background
When the object storage system is used, the problem that read-write abnormality (readable or writable or unreadable) may exist due to the fact that the bottom cluster is inevitably faulty or has regular maintenance time, the clusters need to be switched at the moment to ensure high availability of storage services in the object storage system, at present, the clusters are basically switched through manual operation, and the operation process of manually switching the clusters is complex. And when the clusters are required to be switched in the abnormal cluster burst at the service peak period, the timeliness cannot be ensured by the manual cluster switching mode, and the current manual cluster switching mode can not meet the service requirement. In addition, the current storage service requests also have malicious attack requests, a large number of requests which do not have resources, access delay and even access abnormality (caused by occupying the connection of excessive resources), the conditions can directly influence the high availability of the storage service, and great pressure is caused on the operation and daily operation of the storage service in the underlying object storage system.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, apparatus, computer device, and storage medium for processing a storage service request, for maintaining high availability of a storage service.
A method of processing a storage service request, comprising:
receiving a storage service request, analyzing the storage service request, and dividing the storage service request into a read request and a write request according to an analysis result;
filtering the read request by using a filtering interceptor to obtain an effective read request;
acquiring a corresponding access link according to the write request and the effective read request;
when the first-order cluster corresponding to the first access link in the access links has availability, sending the write request and the effective read request to the first-order cluster, performing current limiting control on the access link of the first-order cluster, and updating the filtering interceptor after the first-order cluster successfully responds to the write request and the effective read request;
when the first bit cluster corresponding to the first access link in the access links does not have availability, selecting a second access link from the access links, sending the write request and the effective read request to the secondary bit cluster corresponding to the second access link, performing current limiting control on the access link of the secondary bit cluster, and updating the filtering interceptor after the secondary bit cluster successfully responds to the write request and the effective read request; and the secondary bit cluster corresponding to the second access link has availability.
A processing apparatus that stores a service request, comprising:
the classification module is used for receiving a storage service request, analyzing the storage service request and dividing the storage service request into a read request and a write request according to an analysis result;
the interception module is used for filtering the read request by utilizing a filtering interceptor to obtain an effective read request;
the acquisition module is used for acquiring a corresponding access link according to the write request and the effective read request;
the first updating module is used for sending the write request and the effective read request to the first cluster when the first cluster corresponding to the first access link in the access links has availability, performing current limiting control on the access link of the first cluster, and updating the filtering interceptor after the first cluster successfully responds to the write request and the effective read request;
a second updating module, configured to, when it is determined that a first bit cluster corresponding to a first access link in the access links does not have availability, select a second access link from the access links, send the write request and the valid read request to a sub bit cluster corresponding to the second access link, perform current limiting control on the access link in which the sub bit cluster is located, and update the filtering interceptor after the sub bit cluster successfully responds to the write request and the valid read request; and the secondary bit cluster corresponding to the second access link has availability.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the above-mentioned method of processing a storage service request when executing the computer program.
A computer readable storage medium storing a computer program which, when executed by a processor, implements the above-described method of processing a storage service request.
According to the processing method, the processing device, the computer equipment and the storage medium for the storage service request, when the bottom layer cluster is problematic, the first cluster can be automatically and dynamically switched, the problem of low timeliness caused by manually switching the cluster is avoided, the high availability of the storage service can be maintained, meanwhile, the current limiting and filtering functions are added, the storage service request can be dynamically managed, the influence of the abnormal storage service request on the high availability of the bottom layer storage system where the cluster is located in a short time is avoided, and the operation and maintenance pressures of the bottom layer storage system can be reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application environment of a method for processing a storage service request according to an embodiment of the invention;
FIG. 2 is a flow chart of a method of processing a storage service request according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a processing device for storing service requests according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a computer device in accordance with an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The processing method of the storage service request provided by the invention can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server through a network. Among other things, clients may include, but are not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a method for processing a storage service request is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
s10, receiving a storage service request, analyzing the storage service request, and dividing the storage service request into a read request and a write request according to an analysis result;
it can be understood that the storage service request is a request sent by a user to an underlying storage system (equivalent to the first cluster and the second cluster in the present invention) in the gateway system, where the gateway system provides gateway services, and the gateway services are mainly built in the underlying storage system, and the functions mentioned below in the present invention can be considered as gateway services provided by the gateway system (the gateway services have an automatic distributed high availability, an automatic selector and a notification delivery function, etc.), where the storage service in the storage service request is implemented by the gateway services, and the storage service is exemplified by CEPH object storage service, and the main service features are reading and writing, where the storage service request is simpler, but the access flow and the access concurrency associated with the storage service request are larger, and at this time, a gateway routing layer can be set in the execution body gateway system in the present invention to provide the gateway services to solve the above problem; the read request is get request, the write request is put request, and the classification result can be obtained from the request message header or the request content after the analyzed storage service request.
S20, filtering the read request by using a filtering interceptor to obtain an effective read request;
it is understood that the filtering interceptor includes, but is not limited to, a bloom filter and other custom extension filters, where the bloom filter may be used to retrieve whether an element is in a set, that is, detect whether a read request is in a specified request set, where an element corresponding to an invalid request exists (the element in the request set is stored through a hash table, the element includes a source ip and access to a resource prefix of an underlying layer and user information in the request, etc.), where the invalid request includes, but is not limited to, a malicious request, an offensive request of an individual user, and a request that needs flow control or concurrency restriction.
S30, acquiring a corresponding access link according to the write request and the effective read request;
the access link is a point-to-point data transmission line, which is equivalent to an access path corresponding to a write request and an effective read request, wherein the first cluster and network resources corresponding to the write request and the effective read request in the first cluster can be inquired and acquired through the access link; each access link may correspond to a cluster (the first cluster and the second cluster as mentioned below).
S40, when the availability of a first-order cluster corresponding to a first access link in the access links is determined, sending the write request and the effective read request to the first-order cluster, performing current limiting control on the access link of the first-order cluster, and updating the filter interceptor after the first-order cluster successfully responds to the write request and the effective read request;
the first cluster is a target cluster, and is obtained by analyzing the access link by a routing policy, wherein the routing policy refers to the process corresponding to the steps S10 to S30, and when the first cluster has availability, it is indicated that there is no abnormality in the first cluster, and at this time, a write request and an effective read request can be sent to the first cluster, and then a storage service of the first cluster is requested to be obtained; the first access link is used for determining an access path (a first cluster) corresponding to the write request and the effective read request and is used for inquiring network resources (such as object storage addresses and object data) corresponding to the write request and the effective read request in the first cluster; the flow limiting control is reflected by QPS (query rate per second) and QOS (quality of service), after the flow or concurrency of the write request and the effective read request is limited, the filtering interceptor is updated according to the current limiting condition, so that when the filtering interceptor encounters the same current limiting condition next time, the writing request and the effective read request corresponding to the limiting condition can be directly limited by the filtering interceptor, and the like.
S50, when the first bit cluster corresponding to the first access link in the access links does not have availability, selecting a second access link from the access links, sending the write request and the effective read request to the secondary bit cluster corresponding to the second access link, performing current limiting control on the access link of the secondary bit cluster, and updating the filter interceptor after the secondary bit cluster successfully responds to the write request and the effective read request; and the secondary bit cluster corresponding to the second access link has availability.
As can be appreciated, the secondary clusters are backup clusters, when the primary clusters are switched off, the secondary clusters are equivalent to the primary positions of the primary clusters, and it is required to be noted that the number of the secondary clusters is not more than one, and when one secondary cluster has no availability, the secondary clusters can be switched to the other secondary cluster, wherein when the secondary clusters have no availability, that is, the secondary clusters have abnormality, at this time, a write request and an effective read request can be sent to the secondary clusters, and then a storage service of the primary clusters can be requested; the second access link is used for determining an access path (secondary bit cluster) corresponding to the write request and the effective read request and is used for inquiring network resources (such as object storage address and object data) corresponding to the write request and the effective read request in the secondary bit cluster; in this embodiment, the current limiting is performed according to the current limiting condition of S40, which is not described in detail.
In the embodiment of steps S10 to S50, when the bottom layer cluster has a problem, the first cluster can be automatically dynamically switched, so that the problem of low timeliness caused by manually switching the clusters is avoided, the high availability of storage service can be maintained, meanwhile, the current limiting and filtering functions are added, the storage service request can be dynamically managed, the influence of the abnormal storage service request in a short time on the high availability of the bottom layer storage system where the clusters are located is avoided, and the operation and maintenance pressures of the bottom layer storage system can be reduced.
Further, the parsing the storage service request and dividing the storage service request into a read request and a write request according to the parsing result includes:
acquiring a corresponding preset routing strategy according to the storage service request;
analyzing a request message header or request content in the storage service request according to the preset routing strategy to obtain the read request, the write request and a first route corresponding to the read request and a second route corresponding to the write request; the first route refers to a path for filtering the read request through a filtering interceptor, and the second route refers to a path for acquiring a corresponding access link according to the write request and the valid read request.
It can be understood that, in this embodiment, the route suitable for the storage service request is analyzed according to a preset routing policy, and the processing path belonging to the storage service request is determined mainly through analysis, so as to improve the processing efficiency of the storage service request. In another embodiment, a corresponding preset routing policy is obtained according to the storage service request, and a request source or other special marking bits in the storage service request are parsed according to the preset routing policy to obtain the read request, the write request, and a first route corresponding to the read request and a second route corresponding to the write request.
Further, after the corresponding access link is obtained according to the write request and the valid read request, the method further includes:
a preset interface or a third party processing platform is called to detect the state of the first cluster so as to detect whether the first cluster has availability;
understandably, the preset interfaces include, but are not limited to, hadoop interfaces and Zookeeper interfaces; third party processing platforms include, but are not limited to, an elastiscearch search server and a kafka Manager cluster management tool; the embodiment is mainly used for timely determining whether the first cluster is abnormal or not by detecting the state of the first cluster.
After determining that the first cluster corresponding to the first access link in the access links does not have availability, the method further includes:
triggering a cluster switching strategy for switching the first cluster to the secondary cluster, and calling the preset interface or a third party processing platform to detect the state of the secondary cluster so as to detect whether the secondary cluster has availability.
It can be understood that the cluster switching strategy is equivalent to a routing strategy, that is, the above-mentioned switching of the first cluster to the second cluster; the method and the device mainly determine the availability of the cluster in time by detecting the processing state of the cluster, so that the cluster can be switched subsequently, and the cluster can be ensured to respond to the storage service request quickly.
Further, the selecting a second access link from the access links, and sending the write request and the valid read request to the secondary bit cluster corresponding to the second access link includes:
determining whether a main write cluster associated with the read request exists in the access link in an identification mode;
when the main write cluster does not exist, a second access link is selected from the access links, the write request and the effective read request are sent to a secondary cluster corresponding to the second access link, and request data fed back by the secondary cluster are acquired; the request data is fed back after the secondary bit cluster with availability receives the successful response of the write request and the valid read request.
Understandably, the request data corresponding to the write request may be the object storage address of the secondary cluster, and the request data corresponding to the valid read request may be the object data downloaded from the storage address of the secondary cluster; in this embodiment, different request data are fed back mainly according to different requests, so that corresponding gateway services are made according to different requests, in addition, when a read request is a read request, the embodiment firstly goes to a main write cluster associated with the read request, so that whether the main write cluster is stored or not needs to be determined by identifying the main write cluster, and when the main write cluster does not exist, an effective read request is sent to a secondary bit cluster, and when the write request is a write request, only the write request needs to be migrated to the secondary bit cluster corresponding to the second access link to realize sending the write request to the secondary bit cluster corresponding to the second access link.
Further, the obtaining the request data fed back by the secondary bit cluster includes: :
determining whether a rename exists between the first request data newly written in the primary storage area in the first-order cluster and the second request data stored in the backup storage area in the secondary cluster; the first request data is request data currently written and stored in the head cluster; the second request data is the request data which is written in advance and stored in the secondary bit cluster; the method comprises the steps of carrying out a first treatment on the surface of the
When a rename exists and a main storage area in the first cluster does not have availability, switching the first cluster to the secondary cluster containing the second request data, and acquiring the request data fed back by the secondary cluster; the request data fed back by the secondary cluster is second request data stored in a backup storage area in the secondary cluster;
after the primary storage area in the first cluster has availability, the request data stored in the backup storage area in the secondary cluster is returned to the primary storage area corresponding to the first cluster, and the request data stored in the backup storage area is deleted.
Understandably, the primary storage area in the primary cluster does not have availability, that is, the primary storage area has abnormality, wherein the primary storage area is arranged in the primary cluster, and the backup storage area is arranged in the secondary cluster; the main problem solved by this embodiment is that the newly written first request data and the old written second request data are renamed in different clusters, and in this case, the request data needs to be stored in the primary storage area of the first cluster during the processing process, so as to ensure the uniqueness of the request data.
Further, after the reading request passes through the filtering interceptor, obtaining an effective reading request includes:
after intercepting the read request through the filtering interceptor, classifying the read request to obtain a classification result;
and acquiring preset processing strategies corresponding to different classification results through the filtering interceptor, and filtering each read request corresponding to the preset processing strategies through the different preset processing strategies to obtain the effective read request.
As can be appreciated, the classification result includes, but is not limited to, malicious requests, aggressive requests, requests requiring flow control or concurrency restriction of individual users, invalid access requests (requests of users with a large amount of access to request data of the underlying cluster which do not exist), when the classification result is a malicious request, an aggressive request and a request requiring flow control or concurrency restriction, the preset processing policy is filtering flow limiting control (mainly realizing that the source ip accesses the resource prefix of the underlying cluster and user information in the request, etc.), when the classification result is an invalid access request, the preset processing policy is direct interception, the invalid access request is not polled to access the underlying cluster, when the filtering interceptor finds that the underlying cluster does not exist, the non-existing result is directly accessed, and the invalid access request is prevented from falling into the underlying cluster directly; the purpose of the interceptor is to intercept the invalid request in the read request, prevent similar buffer breakdown, make the read request calling the bottom layer be an effective request (effective read request), avoid that the invalid request affects the normal service in the gateway system during the service peak period, and improve the performance of the gateway system and the duty ratio of the effective request.
Further, before receiving the storage service request, the method further includes:
processing the storage service request through a target routing strategy corresponding to the storage service; the target routing policy includes one or more combinations of setting pre-route and post-route interfaces, exception handling, drainage and statistics.
It can be understood that this embodiment supports interfaces such as extended pre-routing, post-routing, exception handling, etc., and also supports relevant statistics, traffic statistics, concurrency statistics, access volume statistics, etc., while supporting drainage, and implementing an offline environment for online traffic drainage (after drainage, the use of the offline environment for relevant analysis or new functional test, etc.). In another embodiment, the target routing policy and the preset target routing policy are updated according to the requirement, a Raft protocol is adopted to achieve consistency of gateway services, a leader service is elected through the Raft protocol, and calculation of specific target routing policy change and other follower services are achieved through the leader service.
In summary, the above method for processing a storage service request can automatically dynamically switch the first cluster when a problem occurs in the bottom cluster, so as to avoid the problem of low timeliness caused by manually switching the clusters, also can keep high availability of the storage service, simultaneously has added a current limiting and filtering function, can dynamically manage the storage service request, avoid the influence of an abnormal storage service request in a short time on the high availability of the bottom storage system where the cluster is located, can reduce operation and maintenance pressure of the bottom storage system, simultaneously provides update of a routing strategy, also provides an expanded interface, and further can realize diversification of gateway services.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a processing device for storing a service request is provided, where the processing device for storing a service request corresponds to the processing method for storing a service request in the foregoing embodiment one by one. As shown in fig. 3, the processing device for storing a service request includes a classification module 11, an interception module 12, an acquisition module 13, a first update module 14, and a second update module 15. The functional modules are described in detail as follows:
the classification module 11 is configured to receive a storage service request, parse the storage service request, and divide the storage service request into a read request and a write request according to a parsing result;
the interception module 12 is configured to filter the read request by using a filtering interceptor to obtain an effective read request;
an obtaining module 13, configured to obtain a corresponding access link according to the write request and the valid read request;
a first updating module 14, configured to send the write request and the valid read request to a first cluster corresponding to a first access link among the access links when determining that the first cluster has availability, perform current limit control on the access link where the first cluster is located, and update the filtering interceptor after the first cluster successfully responds to the write request and the valid read request;
a second updating module 15, configured to, when it is determined that a first bit cluster corresponding to a first access link in the access links does not have availability, select a second access link from the access links, send the write request and the valid read request to a sub bit cluster corresponding to the second access link, perform current limiting control on the access link in which the sub bit cluster is located, and update the filtering interceptor after the sub bit cluster successfully responds to the write request and the valid read request; and the secondary bit cluster corresponding to the second access link has availability.
Further, the classification module includes:
the first acquisition sub-module is used for acquiring a corresponding preset routing strategy according to the storage service request;
the first classification sub-module is used for analyzing a request message header or request content in the storage service request according to the preset routing strategy to obtain the read request, the write request and a first route corresponding to the read request and a second route corresponding to the write request; the first route refers to a path for filtering the read request through a filtering interceptor, and the second route refers to a path for acquiring a corresponding access link according to the write request and the valid read request.
Further, the processing device for storing the service request further includes:
the first detection module is used for calling a preset interface or a third party processing platform to detect the state of the first cluster so as to detect whether the first cluster has availability;
the second detection module is used for triggering a cluster switching strategy for switching the first cluster to the secondary cluster, and calling the preset interface or a third party processing platform to detect the state of the secondary cluster so as to detect whether the secondary cluster has availability.
Further, the second updating module includes:
a determining submodule, configured to determine, by using an identification manner, whether a primary write cluster associated with the read request exists in the access link;
the second acquisition sub-module is used for selecting a second access link from the access links when the main write cluster does not exist, sending the write request and the effective read request to a secondary cluster corresponding to the second access link, and acquiring request data fed back by the secondary cluster; the request data is fed back after the secondary bit cluster with availability receives the successful response of the write request and the valid read request.
Further, the second acquisition submodule includes:
the determining unit is used for determining whether the first request data newly written in the primary storage area in the first-order cluster and the second request data stored in the backup storage area in the secondary-order cluster have a duplicate name or not; the first request data is request data currently written and stored in the head cluster; the second request data is the request data which is written in advance and stored in the secondary bit cluster; the method comprises the steps of carrying out a first treatment on the surface of the
The acquisition unit is used for switching the first cluster to the secondary cluster containing the second request data and acquiring the request data fed back by the secondary cluster when a duplicate name exists and a main storage area in the first cluster does not have availability; the request data fed back by the secondary cluster is second request data stored in a backup storage area in the secondary cluster;
and the deleting unit is used for returning the request data stored in the backup storage area in the secondary cluster to the primary storage area corresponding to the primary cluster after the availability of the primary storage area in the primary cluster is achieved, and deleting the request data stored in the backup storage area.
Further, the interception module includes:
the second classification sub-module is used for classifying the read requests after intercepting the read requests through the filtering interceptor to obtain classification results;
and the filtering sub-module is used for acquiring preset processing strategies corresponding to different classification results through the filtering interceptor, and filtering each read request corresponding to the preset processing strategies through different preset processing strategies to obtain the effective read request.
Further, the processing device for storing a service request further includes:
the newly added routing module is used for processing the storage service request through a target routing strategy corresponding to the storage service; the target routing policy includes one or more combinations of setting pre-route and post-route interfaces, exception handling, drainage and statistics.
The specific limitation of the processing device for the storage service request may be referred to as limitation of the processing method for the storage service request hereinabove, and will not be described herein. The above-described respective modules in the processing means for storing a service request may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data involved in the processing method of the storage service request. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method of processing a storage service request.
In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement steps of the method for processing a storage service request in the above embodiment, such as steps S10 to S50 shown in fig. 2. Alternatively, the processor may implement the functions of the respective modules/units of the processing apparatus for storing a service request in the above embodiment, such as the functions of the modules 11 to 15 shown in fig. 3, when executing the computer program. In order to avoid repetition, a description thereof is omitted.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps of the method for processing a storage service request in the above embodiment, such as steps S10 to S50 shown in fig. 2. Alternatively, the computer program when executed by the processor implements the functions of the respective modules/units of the processing apparatus for storing a service request in the above-described embodiment, such as the functions of the modules 11 to 15 shown in fig. 3. In order to avoid repetition, a description thereof is omitted.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. A method of processing a storage service request, comprising:
receiving a storage service request, analyzing the storage service request, and dividing the storage service request into a read request and a write request according to an analysis result;
filtering the read request by using a filtering interceptor to obtain an effective read request; the filtered read request is an invalid request;
acquiring a corresponding access link according to the write request and the effective read request;
when the first-order cluster corresponding to the first access link in the access links has availability, sending the write request and the effective read request to the first-order cluster, performing current limiting control on the access link of the first-order cluster, and updating the filtering interceptor after the first-order cluster successfully responds to the write request and the effective read request;
when the first bit cluster corresponding to the first access link in the access links does not have availability, selecting a second access link from the access links, sending the write request and the effective read request to the secondary bit cluster corresponding to the second access link, performing current limiting control on the access link of the secondary bit cluster, and updating the filtering interceptor after the secondary bit cluster successfully responds to the write request and the effective read request; and the secondary bit cluster corresponding to the second access link has availability.
2. The method for processing a storage service request according to claim 1, wherein analyzing the storage service request and dividing the storage service request into a read request and a write request according to the analysis result comprises:
acquiring a corresponding preset routing strategy according to the storage service request;
analyzing a request message header or request content in the storage service request according to the preset routing strategy to obtain the read request, the write request and a first route corresponding to the read request and a second route corresponding to the write request; the first route refers to a path for filtering the read request through a filtering interceptor, and the second route refers to a path for acquiring an access link corresponding to the write request and the valid read request according to the write request.
3. The method according to claim 1, wherein after the obtaining the corresponding access link according to the write request and the valid read request, further comprising:
a preset interface or a third party processing platform is called to detect the state of the first cluster so as to detect whether the first cluster has availability;
after determining that the first cluster corresponding to the first access link in the access links does not have availability, the method further includes:
triggering a cluster switching strategy for switching the first cluster to the secondary cluster, and calling the preset interface or a third party processing platform to detect the state of the secondary cluster so as to detect whether the secondary cluster has availability.
4. The method for processing a storage service request according to claim 1, wherein selecting a second access link from the access links, and sending the write request and the valid read request to a secondary cluster corresponding to the second access link, includes:
determining whether a main write cluster associated with the read request exists in the access link in an identification mode;
when the main write cluster does not exist, a second access link is selected from the access links, the write request and the effective read request are sent to a secondary cluster corresponding to the second access link, and request data fed back by the secondary cluster are acquired; the request data is fed back after the secondary bit cluster with availability receives the successful response of the write request and the valid read request.
5. The method for processing a storage service request according to claim 4, wherein the obtaining the request data fed back by the secondary cluster includes:
determining whether a rename exists between the first request data newly written in the primary storage area in the first-order cluster and the second request data stored in the backup storage area in the secondary cluster; the first request data is request data currently written and stored in the head cluster; the second request data is the request data which is written in advance and stored in the secondary bit cluster;
when a rename exists and a main storage area in the first cluster does not have availability, switching the first cluster to the secondary cluster containing the second request data, and acquiring the request data fed back by the secondary cluster; the request data fed back by the secondary cluster is second request data stored in a backup storage area in the secondary cluster;
after the primary storage area in the first cluster has availability, the request data stored in the backup storage area in the secondary cluster is returned to the primary storage area corresponding to the first cluster, and the request data stored in the backup storage area is deleted.
6. The method for processing a storage service request according to claim 1, wherein filtering the read request by using a filtering interceptor to obtain a valid read request comprises:
after intercepting the read request through the filtering interceptor, classifying the read request to obtain a classification result;
and acquiring preset processing strategies corresponding to different classification results through the filtering interceptor, and filtering each read request corresponding to the preset processing strategies through the different preset processing strategies to obtain the effective read request.
7. The method for processing a storage service request according to claim 1, further comprising, before receiving the storage service request:
processing the storage service request through a target routing strategy corresponding to the storage service; the target routing policy includes one or more combinations of setting pre-route and post-route interfaces, exception handling, drainage and statistics.
8. A processing apparatus for storing a service request, comprising:
the classification module is used for receiving a storage service request, analyzing the storage service request and dividing the storage service request into a read request and a write request according to an analysis result;
the interception module is used for filtering the read request by utilizing a filtering interceptor to obtain an effective read request; the filtered read request is an invalid request;
the acquisition module is used for acquiring a corresponding access link according to the write request and the effective read request;
the first updating module is used for sending the write request and the effective read request to the first cluster when the first cluster corresponding to the first access link in the access links has availability, performing current limiting control on the access link of the first cluster, and updating the filtering interceptor after the first cluster successfully responds to the write request and the effective read request;
a second updating module, configured to, when it is determined that a first bit cluster corresponding to a first access link in the access links does not have availability, select a second access link from the access links, send the write request and the valid read request to a sub bit cluster corresponding to the second access link, perform current limiting control on the access link in which the sub bit cluster is located, and update the filtering interceptor after the sub bit cluster successfully responds to the write request and the valid read request; and the secondary bit cluster corresponding to the second access link has availability.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements a method of handling storage service requests according to any of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements a method of processing a storage service request according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011555935.8A CN112737962B (en) | 2020-12-24 | 2020-12-24 | Storage service request processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011555935.8A CN112737962B (en) | 2020-12-24 | 2020-12-24 | Storage service request processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112737962A CN112737962A (en) | 2021-04-30 |
| CN112737962B true CN112737962B (en) | 2023-06-02 |
Family
ID=75615613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011555935.8A Active CN112737962B (en) | 2020-12-24 | 2020-12-24 | Storage service request processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112737962B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017088664A1 (en) * | 2015-11-26 | 2017-06-01 | 深圳市中博科创信息技术有限公司 | Data processing method and apparatus for cluster file system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9817860B2 (en) * | 2011-12-13 | 2017-11-14 | Microsoft Technology Licensing, Llc | Generation and application of correctness-enforced executable filters |
| CN106972975A (en) * | 2017-04-27 | 2017-07-21 | 上海斐讯数据通信技术有限公司 | A kind of server management method and system |
| CN107329832B (en) * | 2017-06-30 | 2021-01-22 | 北京奇艺世纪科技有限公司 | Data receiving method and device |
| CN108052664A (en) * | 2017-12-29 | 2018-05-18 | 北京小度信息科技有限公司 | The data migration method and device of database purchase cluster |
| CN111885098B (en) * | 2020-06-03 | 2021-06-29 | 中邮消费金融有限公司 | Proxy access method, system and computer equipment for object storage cluster |
| CN111917846A (en) * | 2020-07-19 | 2020-11-10 | 中信银行股份有限公司 | Kafka cluster switching method, device and system, electronic equipment and readable storage medium |
-
2020
- 2020-12-24 CN CN202011555935.8A patent/CN112737962B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017088664A1 (en) * | 2015-11-26 | 2017-06-01 | 深圳市中博科创信息技术有限公司 | Data processing method and apparatus for cluster file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112737962A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109064345B (en) | Message processing method, system and computer readable storage medium | |
| CN110287709B (en) | User operation authority control method, device, equipment and medium | |
| US11586673B2 (en) | Data writing and reading method and apparatus, and cloud storage system | |
| US9971823B2 (en) | Dynamic replica failure detection and healing | |
| CN109787908B (en) | Server current limiting method, system, computer equipment and storage medium | |
| US7155462B1 (en) | Method and apparatus enabling migration of clients to a specific version of a server-hosted application, where multiple software versions of the server-hosted application are installed on a network | |
| US20180295029A1 (en) | Managing groups of servers | |
| CN105099988A (en) | Method used for supporting gray releasing, access method, devices and system | |
| CN109768879B (en) | Method and device for determining target service server and server | |
| CN106959820B (en) | Data extraction method and system | |
| US10133673B2 (en) | Cache optimization based on predictive routing | |
| US20100262687A1 (en) | Dynamic data partitioning for hot spot active data and other data | |
| CN111475483A (en) | Database migration method and device and computing equipment | |
| CN109739435B (en) | File storage and updating method and device | |
| CN112291365B (en) | Access balance processing method, device, computer equipment and storage medium | |
| CN109981779B (en) | Service providing method, server and computer storage medium | |
| CA3138764A1 (en) | Data processing method, device, computer equipment and storage medium | |
| CN112737962B (en) | Storage service request processing method and device, computer equipment and storage medium | |
| WO2022134768A1 (en) | Method and apparatus for managing ip proxy pool, and computer device and storage medium | |
| CN114143090A (en) | Firewall deployment method, device, equipment and medium based on network security architecture | |
| CN107181773A (en) | Data storage and data managing method, the equipment of distributed memory system | |
| JP4043999B2 (en) | Apparatus and method for automatic and dynamic reconfiguration network preparation | |
| CN110545336A (en) | IP address replacing method, device, computer equipment and storage medium | |
| CN114490681A (en) | Service request response method and device and server cluster | |
| CN110866066B (en) | Service processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |