CN113760978A - Data disaster tolerance method and device - Google Patents
Data disaster tolerance method and device Download PDFInfo
- Publication number
- CN113760978A CN113760978A CN202011166146.5A CN202011166146A CN113760978A CN 113760978 A CN113760978 A CN 113760978A CN 202011166146 A CN202011166146 A CN 202011166146A CN 113760978 A CN113760978 A CN 113760978A
- Authority
- CN
- China
- Prior art keywords
- target
- recommendation bit
- target data
- data
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000011084 recovery Methods 0.000 claims description 89
- 230000006870 function Effects 0.000 claims description 29
- 239000012634 fragment Substances 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008030 elimination Effects 0.000 claims description 7
- 238000003379 elimination reaction Methods 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 4
- 230000008439 repair process Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241000508269 Psidium Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data disaster tolerance method and device, and relates to the technical field of computers. One embodiment of the method comprises: receiving an access request of a user, wherein the access request is used for accessing a target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit. The implementation mode reduces the pressure of data service, improves the stability and the safety of the system, and strives for time for data service repair.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data disaster recovery method and apparatus.
Background
The recommendation position is a relatively important and obvious position on the webpage, is easy to attract attention and can be used for pushing important information. The recommendation positions can be divided into different placing positions, and then different contents are recommended to different positions to be displayed. In the prior art, data related to a recommendation bit mainly originates from a data service, the daily availability and performance of the data service are normal, if a sudden traffic flood occurs, the pressure of the data service may increase sharply, which is not favorable for the stability of the system, may cause the unavailability of the data service, and may also cause a recommendation bit data window, which brings huge loss.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data disaster recovery method and apparatus, which can reduce the pressure of data services, improve the stability and security of a system, strive for time for data service repair, and even if the data services are unavailable, the recommendation bits have data that can be displayed.
In order to achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data disaster recovery method, including:
receiving an access request of a user, wherein the access request is used for accessing a target recommendation bit;
determining whether the target recommendation bit meets a shunting condition;
when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit;
and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
Optionally, the determining whether the target recommendation bit satisfies a diversion condition includes:
determining whether the target recommendation bit is a disaster recovery recommendation bit, and determining whether the target recommendation bit starts a disaster recovery function;
and if the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit starts a disaster recovery function, determining that the target recommendation bit meets a shunting condition.
Optionally, the preset storage unit includes a local cache and a preset database system;
determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit includes:
determining whether target data associated with the target recommendation bit exists in the local cache;
if yes, acquiring target data related to the target recommendation bit from the local cache;
and if not, determining whether target data related to the target recommendation bit exists in the preset database system, and if so, acquiring the target data related to the target recommendation bit from the preset database system and writing the target data into the local cache.
Optionally, the method further comprises: and when the target recommendation bit meets a shunting condition and the target data does not exist in the local cache and the preset database system, calling a data service to obtain the target data and writing the target data into the preset storage unit.
Optionally, writing the target data into the preset storage unit includes: writing the target data into the local cache; and determining whether the target data is written into the preset database system or not according to preset writing conditions, and if so, writing the target data into the preset database system.
Optionally, after receiving the access request of the user, the method further includes: generating a random number in response to the access request;
determining whether to write the target data into the preset database system according to a preset writing condition comprises:
determining whether the random number is within a range defined by the preset writing condition;
and if so, writing the target data into the preset database system.
Optionally, before writing the target data into the preset storage unit, the method further includes: and compressing the target data.
Optionally, the access request includes an identifier of the target recommendation bit and category information of an item corresponding to the target recommendation bit;
writing the target data to the local cache comprises: determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a first key according to the identification and the category information; if not, generating a second key according to the identifier and a preset page number; writing the target data to the local cache based on the first key or the second key;
writing the target data into the preset database system comprises: determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a third key according to the identification, the category information and preset redundant fragment information; if not, generating a fourth key according to the identifier, the preset page number and the preset redundant fragment information; and writing the target data into the preset database system based on the third key or the fourth key.
Optionally, the method further comprises: and deleting the data with the least use times in the local cache by using a least-recently-used elimination algorithm.
Optionally, when the target recommendation bit does not satisfy the splitting condition, invoking a data service to obtain target data related to the target recommendation bit includes:
when the target recommendation bit is not the disaster recovery recommendation bit, calling a data service to acquire target data related to the target recommendation bit,
when the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit does not start a disaster recovery function, calling a data service to obtain target data related to the target recommendation bit, determining whether the target data exists in the preset storage unit, and if so, updating the hit rate of the preset storage unit; and if the target data does not exist in the preset storage unit, updating the hit rate of the preset storage unit, and writing the target data into the preset storage unit.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a data disaster recovery apparatus, including:
the receiving module is used for receiving an access request of a user, wherein the access request is used for accessing the target recommendation bit;
the shunting determining module is used for determining whether the target recommendation bit meets a shunting condition;
the data acquisition module is used for determining that target data related to the target recommendation bit exists in a preset storage unit when the target recommendation bit meets a shunting condition, and acquiring the target data from the preset storage unit; and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
Optionally, the split determination module is further configured to: determining whether the target recommendation bit is a disaster recovery recommendation bit, and determining whether the target recommendation bit starts a disaster recovery function; and if the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit starts a disaster recovery function, determining that the target recommendation bit meets a shunting condition.
Optionally, the preset storage unit includes a local cache and a preset database system;
the data acquisition module is further configured to: determining whether target data associated with the target recommendation bit exists in the local cache; if yes, acquiring target data related to the target recommendation bit from the local cache; and if not, determining whether target data related to the target recommendation bit exists in the preset database system, and if so, acquiring the target data related to the target recommendation bit from the preset database system and writing the target data into the local cache.
Optionally, the data obtaining module is further configured to: and when the target recommendation bit meets a shunting condition and the target data does not exist in the local cache and the preset database system, calling a data service to obtain the target data and writing the target data into the preset storage unit.
Optionally, the data obtaining module is further configured to: writing the target data into the local cache; and determining whether the target data is written into the preset database system or not according to preset writing conditions, and if so, writing the target data into the preset database system.
Optionally, the apparatus further comprises a random number module, configured to generate a random number in response to the access request;
the data acquisition module is further configured to: determining whether the random number is within a range defined by the preset writing condition; and if so, writing the target data into the preset database system.
Optionally, the data obtaining module is further configured to: and compressing the target data.
Optionally, the access request includes an identifier of the target recommendation bit and category information of an item corresponding to the target recommendation bit;
the data acquisition module is further configured to: determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a first key according to the identification and the category information; if not, generating a second key according to the identifier and a preset page number; writing the target data to the local cache based on the first key or the second key; determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a third key according to the identification, the category information and preset redundant fragment information; if not, generating a fourth key according to the identifier, the preset page number and the preset redundant fragment information; and writing the target data into the preset database system based on the third key or the fourth key.
Optionally, the apparatus further includes a de-selection module, configured to delete the data with the least use number in the local cache by using a least recently used de-selection algorithm.
Optionally, the data obtaining module is further configured to: when the target recommendation bit is not the disaster recovery recommendation bit, calling a data service to acquire target data related to the target recommendation bit; when the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit does not start a disaster recovery function, calling a data service to obtain target data related to the target recommendation bit, determining whether the target data exists in the preset storage unit, and if so, updating the hit rate of the preset storage unit; and if the target data does not exist in the preset storage unit, updating the hit rate of the preset storage unit, and writing the target data into the preset storage unit.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided an electronic apparatus including: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the data disaster recovery method of the embodiment of the invention.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, the program implementing the data disaster recovery method of the embodiments of the present invention when executed by a processor.
One embodiment of the above invention has the following advantages or benefits: because the access request of the receiving user is adopted, the access request is used for accessing the target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; when the target recommendation bit does not meet the shunting condition, calling a data service to obtain a technical means of target data related to the target recommendation bit, so that the pressure of the data service is reduced, the stability and the safety of a system are improved, and time is strived for repairing the data service; the disaster recovery recommendation bits are configurable and can be flexibly adjusted according to application requirements, so that the recommendation bits with high access degree are the disaster recovery recommendation bits, and the pressure of data service is further reduced; the local cache retains hot spot data by using a least recently used elimination algorithm, so that the hit rate of the local cache is improved; and data is written into the preset database system based on the preset writing condition, so that the pressure of the database system is reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram of a main flow of a data disaster recovery method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a main flow of a data disaster recovery method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a main flow of a data disaster recovery method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of main modules of a data disaster recovery device according to an embodiment of the present invention;
FIG. 5 is an architecture diagram of a system employing a data disaster recovery device according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 7 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a data disaster recovery method according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
step S101: and receiving an access request of a user, wherein the access request is used for accessing the target recommendation bit. Wherein the access request may include an identification of the target recommendation bit.
Step S102: and determining whether the target recommendation bit meets a shunting condition.
Specifically, the method may include:
determining whether the target recommendation bit is a disaster recovery recommendation bit, and determining whether the target recommendation bit starts a disaster recovery function;
and if the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit starts a disaster recovery function, determining that the target recommendation bit meets a shunting condition.
Whether the recommendation bit is a disaster recovery recommendation bit can be configured in advance according to the position of the recommendation bit, and the recommendation bit configured as the disaster recovery recommendation bit can be recorded in the configuration table. And after receiving an access request of a user, judging whether the target recommendation bit is a disaster recovery recommendation bit according to the configuration table. Whether the disaster recovery function is started or not can be realized by configuring whether mock (simulation) is performed or not, specifically, the disaster recovery function is closed when mock is started, and the disaster recovery function is opened when mock is closed. When the disaster recovery recommendation bit starts the disaster recovery function, it is indicated that the target recommendation bit meets the shunting condition, data related to the target recommendation bit needs to be acquired from a preset storage unit, and a data service does not need to be called, so that the pressure of the data service is reduced.
In an alternative embodiment, the preset configuration table may be stored on a ZooKeeper container, where the ZooKeeper is a distributed, open source distributed application coordination service.
Step S103: when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit.
The storage unit comprises a local cache and a preset database system. Specifically, the step may include:
determining whether target data associated with the target recommendation bit exists in the local cache;
if yes, acquiring target data related to the target recommendation bit from the local cache;
and if not, determining whether target data related to the target recommendation bit exists in the preset database system, and if so, acquiring the target data related to the target recommendation bit from the preset database system and writing the target data into the local cache.
In this embodiment, because the local cache has a limited capacity, only the hotspot data is stored in the local cache, and therefore, if the accessed target data is not obtained from the local cache, the target data is written into the local cache. And the preset database system is used as a second-level cache under the condition that the local cache cannot be hit, so that the hit rate of the storage unit is improved. Specifically, the data with the least use times in the local cache is deleted by using the least recently used algorithm. The Least Recently Used algorithm (LRU) eliminates data based on historical access records for the data, with the core idea that if the data has been accessed Recently, then the probability of future access is higher. Wherein, the hit rate is equal to the ratio of the hit amount to the total access request amount, and hit means that the relevant data of the target recommendation bit exists in the storage unit.
In an optional embodiment, in order to ensure that no empty window occurs in the recommendation bit, when the target recommendation bit meets the split condition and the target data does not exist in the local cache and the preset database system, a data service needs to be invoked to obtain the target data, and the target data is written in the preset storage unit.
As shown in fig. 2, writing the target data into the preset storage unit includes:
step S201: writing the target data into the local cache;
step S202: determining whether the target data is written into the preset database system or not according to preset writing conditions;
step S203: if yes, writing the target data into the preset database system;
step S204: and if not, not writing the target data into the preset database system.
In order to reduce the pressure of the preset database system and reduce the cost, the preset database system cannot store the full amount of data, so that when target data are written into the preset database system, whether the writing condition is met needs to be judged, when the writing condition is met, the target data are written into the preset database system, and when the writing condition is not met, the target data are not written into the preset database system. That is, the embodiment of the present invention defines the proportion of the database system preset for writing through the writing condition. As an example, it may be determined to write the target data into a preset database system by the following process:
determining whether a random number is within a range defined by the writing condition, wherein the random number is generated after receiving a user access request;
and if so, determining to write the target data into a preset database system.
For example, the random number generated after receiving the user access request is between 0 and 50, the writing condition is that the random number is greater than or equal to 1 and less than or equal to 10, and if the random number generated after receiving the user access request is 8, the data related to the target recommendation bit is written into a preset database system.
In an alternative embodiment, the target data may be compressed to reduce the size of the target data before writing the target data to the predetermined memory location. Furthermore, more target data are stored in the local cache with limited capacity, the hit rate is improved, and the shunting effect is enhanced.
Step S104: and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
The target recommendation bit does not meet the shunting condition and comprises that the target recommendation bit is not a disaster recovery recommendation bit and the target recommendation bit is a disaster recovery recommendation bit but does not start a disaster recovery function.
When the target recommendation bit is not the disaster recovery recommendation bit, calling a data service to acquire target data related to the target recommendation bit;
when a target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit does not start a disaster recovery function, calling a data service to obtain target data related to the target recommendation bit, determining whether the target data exists in the preset storage unit, and if so, updating the hit rate of the preset storage unit; and if the target data does not exist in the preset storage unit, updating the hit rate of the preset storage unit, and writing the target data into the preset storage unit. The process of writing the target data into the preset memory cell may refer to the embodiment shown in fig. 2.
In this embodiment, when the target recommendation bit does not start the disaster recovery function, the hit rate of the preset storage unit may be counted to determine whether the setting of the disaster recovery recommendation bit is reasonable and whether the expected shunting effect is achieved. When the hit rate of the preset storage unit is not high, the configuration table of the disaster recovery recommendation bit can be modified. The hit rate of the preset storage unit is equal to the sum of the hit rate of the local cache and the hit rate of the preset database system.
According to the data disaster recovery method, an access request of a user is received, wherein the access request is used for accessing a target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit, so that the pressure of the data service is reduced, the stability and the safety of a system are improved, and time is strived for data service restoration; the disaster recovery recommendation bits are configurable and can be flexibly adjusted according to application requirements, so that the recommendation bits with high access degree are the disaster recovery recommendation bits, and the pressure of data service is further reduced; the local cache retains hot spot data by using a least recently used elimination algorithm, so that the hit rate of the local cache is improved; and data is written into the preset database system based on the preset writing condition, so that the pressure of the database system is reduced.
In an alternative embodiment, the target data is stored in the form of key-value in the local cache and in a pre-set database system. Then, before writing the target data into the local cache and the preset database system, a key of the target data needs to be generated. For local caching, it can be determined according to the following procedure:
determining whether the target recommendation bit is a particular recommendation bit;
if so, generating a first key according to the identification of the target recommendation position and the category information of the article corresponding to the target recommendation position;
and if not, generating a second key according to the identifier and a preset page number.
For a preset database system, the determination can be made according to the following process:
determining whether the target recommendation bit is a particular recommendation bit;
if yes, generating a third key according to the identification of the target recommendation position, the category information of the article corresponding to the target recommendation position and preset redundant fragment information;
and if not, generating a fourth key according to the identifier, the preset page number and the preset redundant fragment information.
The identification of the target recommendation bit and the category information of the article corresponding to the target recommendation bit can be obtained from a user access request.
The setting of the specific recommendation bit can be flexibly set according to the application scene. Taking the recommendation sites on an e-commerce website as an example, a particular recommendation site may be a recommendation site on an item detail page. Because the recommendation position on the item detail page needs to have a certain item type correlation with the item on the item detail page, a specific recommendation position can be set as the recommendation position on the item detail page. The preset page number can be configured according to the depth of the target recommended bit browsed by the user, so that data can be browsed within the browsing depth range of the user. As an example, the preset page number is 5. The category information may be a category to which the article belongs. In order to avoid the database system generating data access hot-spots, the database system redundantly stores the target data. For example, if there are 16 fragments in the database system for storing the target data, the preset redundant fragment information is 16, thereby ensuring that each fragment has one copy of the target data.
As an example, a character string obtained by splicing the identifier of the target recommendation bit and the category information of the article corresponding to the target recommendation bit may be used as the first key. The second, third and fourth keys are generated in a manner similar to the first key.
Fig. 3 is a schematic diagram of a main flow of a data disaster recovery method according to another embodiment of the present invention, as shown in fig. 3, the method includes:
step S301: receiving an access request of a user, wherein the access request is used for accessing a target recommendation position, and the access request comprises an identifier of the target recommendation position and category information of an article corresponding to the target recommendation position;
step S302: determining whether the target recommendation bit is a disaster recovery recommendation bit or not based on a preset configuration table; if yes, executing step S303, otherwise executing step S309;
step S303: judging whether the target recommendation bit starts a disaster recovery function or not; if yes, executing step S304, otherwise, executing step S310;
step S304: determining whether relevant data corresponding to the target recommendation bit exists in a local cache; if yes, go to step S307, otherwise go to step S305;
step S305: determining whether a preset database system has related data corresponding to the target recommendation bit; if yes, executing step S306, otherwise executing step S310;
step S306: writing the target data into a local cache;
step S307: decompressing the target data;
step S308: sending the decompressed target data to a client;
step S309: calling basic data service to obtain target data related to the target recommendation bit;
step S310: calling basic data service to obtain target data related to the target recommendation bit, and counting the hit rate of a local cache and a preset database system;
step S311: sending the acquired target data to a client;
step S312: compressing the target data;
step S313: writing the compressed target data into a local cache;
step S314: judging whether the compressed target data is written into a preset database system or not; if yes, go to step S315, otherwise go to step S316;
step S315: writing the compressed target data into a preset database system;
step S316: and the compressed target data is not written into a preset database system.
Specific implementation details of this embodiment may refer to the embodiment shown in fig. 1, and the present invention is not described herein again.
According to the data disaster recovery method, an access request of a user is received, wherein the access request is used for accessing a target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; when the target recommendation bit does not meet the shunting condition, calling a technical means of data service to obtain target data related to the target recommendation bit, so that the pressure of the data service is reduced, the stability and the safety of a system are improved, and time is strived for data service restoration; the disaster recovery recommendation bits are configurable and can be flexibly adjusted according to application requirements, so that the recommendation bits with high access degree are the disaster recovery recommendation bits, and the pressure of data service is further reduced; the local cache retains hot spot data by using a least recently used elimination algorithm, so that the hit rate of the local cache is improved; and data is written into the preset database system based on the preset writing condition, so that the pressure of the database system is reduced.
Fig. 4 is a schematic diagram of main modules of a data disaster recovery device 400 according to an embodiment of the present invention, and as shown in fig. 4, the device 400 includes:
a receiving module 401, configured to receive an access request of a user, where the access request is used to access a target recommendation bit;
an offloading determination module 402, configured to determine whether the target recommendation bit satisfies an offloading condition;
a data obtaining module 403, configured to determine that target data related to the target recommendation bit exists in a preset storage unit when the target recommendation bit meets a shunting condition, and obtain the target data from the preset storage unit; and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
Optionally, the split determination module 402 is further configured to: determining whether the target recommendation bit is a disaster recovery recommendation bit, and determining whether the target recommendation bit starts a disaster recovery function; and if the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit starts a disaster recovery function, determining that the target recommendation bit meets a shunting condition.
Optionally, the preset storage unit includes a local cache and a preset database system;
the data obtaining module 403 is further configured to: determining whether target data associated with the target recommendation bit exists in the local cache; if yes, acquiring target data related to the target recommendation bit from the local cache; and if not, determining whether target data related to the target recommendation bit exists in the preset database system, and if so, acquiring the target data related to the target recommendation bit from the preset database system and writing the target data into the local cache.
Optionally, the data obtaining module 403 is further configured to: and when the target recommendation bit meets a shunting condition and the target data does not exist in the local cache and the preset database system, calling a data service to obtain the target data and writing the target data into the preset storage unit.
Optionally, the data obtaining module 403 is further configured to: writing the target data into the local cache; and determining whether the target data is written into the preset database system or not according to preset writing conditions, and if so, writing the target data into the preset database system.
Optionally, the apparatus further comprises a random number module, configured to generate a random number in response to the access request;
the data obtaining module 403 is further configured to: determining whether the random number is within a range defined by the preset writing condition; and if so, writing the target data into the preset database system.
Optionally, the data obtaining module 403 is further configured to: and compressing the target data.
Optionally, the access request includes an identifier of the target recommendation bit and category information of an item corresponding to the target recommendation bit;
the data acquisition module is further configured to: determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a first key according to the identification and the category information; if not, generating a second key according to the identifier and a preset page number; writing the target data to the local cache based on the first key or the second key; determining whether the target recommendation bit is a particular recommendation bit; if yes, generating a third key according to the identification, the category information and preset redundant fragment information; if not, generating a fourth key according to the identifier, the preset page number and the preset redundant fragment information; and writing the target data into the preset database system based on the third key or the fourth key.
Optionally, the apparatus further includes a de-selection module, configured to delete the data with the least use number in the local cache by using a least recently used de-selection algorithm.
Optionally, the data obtaining module is further configured to: when the target recommendation bit is not the disaster recovery recommendation bit, calling a data service to acquire target data related to the target recommendation bit; when the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit does not start a disaster recovery function, calling a data service to obtain target data related to the target recommendation bit, determining whether the target data exists in the preset storage unit, and if so, updating the hit rate of the preset storage unit; and if the target data does not exist in the preset storage unit, updating the hit rate of the preset storage unit, and writing the target data into the preset storage unit.
According to the data disaster recovery device, an access request of a user is received, wherein the access request is used for accessing a target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; when the target recommendation bit does not meet the shunting condition, calling a technical means of data service to obtain target data related to the target recommendation bit, so that the pressure of the data service is reduced, the stability and the safety of a system are improved, and time is strived for data service restoration; the disaster recovery recommendation bits are configurable and can be flexibly adjusted according to application requirements, so that the recommendation bits with high access degree are the disaster recovery recommendation bits, and the pressure of data service is further reduced; the local cache retains hot spot data by using a least recently used elimination algorithm, so that the hit rate of the local cache is improved; and data is written into the preset database system based on the preset writing condition, so that the pressure of the database system is reduced.
The device can execute the method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
Fig. 5 is a schematic structural diagram of a system to which the data disaster recovery device according to the embodiment of the present invention is applied, and as shown in fig. 5, the system includes a configuration center, a ZooKeeper container, a data disaster recovery device cluster, a recommendation bit client module, a snapshot (for compression/decompression), a local cache, a preset database system, and a data service.
The configuration center is used for managing configuration information, such as configuration disaster recovery recommendation bits, preset page numbers, whether to start a disaster recovery function and writing conditions of the database system. The ZooKeeper container is used to store configuration information. For the configuration stored on the ZooKeeper, it is placed under each path, e.g.,/rec/cart,/rec/ware, etc., and the configuration information corresponding to the recommended bit is stored under these paths. The recommendation bit client is used for sending access requests to the data disaster recovery device cluster, wherein each access request comprises an identifier of a target recommendation bit, and the identifier is used for distinguishing different recommendation bits. And the data disaster tolerance device cluster returns data related to the target recommendation bit to the recommendation bit client, each service instance in the data disaster tolerance device cluster caches configuration information of a configuration center, each configuration path is monitored, and if the content of a certain node on the configuration center changes, all service instance cache configuration information in the cluster is synchronously modified. Snappy is a Google open source compression/decompression library used to compress/decompress target data. The local cache can be realized by adopting an open source guava class library provided by Google, the capacity of operating local cache information is provided, the hit rate can be increased under the condition of high concurrency, and the large-flow impact on a database system is reduced. The database system can be a database cluster based on redis, can provide high-performance data query and storage capacity, and is used as a second-level cache under the condition that a local cache cannot be hit, so that the cache hit rate is increased. The data service is used for providing relevant data of the recommendation bit.
Fig. 6 shows an exemplary system architecture 600 of a data disaster recovery method or a data disaster recovery apparatus to which an embodiment of the present invention can be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 606. The network 604 serves to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. Network 604 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages or the like. Various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like, may be installed on the terminal devices 601, 602, and 603.
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 605 may be a server that provides various services, such as a background management server that supports shopping websites browsed by users using the terminal devices 601, 602, and 603. The background management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (e.g., target push information and product information) to the terminal device.
It should be noted that the data disaster recovery method provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the data disaster recovery device is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks, and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a sending module, an obtaining module, a determining module, and a first processing module. The names of these modules do not in some cases constitute a limitation on the unit itself, and for example, the sending module may also be described as a "module that sends a picture acquisition request to a connected server".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
receiving an access request of a user, wherein the access request is used for accessing a target recommendation bit;
determining whether the target recommendation bit meets a shunting condition;
when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit;
and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
According to the technical scheme of the embodiment of the invention, an access request of a user is received, wherein the access request is used for accessing a target recommendation bit; determining whether the target recommendation bit meets a shunting condition; when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit; when the target recommendation bit does not meet the shunting condition, calling a technical means of data service to obtain target data related to the target recommendation bit, so that the pressure of the data service is reduced, the stability and the safety of a system are improved, and time is strived for data service restoration; the disaster recovery recommendation bits are configurable and can be flexibly adjusted according to application requirements, so that the recommendation bits with high access degree are the disaster recovery recommendation bits, and the pressure of data service is further reduced; the local cache retains hot spot data by using a least recently used elimination algorithm, so that the hit rate of the local cache is improved; and data is written into the preset database system based on the preset writing condition, so that the pressure of the database system is reduced.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A method for data disaster recovery, comprising:
receiving an access request of a user, wherein the access request is used for accessing a target recommendation bit;
determining whether the target recommendation bit meets a shunting condition;
when the target recommendation bit meets a shunting condition, determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit;
and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
2. The method of claim 1, wherein determining whether the target recommendation bit satisfies a split condition comprises:
determining whether the target recommendation bit is a disaster recovery recommendation bit, and determining whether the target recommendation bit starts a disaster recovery function;
and if the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit starts a disaster recovery function, determining that the target recommendation bit meets a shunting condition.
3. The method of claim 2, wherein the predetermined storage unit comprises a local cache and a predetermined database system;
determining that target data related to the target recommendation bit exists in a preset storage unit, and acquiring the target data from the preset storage unit includes:
determining whether target data associated with the target recommendation bit exists in the local cache;
if yes, acquiring target data related to the target recommendation bit from the local cache;
and if not, determining whether target data related to the target recommendation bit exists in the preset database system, and if so, acquiring the target data related to the target recommendation bit from the preset database system and writing the target data into the local cache.
4. The method of claim 3, further comprising: and when the target recommendation bit meets a shunting condition and the target data does not exist in the local cache and the preset database system, calling a data service to obtain the target data and writing the target data into the preset storage unit.
5. The method of claim 4, wherein writing the target data to the predetermined memory location comprises:
writing the target data into the local cache;
and determining whether the target data is written into the preset database system or not according to preset writing conditions, and if so, writing the target data into the preset database system.
6. The method of claim 5, wherein after receiving the user's access request, the method further comprises: generating a random number in response to the access request;
determining whether to write the target data into the preset database system according to a preset writing condition comprises:
determining whether the random number is within a range defined by the preset writing condition;
and if so, writing the target data into the preset database system.
7. The method of claim 6, wherein prior to writing the target data to the predetermined memory location, the method further comprises: and compressing the target data.
8. The method of claim 7, wherein the access request comprises an identification of the target recommendation bit and category information of an item corresponding to the target recommendation bit;
writing the target data to the local cache comprises:
determining whether the target recommendation bit is a particular recommendation bit;
if yes, generating a first key according to the identification and the category information;
if not, generating a second key according to the identifier and a preset page number;
writing the target data to the local cache based on the first key or the second key;
writing the target data into the preset database system comprises:
determining whether the target recommendation bit is a particular recommendation bit;
if yes, generating a third key according to the identification, the category information and preset redundant fragment information;
if not, generating a fourth key according to the identifier, the preset page number and the preset redundant fragment information;
and writing the target data into the preset database system based on the third key or the fourth key.
9. The method of claim 8, further comprising:
and deleting the data with the least use times in the local cache by using a least-recently-used elimination algorithm.
10. The method of any one of claims 1-9, wherein when the target recommendation bit does not satisfy the forking condition, invoking a data service to obtain target data related to the target recommendation bit comprises:
when the target recommendation bit is not the disaster recovery recommendation bit, calling a data service to acquire target data related to the target recommendation bit;
when the target recommendation bit is a disaster recovery recommendation bit and the target recommendation bit does not start a disaster recovery function, calling a data service to obtain target data related to the target recommendation bit, determining whether the target data exists in the preset storage unit, and if so, updating the hit rate of the preset storage unit; and if the target data does not exist in the preset storage unit, updating the hit rate of the preset storage unit, and writing the target data into the preset storage unit.
11. A data disaster recovery apparatus, comprising:
the receiving module is used for receiving an access request of a user, wherein the access request is used for accessing the target recommendation bit;
the shunting determining module is used for determining whether the target recommendation bit meets a shunting condition;
the data acquisition module is used for determining that target data related to the target recommendation bit exists in a preset storage unit when the target recommendation bit meets a shunting condition, and acquiring the target data from the preset storage unit; and when the target recommendation bit does not meet the shunting condition, calling a data service to acquire target data related to the target recommendation bit.
12. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.
13. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011166146.5A CN113760978A (en) | 2020-10-27 | 2020-10-27 | Data disaster tolerance method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011166146.5A CN113760978A (en) | 2020-10-27 | 2020-10-27 | Data disaster tolerance method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113760978A true CN113760978A (en) | 2021-12-07 |
Family
ID=78785888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011166146.5A Pending CN113760978A (en) | 2020-10-27 | 2020-10-27 | Data disaster tolerance method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760978A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017201970A1 (en) * | 2016-05-21 | 2017-11-30 | 乐视控股（北京）有限公司 | Branch base database system and routing method therefor |
| CN109600672A (en) * | 2018-10-19 | 2019-04-09 | 北京微播视界科技有限公司 | Video content method for pushing and device |
| CN110784498A (en) * | 2018-07-31 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Personalized data disaster tolerance method and device |
| CN111488521A (en) * | 2020-04-07 | 2020-08-04 | 支付宝(杭州)信息技术有限公司 | Data processing method and device |
-
2020
- 2020-10-27 CN CN202011166146.5A patent/CN113760978A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017201970A1 (en) * | 2016-05-21 | 2017-11-30 | 乐视控股（北京）有限公司 | Branch base database system and routing method therefor |
| CN110784498A (en) * | 2018-07-31 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Personalized data disaster tolerance method and device |
| CN109600672A (en) * | 2018-10-19 | 2019-04-09 | 北京微播视界科技有限公司 | Video content method for pushing and device |
| CN111488521A (en) * | 2020-04-07 | 2020-08-04 | 支付宝(杭州)信息技术有限公司 | Data processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 王迪;: "智能网容灾方案的研究与实现", 中国新通信, no. 21, 5 November 2012 (2012-11-05) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109947668B (en) | Method and device for storing data | |
| CN108984553B (en) | Caching method and device | |
| CN107911799B (en) | Method for utilizing intelligent route | |
| CN112182004B (en) | Method, device, computer equipment and storage medium for checking data in real time | |
| CN110648216A (en) | Wind control method and device | |
| CN112925859A (en) | Data storage method and device | |
| CN112783887A (en) | Data processing method and device based on data warehouse | |
| CN112748866A (en) | Method and device for processing incremental index data | |
| CN113761565B (en) | Data desensitization method and device | |
| CN113282589A (en) | Data acquisition method and device | |
| CN113641706A (en) | Data query method and device | |
| EP4080383A1 (en) | Method and apparatus for presenting information, electronic device, storage medium, and program product | |
| CN113760978A (en) | Data disaster tolerance method and device | |
| CN112699116A (en) | Data processing method and system | |
| CN113760928A (en) | Cache data updating system and method | |
| CN113704222A (en) | Method and device for processing service request | |
| CN112711572A (en) | Online capacity expansion method and device suitable for sub-warehouse and sub-meter | |
| CN113347052A (en) | Method and device for counting user access data through access log | |
| CN109446183B (en) | Global anti-duplication method and device | |
| US11842077B2 (en) | Method, device, and computer program product for transmitting data for object storage | |
| CN113760860B (en) | Data reading method and device | |
| CN113778909B (en) | Method and device for caching data | |
| US20230131765A1 (en) | Backup and restore of arbitrary data | |
| US20230083822A1 (en) | Optimized rendering of web pages | |
| CN110750410B (en) | Method and device for monitoring database logs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |