CN110990176A - Disaster recovery method, system and device based on multithreading processing - Google Patents

Disaster recovery method, system and device based on multithreading processing Download PDF

Info

Publication number
CN110990176A
CN110990176A CN201911005259.4A CN201911005259A CN110990176A CN 110990176 A CN110990176 A CN 110990176A CN 201911005259 A CN201911005259 A CN 201911005259A CN 110990176 A CN110990176 A CN 110990176A
Authority
CN
China
Prior art keywords
data
queue
user
disaster recovery
method comprises
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911005259.4A
Other languages
Chinese (zh)
Inventor
张俊伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Lajin Zhongbo Technology Co ltd
Original Assignee
Wuxi Tvmining Juyuan Media Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuxi Tvmining Juyuan Media Technology Co Ltd filed Critical Wuxi Tvmining Juyuan Media Technology Co Ltd
Priority to CN201911005259.4A priority Critical patent/CN110990176A/en
Publication of CN110990176A publication Critical patent/CN110990176A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0715—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a system implementing multitasking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a disaster recovery method based on multithread processing, which comprises the following steps: receiving a user request, and initializing and creating a plurality of threads; judging the user request according to a preset judgment condition; according to the judgment result, transmitting the data requested by the user to the cache queue through the thread; and storing the data requested by the user according to the setting of the buffer queue. The system comprises: the device comprises an operation module, a judgment module, a transmission module and a storage module. The device comprises a memory and a processor for executing the disaster recovery method based on the multithreading processing. By using the invention, the loss caused by the downtime of the server for storing the data can be avoided. The disaster recovery method, the system and the device based on the multithreading processing can be widely applied to the field of data processing.

Description

Disaster recovery method, system and device based on multithreading processing
Technical Field
The present invention relates to the field of data processing, and in particular, to a method, system, and apparatus for disaster recovery based on multithread processing.
Background
At present, most disaster recovery schemes store the same and all data by two or more servers, and when one server is down or maintained, another server is started, wherein most of the servers use a database such as mysql to store the data, and when a large number of concurrent requests occur, the database is not carried over, and in some cases, the requests need to be confirmed and the data needs to be read quickly in real time, but the data of mysql is stored on a hard disk and is read at a lower speed, which is not suitable for the situation facing real-time storage and data access.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method, a system, and an apparatus for disaster recovery based on multithread processing, which can avoid loss caused by a server crash of a storage data.
The first technical scheme adopted by the invention is as follows: a disaster recovery method based on multithread processing comprises the following steps:
receiving a user request, and initializing and creating a plurality of threads;
judging the user request according to a preset judgment condition;
according to the judgment result, transmitting the data requested by the user to the cache queue through the thread;
and storing the data requested by the user according to the setting of the buffer queue.
Further, the buffer queue is implemented based on redis.
Further, a first thread of the multiple threads is used for running a task for judging whether the request meets the requirement, and the other threads are used for running tasks for transmitting data to each cache queue.
Further, the cache queue comprises a first type of redis cache queue and a second type of redis cache queue, wherein the first type of redis cache queue is used for storing data in a preset period, and the second type of redis cache queue is used for storing all data.
Further, the method for storing the data requested by the user according to the setting of the buffer queue further comprises the following steps:
and storing the data of the second type of redis cache queue to the hard disk.
Further, when one of the buffer queues needs to be maintained or an error occurs, other buffer queues are called to store the data requested by the user.
Further, the preset judgment condition is judged by a time limit.
The second technical scheme adopted by the invention is as follows: a multi-thread processing based disaster recovery system, comprising:
the running module is used for receiving a user request and initializing and creating a plurality of threads;
the judging module is used for judging the user request according to a preset judging condition;
the transmission module is used for transmitting the data requested by the user to the cache queue through the thread at the same time according to the judgment result;
and the storage module is used for storing the data requested by the user according to the setting of the buffer queue.
The third technical scheme adopted by the invention is as follows: a disaster recovery apparatus based on multithread processing, comprising:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the method for disaster recovery based on multi-thread processing.
The method, the device and the system have the beneficial effects that: the invention stores data through the cache queues realized based on redis, has a plurality of storage queues, processes independent events based on the function of each cache queue, can call other cache queues to perform normal storage and query functions when one cache queue needs maintenance or has errors, and also stores all data in a hard disk for all data backup.
Drawings
FIG. 1 is a flowchart illustrating the steps of a multi-thread processing-based disaster recovery method according to the present invention;
FIG. 2 is a block diagram of an embodiment of the present invention;
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
As shown in fig. 1, the present invention provides a disaster recovery method based on multithread processing, which includes the following steps:
s101, receiving a user request, and initializing and creating a plurality of threads;
specifically, the number of the plurality of threads should be set to be one more than the number of the buffer queues because the first thread is used to perform a task of judging a user request.
S102, judging a user request according to a preset judgment condition;
s103, according to the judgment result, transmitting the data requested by the user to a cache queue through a thread;
specifically, according to the judgment result, if the user request is a qualified user request, the data is transmitted to the buffer queue, if the user request is not qualified, the data is deleted and alarm information is sent, wherein the number and the setting of the buffer queue can be changed according to the user requirements.
S104, storing the data requested by the user according to the setting of the cache queue;
further as a preferred embodiment of the method, the buffer queue is a buffer queue implemented based on redis.
Further as a preferred embodiment of the method, a first thread of the plurality of threads is used for running a task that determines whether the request meets the requirement, and the remaining threads are used for running tasks that transmit data to the respective cache queues.
Further, as a preferred embodiment of the method, the buffer queue includes a first type of redis buffer queue and a second type of redis buffer queue, where the first type of redis buffer queue is used to store data in a preset period, and the second type of redis buffer queue is used to store all data.
Specifically, because the storage space of the redis cache queue is limited, the first type of cache queue includes a plurality of cache queues, each cache queue respectively stores part of data which accords with the setting of the cache queue, and the data stored in each queue has the same part between every two cache queues, so that the purpose of disaster recovery backup is achieved, and the problem of insufficient storage space of the redis cache queue can be solved.
As a preferred embodiment of the method, the method stores the data requested by the user according to the setting of the buffer queue, and further includes the following steps:
and storing the data of the second type of redis cache queue to the hard disk.
Particularly, the data of the fully stored buffer queue is stored in the hard disk, because the capacity of the hard disk is enough, all the data can be stored, and the safety of data storage is further improved.
Further, as a preferred embodiment of the method, when one of the buffer queues needs to be maintained or an error occurs, the other buffer queues are called to store the data requested by the user.
As a further preferred embodiment of the method, the preset judgment condition is a target time limit.
The preset judgment condition can be changed into the time for sending the request or other conditions according to the requirement.
The invention is described in further detail below with reference to preferred practical application examples.
The specific embodiment of the invention is as follows:
receiving a user request, initializing to newly build six threads, wherein the first thread is used for determining whether the user request meets the requirement, the remaining threads are used for running a task of transmitting data to a cache queue, and a preset judgment condition is that a target time of the user request is less than five minutes after the current time and is within ten years after the current time, and if the user request meets the requirement, transmitting the data of the user request to the cache queue, wherein in the specific embodiment, the cache queue is implemented based on redis and comprises a first redis cache queue, a second redis cache queue, a third redis cache queue, a fourth redis cache queue and a fifth redis cache queue, the first redis cache queue is used for caching single-day data, the second redis cache queue is used for caching double-day data, and the third redis cache queue is used for caching data of nearly three days, and the fourth redis cache queue is used for caching all data, the fifth redis cache queue is used for caching data for nearly 3 hours, when the data are transmitted to the redis cache queue, if the data conform to the set range of the redis cache queue, the storage operation is carried out, if the data do not conform to the set range of the redis cache queue, the storage operation is ignored, if the data do not conform to the set range of the redis cache queue, the excess data are deleted through the deleting module if the data exceed the set range of the redis cache queue, and finally the data of the redis fourth cache queue are stored in the hard disk to carry out all data backup.
As shown in fig. 2, a disaster recovery system based on multi-thread processing includes:
the running module is used for receiving a user request and initializing and creating a plurality of threads;
the judging module is used for judging the user request according to a preset judging condition;
the transmission module is used for transmitting the data requested by the user to the cache queue through the thread at the same time according to the judgment result;
and the storage module is used for storing the data requested by the user according to the setting of the buffer queue.
Preferably, the disaster recovery system based on multithread processing further includes:
and the deleting module is used for deleting the data exceeding the setting of the buffer queue.
In some redis cache queues, in order to perform a read operation more quickly, a setting range of the cache queue may be small, and a deletion module needs to be operated at any time.
Based on the above method, the present invention further provides a disaster recovery device based on multithread processing, which includes:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the method for disaster recovery based on multi-thread processing.
The contents in the above method embodiments are all applicable to the present apparatus embodiment, the functions specifically implemented by the present apparatus embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present apparatus embodiment are also the same as those achieved by the above method embodiments.
From the above, the scheme of the present invention has the following advantages: the data are stored by utilizing a plurality of buffer queues realized by redis, and can be quickly read when the data need to be read; partial data is stored in each buffer queue, and partial same data is stored between the buffer queues, so that the purpose of disaster recovery backup can be achieved, and the problem of insufficient storage capacity caused by the fact that a redis buffer queue stores all data can be avoided; due to the fact that the plurality of buffer queues are arranged, when one of the buffer queues is in error or needs maintenance, the other buffer queues can be called to store data.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. A disaster recovery method based on multithread processing is characterized in that: the method comprises the following steps:
receiving a user request, and initializing and creating a plurality of threads;
judging the user request according to a preset judgment condition;
according to the judgment result, transmitting the data requested by the judged user to the cache queue through the thread at the same time;
and storing the data requested by the user according to the setting of the buffer queue.
2. The method of claim 1, wherein the method comprises: the buffer queue is realized based on redis.
3. The method of claim 1, wherein the method comprises: and the first thread in the multiple threads is used for running a task for judging whether the request meets the requirement, and the other threads are used for running the task for transmitting the data to each buffer queue.
4. The method of claim 2, wherein the method comprises: the cache queues comprise a first type of redis cache queue and a second type of redis cache queue, wherein the first type of redis cache queue is used for storing data in a preset period, and the second type of redis cache queue is used for storing all data.
5. The method of claim 4, wherein the method comprises: also comprises the following steps:
and storing the data of the second type of redis cache queue to the hard disk.
6. The method of claim 1, wherein the method comprises: further comprising: when one of the buffer queues needs maintenance or an error occurs, other buffer queues are called to store data requested by a user.
7. The method of claim 1, wherein the method comprises: the preset judgment condition is judged by a time limit.
8. A disaster recovery system based on multithread processing is characterized in that: the method comprises the following steps:
the running module is used for receiving a user request and initializing and creating a plurality of threads;
the judging module is used for judging the user request according to a preset judging condition;
the transmission module is used for transmitting the data requested by the user to the cache queue through the thread at the same time according to the judgment result;
and the storage module is used for storing the data requested by the user according to the setting of the buffer queue.
9. A multi-thread processing-based disaster recovery system according to claim 8, wherein: the method comprises the following steps:
and the deleting module is used for deleting the data exceeding the setting of the buffer queue.
10. A disaster recovery device based on multithread processing is characterized in that: the method comprises the following steps:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a method of disaster recovery based on multi-threaded processing as recited in any one of claims 1-7.
CN201911005259.4A 2019-10-22 2019-10-22 Disaster recovery method, system and device based on multithreading processing Pending CN110990176A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911005259.4A CN110990176A (en) 2019-10-22 2019-10-22 Disaster recovery method, system and device based on multithreading processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911005259.4A CN110990176A (en) 2019-10-22 2019-10-22 Disaster recovery method, system and device based on multithreading processing

Publications (1)

Publication Number Publication Date
CN110990176A true CN110990176A (en) 2020-04-10

Family

ID=70082284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911005259.4A Pending CN110990176A (en) 2019-10-22 2019-10-22 Disaster recovery method, system and device based on multithreading processing

Country Status (1)

Country Link
CN (1) CN110990176A (en)

Similar Documents

Publication Publication Date Title
EP3285187B1 (en) Optimized merge-sorting of data retrieved from parallel storage units
US20060288159A1 (en) Method of controlling cache allocation
US20180060145A1 (en) Message cache management for message queues
CN110196770B (en) Cloud system memory data processing method, device, equipment and storage medium
US9298765B2 (en) Apparatus and method for handling partially inconsistent states among members of a cluster in an erratic storage network
US9904470B2 (en) Tracking ownership of memory in a data processing system through use of a memory monitor
CN112039999A (en) Method and system for accessing distributed block storage system in kernel mode
CN108733585B (en) Cache system and related method
WO2022135160A1 (en) Releasing method and releasing system for buffer space, and electronic device and storage medium
US20080276045A1 (en) Apparatus and Method for Dynamic Cache Management
US20110302377A1 (en) Automatic Reallocation of Structured External Storage Structures
EP3391221B1 (en) System reliability by prioritizing recovery of objects
CN113190384A (en) Data recovery control method, device, equipment and medium based on erasure codes
CN117056054A (en) Interrupt control method, interrupt controller, computer device, and storage medium
US20110047553A1 (en) Apparatus and method for input/output processing of multi-thread
CN110990176A (en) Disaster recovery method, system and device based on multithreading processing
US10909094B1 (en) Migration scheduling for fast-mutating metadata records
US20090288089A1 (en) Method for prioritized event processing in an event dispatching system
US20230393782A1 (en) Io request pipeline processing device, method and system, and storage medium
CN111324668B (en) Database data synchronous processing method, device and storage medium
CN114020792A (en) Method, system, storage medium and server for supplementing historical data to real-time database
CN112181686A (en) Data processing method, device and system, electronic equipment and storage medium
CN112231290A (en) Method, device and equipment for processing local log and storage medium
US20060101469A1 (en) Method, controller, program product and services for managing resource element queues
CN112003860B (en) Memory management method, system and medium suitable for remote direct memory access

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20220929

Address after: Room 1602, 16th Floor, Building 18, Yard 6, Wenhuayuan West Road, Beijing Economic and Technological Development Zone, Daxing District, Beijing 100176

Applicant after: Beijing Lajin Zhongbo Technology Co.,Ltd.

Address before: 3rd Floor, Sagittarius Building A (Building), Wuxi Software Park, No. 18 Zhenze Road, Wuxi New District, Wuxi City, Jiangsu Province, 214000

Applicant before: WUXI TVMINING MEDIA SCIENCE & TECHNOLOGY Co.,Ltd.