CN116308639A - Design method for high concurrency order ID under distributed architecture - Google Patents
Design method for high concurrency order ID under distributed architecture Download PDFInfo
- Publication number
- CN116308639A CN116308639A CN202310181856.2A CN202310181856A CN116308639A CN 116308639 A CN116308639 A CN 116308639A CN 202310181856 A CN202310181856 A CN 202310181856A CN 116308639 A CN116308639 A CN 116308639A
- Authority
- CN
- China
- Prior art keywords
- order
- redis
- bit
- distributed architecture
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000013461 design Methods 0.000 title claims abstract description 9
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 22
- 241000533950 Leucojum Species 0.000 claims abstract description 20
- 238000013515 script Methods 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 5
- 230000006870 function Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
- G06F9/45508—Runtime interpretation or emulation, e g. emulator loops, bytecode interpretation
- G06F9/45512—Command shells
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Physics & Mathematics (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a design method for a high concurrency order ID under a distributed architecture, which comprises the following specific steps: step one, building a high-availability Redis Cluster as a distributed ID generation server; step two, deploying the Lua script based on the snowflake algorithm to a Redis Cluster; step three, the order service generates a global unique ID by calling a Lua script function or generates a self-increasing ID by calling a Redis built-in command INCR; and fourthly, distinguishing order services with personalized order ID requirements through parameters, so as to deploy a plurality of different Lua scripts. Through the excellent and reliable Redis cluster as the ID-Generator-Server and combining with the Lua script and the Redis atomic self-addition command, a set of system is realized for multiple choices, and the system is high in concurrence, high in availability, high in reliability, easy to deploy and extensible.
Description
Technical Field
The invention relates to an Internet order ID generation system, in particular to a design method of a high concurrency order ID under a distributed architecture.
Background
The existing order ID (identity identification number) generation method mainly comprises a UUID (universal unique identification code) method and a SnowFlake algorithm (SnowFlake) method, and a database self-increasing method. Although the UUID method can keep global uniqueness, the UUID method has no practical meaning, can not be used for sorting and indexing, and has less practical service use; snowflake algorithm generally depends on Java Web service (Java Web, which is a technical stack for solving the related web Internet field by Java technology) and a Zookeeper registry (Zookeeper is a distributed, open source distributed application coordination service, is the realization of Chubbby of Google, is an open source and is an important component of Hadoop and Hbase), and the practical concurrency capability is limited by a Java Web container, so that more machines need to be deployed for improving the concurrency of a system; the database self-increasing method is simple to realize, but is easily limited by the performance of the database, and the ID rule of the database self-increasing method is easily found.
Disclosure of Invention
Aiming at the problems, the invention provides a design method for a high concurrency order ID under a distributed architecture, which realizes a set of multiple choices of a system by combining a Lua script and a Redis atomic self-increasing command (capable of setting a self-increasing step length) through taking an excellent and reliable Redis cluster as an ID-Generator-Server, and has the advantages of high concurrency, high availability, high reliability, easy deployment and expandability.
Noun interpretation:
1. redis: an open source high performance, easily expandable key-value pair cache database written in ANSIC language.
2. Lua: the method is a lightweight script language, can be conveniently built in Redis, can be easily called by C/C++ codes, and can also be used for calling functions of C/C++ in reverse.
3. ID-Generator-Server: the order ID generation system designed by the invention provides high-concurrency and high-reliability service based on the Redis cluster.
4. The method has high availability: refers to improving the usability of systems and applications by minimizing downtime due to routine maintenance operations (planned) and sudden system crashes (unplanned).
In order to achieve the technical purpose and the technical effect, the invention is realized by the following technical scheme: a design method for high concurrency order ID under a distributed architecture comprises the following specific steps:
step one, building a high-availability Redis cluster as a distributed ID generation server;
step two, deploying the Lua script based on the snowflake algorithm to a Redis cluster;
step three, the order service generates a global unique ID by calling a Lua script function or generates a self-increasing ID by calling a Redis built-in command INCR;
and fourthly, distinguishing order services with personalized order ID requirements through parameters, so as to deploy a plurality of different Lua scripts.
Furthermore, the snowflake algorithm is built in the order service, so that the high availability of the system is met through the realization of the snowflake algorithm built in the service, and the service unavailability caused by downtime of the Redis cluster is prevented.
Further, an autonomous selection ID generation algorithm can be performed according to id_strategy (order ID generation policy), which includes two kinds of ID generation algorithms:
the first initial version snowflake algorithm is defaulted, namely, the snowflake algorithm consists of 1-bit identification bit, 41-bit timestamp bit, 10-bit working machine ID and 12-bit random number, and can support 69 years;
the second type is a snowflake algorithm of an oversized cluster version, namely, the snowflake algorithm consists of 1-bit identification bit, 39-bit timestamp bit, 12-bit working machine ID and 12-bit random number, and can support 17 years.
Furthermore, the Redis cluster version should be greater than or equal to 5.0, and the Redis Cluster5.0 defaults to adopt a result synchronization mode, so that performance loss caused by multiple distribution of the Lua script is avoided.
Further, when the INCR command is adopted to acquire the global ID, setting the step size through parameters is supported, and the default step size is 1.
Further, when the snowflake algorithm is adopted, besides returning the snowflake ID, the hashed machine ID is returned so as to be used by a local machine cache.
Further, a monitoring fusing mechanism is adopted:
if the Redis cluster reading failure times exceeds a threshold value within a certain period of time, switching to a local mode, namely adopting business service to locally generate an order ID;
after the local mode operates for a period of time, the Redis clusters are tried to be read at intervals of a certain period, if the success times of the period of time exceeds a threshold value, the mode is switched to the cluster mode, the Redis clusters are preferentially read, and then the business service is adopted to locally generate the order ID.
The monitoring fusing mechanism ensures the fault tolerance of the middleware and realizes high availability and high concurrency in the whole system level.
Compared with the prior art, the invention has the beneficial effects that:
the method comprises the following steps: through the excellent and reliable Redis cluster as the ID-Generator-Server, and the combination of the Lua script and the Redis atomic self-increasing command, a set of system multiple choices is realized.
And two,: high concurrency, redis cluster local environment is observed to reach 20w/s concurrency, which is far more than Java Web service based on Tomcat.
And thirdly,: the availability is high, the Redis clustering deployment greatly enhances the system availability, and meanwhile, the system availability is ensured by combining with the local machine ID cache after Hash.
Fourth, it is: the method is extensible, and through the plug-in deployment of the Lua script, the strategy of order ID generation is greatly expanded, and the client does not need to make any modification.
Fifth, it is: the method is easy to deploy, quick deployment can be realized by migrating the related Lua script, and the method is simple and easy to operate.
Drawings
FIG. 1 is a schematic diagram of a method for designing a high concurrency order ID under a distributed architecture according to the present invention.
Detailed Description
The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, so that those skilled in the art can better understand the present invention and implement it, but the examples are not limited thereto.
As shown in fig. 1, a design method for high concurrency order ID under a distributed architecture specifically includes the following steps:
step one, constructing a highly available Redis cluster as an ID-Generator-Server; redis clusters must be sufficiently stable to provide fast recovery from failure.
Deploying the Lua script to a Redis cluster;
STEP three, the business Service is connected to the Redis cluster and transmits related request parameters, such as ID_Strategy (order ID generation Strategy), service_Name (Service Name), IP (order IP address), ID_STEP (Redis self-increment STEP size);
step four, when the ID_Strategy is selected as SNOWFLAKE, the service_Name and the IP are necessary parameters for calculating the machine ID by using a consistency Hash algorithm, and different services are distinguished by using different Redis Key (Key of a Redis string data structure); the return value includes a machine ID in addition to the order ID;
fifthly, when the ID_Strategy is selected as INCR, the service_Name is a necessary parameter for distinguishing different services by using different Redis keys; id_step is an optional parameter, and represents a self-increment STEP size, with a default value of 1, different services may transmit different STEP sizes, and the order IDs are independent from each other.
The monitoring process of the fusing mechanism can continuously scan the service system and the Redis cluster middleware to realize policy implementation:
1. if the Redis cluster read failure times exceeds a threshold value within a certain period of time, switching to a local mode, namely adopting business service to locally generate order IDs.
2. After the local mode operates for a period of time, the Redis clusters are tried to be read at intervals of a certain period, if the success times of the period of time exceeds a threshold value, the mode is switched to the cluster mode, the Redis clusters are preferentially read, and then the business service is adopted to locally generate the order ID.
3. When the fault cannot be recovered, the Redis cluster can be restarted manually, binary data of the Redis (backup Redis data) can be restarted when necessary, and meanwhile, only a local mode, namely, the local generation order ID of the business service is used. In addition, in actual production, monitoring is an essential important link, and in the invention, the monitoring can have two dimensions: redis memory size monitoring and Redis response time-consuming monitoring.
The design method of the high concurrency order ID under the distributed architecture realizes a set of multiple choices of the system by taking the excellent and reliable Redis cluster as the ID-Generator-Server and combining with the Lua script and the Redis INCR command, and has the advantages of high concurrency, high availability, high reliability, easy deployment and expandability. Meanwhile, two modes of local and Redis clustering are combined to generate order IDs, so that high availability of service is guaranteed. The monitoring fusing mechanism ensures the fault tolerance of the middleware and realizes high availability and high concurrency in the whole system level.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures disclosed herein or modifications in equivalent processes, or any application, directly or indirectly, within the scope of the invention.
Claims (7)
1. A design method for a high concurrency order ID under a distributed architecture is characterized in that: the method comprises the following specific steps:
step one, building a high-availability Redis cluster as a distributed ID generation server;
step two, deploying the Lua script based on the snowflake algorithm to a Redis cluster;
step three, the order service generates a global unique ID by calling a Lua script function or generates a self-increasing ID by calling a Redis built-in command INCR;
and fourthly, distinguishing order services with personalized order ID requirements through parameters, so as to deploy a plurality of different Lua scripts.
2. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: an order service is internally provided with a snowflake algorithm, so that service unavailability caused by downtime of a Redis cluster is prevented.
3. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: an autonomous selection ID generation algorithm can be performed according to id_strategy, which includes two types:
the first initial version snowflake algorithm is defaulted, namely, the snowflake algorithm consists of 1-bit identification bit, 41-bit timestamp bit, 10-bit working machine ID and 12-bit random number, and can support 69 years;
the second type is a snowflake algorithm of an oversized cluster version, namely, the snowflake algorithm consists of 1-bit identification bit, 39-bit timestamp bit, 12-bit working machine ID and 12-bit random number, and can support 17 years.
4. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: redis cluster version should be greater than or equal to 5.0, and Redis Cluster5.0 defaults to adopt a result synchronization mode, so that performance loss caused by multiple distribution of Lua scripts is avoided.
5. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: when the INCR command is adopted to acquire the global ID, the default step length is 1 by setting the step length through parameters.
6. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: when the snowflake algorithm is adopted, besides returning the snowflake ID, the hashed machine ID is returned so as to be used by a local machine cache.
7. The method for designing a high concurrency order ID under a distributed architecture according to claim 1, wherein: the monitoring fusing mechanism is adopted:
if the Redis cluster reading failure times exceeds a threshold value within a certain period of time, switching to a local mode, namely adopting business service to locally generate an order ID;
after the local mode operates for a period of time, the Redis clusters are tried to be read at intervals of a certain period, if the success times of the period of time exceeds a threshold value, the mode is switched to the cluster mode, the Redis clusters are preferentially read, and then the business service is adopted to locally generate the order ID.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310181856.2A CN116308639A (en) | 2023-03-01 | 2023-03-01 | Design method for high concurrency order ID under distributed architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310181856.2A CN116308639A (en) | 2023-03-01 | 2023-03-01 | Design method for high concurrency order ID under distributed architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116308639A true CN116308639A (en) | 2023-06-23 |
Family
ID=86802481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310181856.2A Pending CN116308639A (en) | 2023-03-01 | 2023-03-01 | Design method for high concurrency order ID under distributed architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116308639A (en) |
-
2023
- 2023-03-01 CN CN202310181856.2A patent/CN116308639A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
| US7363346B2 (en) | Reliably storing information across multiple computers such as in a hive of computers | |
| US20190079839A1 (en) | Locality based quorums | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| CN110012088B (en) | Kubernets-based Redis master-slave cluster automatic deployment scheme | |
| US20070094659A1 (en) | System and method for recovering from a failure of a virtual machine | |
| US20090144338A1 (en) | Asynchronously replicated database system using dynamic mastership | |
| US20090144220A1 (en) | System for storing distributed hashtables | |
| CN101751415B (en) | Metadata service system, metadata synchronized method and writing server updating method | |
| WO2017071563A1 (en) | Data storage method and cluster management node | |
| WO2016177130A1 (en) | Method and device for selecting communication node | |
| US20120278429A1 (en) | Cluster system, synchronization controlling method, server, and synchronization controlling program | |
| CN109086295A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| CN111259071A (en) | Concurrent access control method in distributed database system | |
| US20160070494A1 (en) | Highly performant reliable message storage using in-memory replication technology | |
| JP2011210107A (en) | Message queue management system, lock server, message queue management method, and message queue management program | |
| Khan et al. | Highly available Hadoop namenode architecture | |
| US10048983B2 (en) | Systems and methods for enlisting single phase commit resources in a two phase commit transaction | |
| CN116680256B (en) | Database node upgrading method and device and computer equipment | |
| KR101778907B1 (en) | Cache and non-cache usage in a distributed storage system | |
| CN107943615B (en) | Data processing method and system based on distributed cluster | |
| US20090144333A1 (en) | System for maintaining a database | |
| CN112925828A (en) | Distributed multi-node query method and system | |
| CN115955488B (en) | Distributed storage copy cross-machine room placement method and device based on copy redundancy | |
| CN116308639A (en) | Design method for high concurrency order ID under distributed architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |