CN111651459A - Service ID generation method based on Redis - Google Patents
Service ID generation method based on Redis Download PDFInfo
- Publication number
- CN111651459A CN111651459A CN202010513461.4A CN202010513461A CN111651459A CN 111651459 A CN111651459 A CN 111651459A CN 202010513461 A CN202010513461 A CN 202010513461A CN 111651459 A CN111651459 A CN 111651459A
- Authority
- CN
- China
- Prior art keywords
- node
- redis
- service
- current
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 5
- 238000013515 script Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 3
- 241000533950 Leucojum Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention relates to a service ID generating method based on Redis, which comprises the following steps: the ID generation center is connected with a Redis cluster, and initialization configuration is carried out on a node ID table of the Redis cluster, wherein the initialization configuration at least comprises the node ID for configuring each node in the Redis cluster and an initial timestamp for defining and generating the ID; the application service acquires a node ID of a certain node, a current timestamp of Redis and a node self-growth ID value of the current node; and generating a 64-bit service ID by adopting an ID generation algorithm based on the initial timestamp, the current timestamp, the node ID and the node self-increment ID value. The service ID generated by the invention has high performance, high availability and stable performance, can be flexibly expanded, and generates an ID presenting increasing trend.
Description
Technical Field
The invention relates to the technical field of ID generation, in particular to a service ID generation method based on Redis.
Background
With the development of services, the user population is larger and larger, and the service data volume is also continuously increased. When the service data volume is not large, the single-library single table can completely support the existing service, the data is larger, and the MySQL master-slave synchronous read-write separation can also support. However, as data grows, master-slave synchronization is not applicable, and the database needs to be divided into banks and tables, but a unique ID is needed to identify a piece of data after the data are divided into the banks and tables, and the self-increment ID of the database obviously cannot meet the requirement; the order and the coupon of a special point need to have a unique ID for identification. A system capable of generating a globally unique service ID is then necessary.
At present, two main unique ID generation schemes are available, the first is database ID self-increment, and the second is generation of unique UUID.
Although the first method is stable and convenient to generate, the pressure of the database is increased and is limited by the data volume; although the second method does not depend on the outside, the second method is not flexible enough, the generated ID has no meaning and no sequence, the storage of the data in the database is greatly influenced, and the query efficiency is low. The method designed by the inventor aims to overcome the defects of the two methods and design and realize a unique service ID generation method which is high in performance, stable, flexible in expansion and capable of presenting an increasing trend.
Disclosure of Invention
In view of the above, the present invention has been made to provide a Redis-based service ID generation method that overcomes or at least partially solves the above-mentioned problems.
According to one aspect of the invention, a Redis-based service ID generation method is provided, which comprises that an ID generation center is connected with a Redis cluster, and an initialization configuration is carried out on a node ID table of the Redis cluster, wherein the initialization configuration at least comprises the configuration of the node ID of each node in the Redis cluster and the definition of an initial timestamp for generating the ID; the application service acquires a node ID of a certain node, a current timestamp of Redis and a node self-growth ID value of the current node; and generating a 64-bit service ID by adopting an ID generation algorithm based on the initial timestamp, the current timestamp, the node ID and the node self-increment ID value.
According to a possible design, the acquiring, by the application service, the node ID of a certain node specifically includes: and calling an ID generation center interface by the application service, wherein the ID generation center acquires the node ID of a certain node in the cluster from the Redis node ID table.
According to a possible design, the acquiring, by the application service, a current timestamp of Redis and a node self-growth ID value of a current node specifically includes: and calling an ID generation center interface by the application service, wherein the ID generation center executes a function by using the Lua script, and acquires the current time stamp of Redis and the self-increment value of the current node.
According to one possible design, the ID generation center performs functions using Lua scripts to obtain the current timestamp of Redis and the incremental value of the current node: and acquiring the current timestamp of Redis and the node self-growth ID value of the current node according to the Redis TIME command by adopting a Lua script.
According to one possible design, the 64-bit traffic ID includes: a 1-bit identifier, a 41-bit timestamp, and a 22-bit set of ID values that includes a node ID and a node growth ID value.
According to one possible design, the node ID takes 12 bits and the node growth ID takes 10 bits in the 22-bit set of ID values.
According to one possible design, the generating the service ID specifically includes; firstly, acquiring a timestamp of Redis at the current millisecond level, subtracting an initial preset timestamp to obtain the millisecond number of a difference value, leftwards shifting 64-1-41 to 22 bits, then adding a value obtained by leftwards shifting 10 bits of a node ID of a current node, and finally adding a self-increment ID value of the node, wherein the generation formula is as follows:
(current timestamp-initial timestamp) < <22+ node ID < <10+ node self-increment ID value.
According to a second aspect, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed on a computer, causes the computer to perform the method of the first aspect or the method of any one of the possible designs of the first aspect.
According to a third aspect, there is provided a computing device comprising a memory having stored therein executable code, and a processor which, when executing the executable code, implements the method of any of the first aspects or any one of the possible designs of the first aspects.
The service ID generated by the invention has high performance, high availability and stable performance, can be flexibly expanded, and generates an ID presenting increasing trend.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow diagram of a method for generating a service ID based on Redis according to an embodiment of the present invention;
fig. 2 is an application scenario diagram of a service ID generation method based on Redis according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a generated service ID according to an embodiment of the present invention.
Detailed Description
The terms "first," "second," and the like in the description and in the claims and in the drawings of the present invention are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises" and "comprising," as well as any variations thereof, are intended to cover a non-exclusive inclusion, such as a list of steps or elements. A method, system, article, or apparatus is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, system, article, or apparatus.
Before explaining the embodiments of the present application in detail, some abbreviations and terms referred to in the embodiments of the present application will be introduced.
Redis: is a self-growing ID value used to maintain a table of node IDs and each node generation ID, hereinafter collectively referred to as Redis;
an ID generation center: the service is a service for generating IDs, and all service IDs are generated from an ID generation center, and hereinafter, all are referred to as an ID generation center.
Application service: the application service is an application service for each service for which an ID is to be generated, and is all referred to as an application service in the past.
The technical solution of the present invention is further described in detail with reference to the accompanying drawings and embodiments.
As shown in fig. 1 to 3, an embodiment of the present invention provides a method for generating a service ID based on Redis, including:
And 103, generating a 64-bit service ID based on the initial timestamp, the current timestamp, the node ID and the node self-increment ID value by adopting an ID generation algorithm.
In one example, in step 102, the obtaining, by the application service, a node ID of a certain node specifically includes:
and calling an ID generation center interface by the application service, wherein the ID generation center acquires the node ID of a certain node in the cluster from the Redis node ID table.
In one example, in step 102, the acquiring, by the application service, a current timestamp of Redis and a node self-growth ID value of a current node specifically includes:
and calling an ID generation center interface by the application service, wherein the ID generation center executes a function by using the Lua script, and acquires the current time stamp of Redis and the self-increment value of the current node. Specifically, a Lua script is adopted to obtain a current timestamp of Redis and a node self-growth ID value of a current node according to a Redis TIME command.
In one example, in step 103, the generating a service ID specifically includes:
firstly, acquiring a timestamp of Redis at the current millisecond level, subtracting an initial preset timestamp to obtain the millisecond number of a difference value, shifting the millisecond number to the left by 64-1-41 bits to 22 bits, then adding a value obtained by shifting the node ID of the current node to the left by 10 bits, and finally adding the self-increment ID value of the node. The generation formula is as follows:
(Current timestamp-initial timestamp) < <22+ node ID < <10+ node self-increment ID value
Illustratively, for example, the current timestamp is 1589376049749, the initial timestamp is 1577808000000, the difference is 11568049749, and the left shift is 22 bits 48519917334429696; assuming that the current node ID is 1, left-shifted by 10 bits to 1024, and finally adding the self-increment ID value of the current node, e.g., 5, the resulting service ID is
48519917334429696+1024+5＝48519917334430725。
The principle of the method is as follows:
and the ID generation center avoids abnormal scenes under high concurrency by utilizing Redis single-thread operation, performs high-performance atomic execution by utilizing a Lua script execution function, and generates a service ID at each node.
2. Service ID description
The generated service ID is 64 bits, and long type storage can be directly used in Java, so that the method is more convenient and better in universality compared with 96bit and 128bit shift operation. The generation method refers to a snowflake algorithm. The specific division is as shown in fig. 3:
comprises 4 parts
A: 1bit identification, since long basic type is signed in Java, the most significant bit is the sign bit, the positive number is 0, the negative number is 1, so id is generally a positive number, and the most significant bit is 0.
And B, storing the time stamp difference value of Redis by using 41 bits, wherein the time stamp difference value is accurate to millisecond level. (1L < <41)/(1000L 3600 x 24 x 365) can be used for 69 years.
C: the ID of the Redis node is stored by using 12 bits, and the ID of the Redis node is stored by 4096 nodes at most.
D: using 10 bits to deposit the self-growing ID value means that each node can generate up to 1024 IDs per millisecond.
The unit time period for generating the ID is accurate to 1 millisecond by adopting a millisecond-level timestamp difference value, so that the unit time is reduced to the maximum extent, and the repeatability of the ID is avoided; the smaller the unit time, the larger the number of supportable generation IDs; the usable time length is increased by adopting the timestamp difference; because the difference value is millisecond and is larger and larger, the generated ID is increased along with the trend of millisecond, when data is written in the database, frequent addressing of the disk is avoided, and the IO speed of the disk is greatly accelerated.
The Redis node ID realizes the function of horizontal extension according to the node extension of the Redis cluster, so that the nodes can be extended at any time when the subsequent service data volume is increased, and the upper limit which can be generated by the ID in unit time is increased;
the ID value of the Redis node can be increased by a single node in unit time (1 millisecond), and the maximum 1024 self-increment values are supported. The self-increment ID of the single node uses Lua script and an INCR self-increment counting command of Redis, and the failure time is set to be 1 millisecond.
In the above embodiment, the last 22 bits of the 64-bit ID value are divided in a pattern in which the first 12 bits are used as the node ID value and the last 10 bits are used as the self-increment ID value, i.e., 12+ 10. The last 22 bits can also be divided in other ways according to the meaning represented by the actual usage scenario, for example, the first 7 bits are used as node ID values, and the last 15 bits are used as self-increment ID values, i.e. the pattern division of 7+15, and other divisions, such as the pattern division of 14+8, are also possible.
According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with fig. 1.
According to an embodiment of yet another aspect, there is also provided a computing device comprising a memory and a processor, the memory having stored therein executable code, the processor, when executing the executable code, implementing the method described in connection with fig. 1.
The above embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, it should be understood that the above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A service ID generation method based on Redis is characterized by comprising the following steps:
the ID generation center is connected with a Redis cluster, and initialization configuration is carried out on a node ID table of the Redis cluster, wherein the initialization configuration at least comprises the node ID for configuring each node in the Redis cluster and an initial timestamp for defining and generating the ID;
the application service acquires a node ID of a certain node, a current timestamp of Redis and a node self-growth ID value of the current node;
and generating a 64-bit service ID by adopting an ID generation algorithm based on the initial timestamp, the current timestamp, the node ID and the node self-increment ID value.
2. The method for Redis-based service ID generation according to claim 1, wherein the acquiring, by the application service, the node ID of a certain node specifically comprises:
and calling an ID generation center interface by the application service, wherein the ID generation center acquires the node ID of a certain node in the cluster from the Redis node ID table.
3. The method for generating a business ID based on Redis according to claim 1, wherein the acquiring, by the application service, a current timestamp of Redis and a node self-growth ID value of a current node specifically includes:
and calling an ID generation center interface by the application service, wherein the ID generation center executes a function by using the Lua script, and acquires the current time stamp of Redis and the self-increment value of the current node.
4. The Redis-based service ID generation method according to claim 3, wherein the ID generation center performs a function using a Lua script, obtains a Redis current timestamp, and a self-increment value of a current node:
and acquiring the current timestamp of Redis and the node self-growth ID value of the current node according to the Redis TIME command by adopting a Lua script.
5. The Redis-based service ID generation method according to claim 1, wherein the 64-bit service ID comprises:
a 1-bit identifier, a 41-bit timestamp, and a 22-bit set of ID values that includes a node ID and a node growth ID value.
6. A Redis-based traffic ID generation method according to claim 5, wherein in said set of 22-bit ID values, a node ID takes 12 bits and a node growth ID value takes 10 bits.
7. The Redis-based service ID generation method according to claim 1, wherein the generating a service ID specifically comprises;
firstly, acquiring a timestamp of Redis at the current millisecond level, subtracting an initial preset timestamp to obtain the millisecond number of a difference value, leftwards shifting 64-1-41 to 22 bits, then adding a value obtained by leftwards shifting 10 bits of a node ID of a current node, and finally adding a self-increment ID value of the node, wherein the generation formula is as follows:
(current timestamp-initial timestamp) < <22+ node ID < <10+ node self-increment ID value.
8. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-7.
9. A computing device comprising a memory and a processor, wherein the memory has stored therein executable code that, when executed by the processor, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010513461.4A CN111651459A (en) | 2020-06-08 | 2020-06-08 | Service ID generation method based on Redis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010513461.4A CN111651459A (en) | 2020-06-08 | 2020-06-08 | Service ID generation method based on Redis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111651459A true CN111651459A (en) | 2020-09-11 |
Family
ID=72347102
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010513461.4A Pending CN111651459A (en) | 2020-06-08 | 2020-06-08 | Service ID generation method based on Redis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111651459A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204441A (en) * | 2021-05-07 | 2021-08-03 | 郑州信大捷安信息技术股份有限公司 | Multi-application ID generation method and system |
| CN114172792A (en) * | 2021-12-13 | 2022-03-11 | 武汉众邦银行股份有限公司 | Method and device for realizing sequence number generation method for ensuring high availability of service |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769667A (en) * | 2012-06-29 | 2012-11-07 | 北京奇虎科技有限公司 | Method, equipment and system for generating ID |
| CN103823810A (en) * | 2012-11-19 | 2014-05-28 | 中国电信股份有限公司 | Method, device and system for generating unique keys in distributed system |
| CN109639775A (en) * | 2018-11-27 | 2019-04-16 | 湖南蚁为软件有限公司 | Global monotonic increase ID generation method, device, system and equipment |
| CN110147281A (en) * | 2019-05-15 | 2019-08-20 | 上海淇毓信息科技有限公司 | Optimize method, apparatus, the electronic equipment that snowflake algorithm is applied in financial business |
| CN110263565A (en) * | 2019-06-28 | 2019-09-20 | 北京百度网讯科技有限公司 | Method and apparatus for calling service |
| CN110554732A (en) * | 2019-08-22 | 2019-12-10 | 北京奇艺世纪科技有限公司 | identification number generation method and device, electronic equipment and storage medium |
| CN110737806A (en) * | 2019-09-09 | 2020-01-31 | 贝壳技术有限公司 | Global ID generation method and device for business entity |
| CN111046057A (en) * | 2019-12-26 | 2020-04-21 | 京东数字科技控股有限公司 | Data processing method and device for server cluster, computer equipment and medium |
-
2020
- 2020-06-08 CN CN202010513461.4A patent/CN111651459A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769667A (en) * | 2012-06-29 | 2012-11-07 | 北京奇虎科技有限公司 | Method, equipment and system for generating ID |
| CN103823810A (en) * | 2012-11-19 | 2014-05-28 | 中国电信股份有限公司 | Method, device and system for generating unique keys in distributed system |
| CN109639775A (en) * | 2018-11-27 | 2019-04-16 | 湖南蚁为软件有限公司 | Global monotonic increase ID generation method, device, system and equipment |
| CN110147281A (en) * | 2019-05-15 | 2019-08-20 | 上海淇毓信息科技有限公司 | Optimize method, apparatus, the electronic equipment that snowflake algorithm is applied in financial business |
| CN110263565A (en) * | 2019-06-28 | 2019-09-20 | 北京百度网讯科技有限公司 | Method and apparatus for calling service |
| CN110554732A (en) * | 2019-08-22 | 2019-12-10 | 北京奇艺世纪科技有限公司 | identification number generation method and device, electronic equipment and storage medium |
| CN110737806A (en) * | 2019-09-09 | 2020-01-31 | 贝壳技术有限公司 | Global ID generation method and device for business entity |
| CN111046057A (en) * | 2019-12-26 | 2020-04-21 | 京东数字科技控股有限公司 | Data processing method and device for server cluster, computer equipment and medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204441A (en) * | 2021-05-07 | 2021-08-03 | 郑州信大捷安信息技术股份有限公司 | Multi-application ID generation method and system |
| CN113204441B (en) * | 2021-05-07 | 2022-04-05 | 郑州信大捷安信息技术股份有限公司 | Multi-application ID generation method and system |
| CN114172792A (en) * | 2021-12-13 | 2022-03-11 | 武汉众邦银行股份有限公司 | Method and device for realizing sequence number generation method for ensuring high availability of service |
| CN114172792B (en) * | 2021-12-13 | 2023-07-28 | 武汉众邦银行股份有限公司 | Method and device for realizing sequence number generation method for ensuring high availability of service |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110442560B (en) | Log replay method, device, server and storage medium | |
| CN108241715A (en) | A kind of distributed data base major key generation method and device | |
| CN111651459A (en) | Service ID generation method based on Redis | |
| CN105824849A (en) | Data import method and adapter | |
| CN102194009A (en) | Database hosting method and database hosting platform system | |
| CN112612799B (en) | Data synchronization method and terminal | |
| CN109165175B (en) | Equipment identifier generation method and device | |
| CN109145053B (en) | Data processing method and device, client and server | |
| CN112163038A (en) | Cross-cluster data synchronization method, device, equipment and storage medium | |
| CN112241437A (en) | Loop control method, device and equipment for multi-master synchronization of database and storage medium | |
| CN114297198A (en) | Method and device for generating serial number in distributed system | |
| CN113064903A (en) | Global ID generation method, device, equipment and storage medium | |
| CN106993022B (en) | Identification management method for cluster, identification server and corresponding system | |
| CN112764679B (en) | Dynamic capacity expansion method and terminal | |
| CN111339055B (en) | Big data cluster capacity expansion method and device | |
| CN111522827B (en) | Data updating method and device and electronic equipment | |
| CN116303789A (en) | Parallel synchronization method and device for multi-fragment multi-copy database and readable medium | |
| CN113835625B (en) | Data storage method, device, equipment and storage medium based on sub-path | |
| CN114153893A (en) | Redis-based distributed ID generation method, device and medium | |
| CN115794813A (en) | Data line identifier generation method and query and partition exchange method and device | |
| CN113010495B (en) | Database optimization method and device | |
| CN110098960B (en) | Secret-free automatic configuration method and device for distributed block storage system | |
| CN114547206A (en) | Data synchronization method and data synchronization system | |
| CN114253870A (en) | Method, system, device and medium for updating L2P table | |
| CN114254030A (en) | Data synchronization method, system, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200911 |