CN114116060A - Globally unique distributed ID generation method - Google Patents
Globally unique distributed ID generation method Download PDFInfo
- Publication number
- CN114116060A CN114116060A CN202111421682.XA CN202111421682A CN114116060A CN 114116060 A CN114116060 A CN 114116060A CN 202111421682 A CN202111421682 A CN 202111421682A CN 114116060 A CN114116060 A CN 114116060A
- Authority
- CN
- China
- Prior art keywords
- globally unique
- machine
- bit
- distributed
- generation method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
Abstract
The invention discloses a globally unique distributed ID generation method, which comprises the steps of pre-configuring an embedded release mode, and acquiring a local IP when a service instance is started; the method comprises the steps that a machine serial number is returned after a local machine generates an instance list file and registers a service instance; performing bit operation on the IP and the machine serial number to obtain a globally unique machine bit; performing right bit operation on the generated ID through various displacement number information and the maximum limiting sequence of the bit when the IDMeta defines the ID generation, and performing AND operation on the ID and the maximum limiting sequence of the bit to obtain a value when the ID is generated; and performing left bit operation on the machine bit, the version number defined by the IdMeta, the time type, the current time and the current machine serial number to obtain the unique distributed ID. The method of the invention can generate the ID with global uniqueness without depending on the central service when in embedded use, solves the problem of ID repetition, saves storage space, improves performance and meets the requirement of faster service growth speed.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a globally unique distributed ID generation method.
Background
With the increasing complexity of information systems, the number of services and middleware required by the systems is also increasing, a large amount of data and messages need to be uniquely identified, and when the amount of service data is not large, the service ID can be generated by a plurality of methods, mainly by self-increment ID. The ID of the current service system uses the self-increment field of the database and completely depends on the database, but in a large complex system, orders, products and after-sales can be divided into databases and tables, and can be distributed in a plurality of tables and a plurality of libraries, so that a large number of repeated IDs are generated, and the service is in a problem. For large-scale complex service and distributed high-concurrency application scenes, troubles are brought when database transplantation, capacity expansion, data washing, database splitting, table splitting and other operations are carried out, and the traditional UUID or database self-increment ID cannot meet the requirements.
The distributed ID can generate the unique ID quickly and stably and has orderliness, and the current popular distributed ID generation scheme uses a snowflake algorithm. However, when the ID is generated in an embedded manner in a distributed system environment, if the system does not rely on a central service, the situation of generating a repeated ID exists, manual assignment is required during table ID assignment, deep integration is not performed, and meanwhile, necessary APIs are lacked, such as inverse solution and translation time, so that the requirements of services cannot be met. Therefore, a high-performance and highly reliable distributed ID generation method is needed.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a globally unique distributed ID generation method for overcoming the defects involved in the background art, so that the ID generated in the embedded release mode has global uniqueness without increasing the ID length.
The technical problems of the invention are realized by adopting the following technical scheme:
a globally unique distributed ID generation method, comprising:
an embedded release mode is configured in advance, and a local IP is obtained when a service instance is started;
the method comprises the steps that an instance list file is generated in a local machine, the service instance is registered, and a machine serial number is returned after the registration;
performing bit operation on the IP and the machine serial number to obtain a globally unique machine bit;
defining each item of displacement digit information and the maximum limiting sequence of the digit when the ID is generated through idMeta, carrying out right digit operation on the generated ID, and carrying out AND operation on the right digit and the maximum limiting sequence of the digit to obtain a generation time numerical value of the ID;
and performing left bit operation on the machine bit, the version number defined by the idMeta, the time type, the current time and the current machine serial number to obtain the unique distributed ID.
Preferably, the globally unique distributed ID generation method further includes integrating a Spring frame and a Mybatis frame, developing an interceptor using a Mybatis frame interceptor interface, intercepting when data is inserted into the database, and generating a distributed ID for assignment if an ID field of a data entity is empty after intercepting a corresponding data entity; and then integrating the globally unique distributed ID generation method with the interception and springboot framework to generate a rapidly integrated starter dependent package.
Preferably, the numerical key information of the ID generation includes, but is not limited to, generation time, service type, machine bit and serial number.
Preferably, the optimal operation bit number is found by simulating bit operation, and the QPS is balanced under two performance scenes of maximum peak value and minimum granularity.
Preferably, the method comprises the steps of combining and packaging with the SpingBoot, automatically generating and assigning the distributed ID when the table data is inserted, and independently obtaining the ID service.
Preferably, the API is provided with a generation ID, an inverse solution ID, a translation time and a translation machine bit when the method is implemented.
Preferably, when the local IP is acquired when the system service instance is started, the local port number and the IP are acquired, whether the IP and the port number are correct or not is determined, if the IP and the port number are abnormal, an abnormal alarm instruction is sent to notify the abnormality, and if the IP and the port number are correct, the local is continuously operated to generate the instance list file.
Preferably, in the process of starting the service instance, a generation thread is started, a preset number of IDs are generated in a preset memory, and after the service instance is started, the IDs are taken out from the preset memory, and the preset number of IDs are generated again in the emptied memory.
Preferably, the returned sequence number is divided into two sections, the first section serves a central server and an REST issuing mode, multithreading processing is used, and the second section serves an embedded issuing mode; if one of the service modes is not available, fallback to the other publishing mode uses the locally provisioned machine ID.
Preferably, the server machine number is divided into two segments, a first segment starting at 0 and going up and a second segment starting at 128 and going down, and a dynamically adjusted boundary is implemented.
The invention has the advantages that the generation of the integrated distributed ID of the distributed system is more efficient and convenient, the embedded system does not depend on central service when in use, the ID with global uniqueness can be generated, the problem of ID repetition is solved, the production time of the distributed ID and the service port information of the service can be received in real time through rich API interfaces, the storage space is saved, the performance is improved, and the requirement of high service growth speed is met.
Detailed Description
The technical solution of the present invention will be described in further detail below. It is understood that the embodiments described are only a few embodiments, not all embodiments, and that all other embodiments obtained by those skilled in the art without the use of inventive faculty are within the scope of the invention.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components and/or sections, these elements, components and/or sections should not be limited by these terms.
A globally unique distributed ID generation method, comprising:
an embedded release mode is configured in advance, and a local IP is obtained when a service instance is started;
the method comprises the steps that an instance list file is generated in a local machine, the service instance is registered, and a machine serial number is returned after the registration;
performing bit operation on the IP and the machine serial number to obtain a globally unique machine bit;
defining each item of displacement digit information and the maximum limiting sequence of the digit when the ID is generated through idMeta, carrying out right digit operation on the generated ID, and carrying out AND operation on the right digit and the maximum limiting sequence of the digit to obtain a generation time numerical value of the ID;
and performing left bit operation on the machine bit, the version number defined by the idMeta, the time type, the current time and the current machine serial number to obtain the unique distributed ID.
In this embodiment, the globally unique distributed ID generation method further includes integrating a Spring frame and a Mybatis frame, developing an interceptor using a Mybatis frame interceptor interface, intercepting when data is inserted into the database, and generating a distributed ID for assignment if an ID field of a data entity is empty after intercepting a corresponding data entity; and integrating the globally unique distributed ID generation method with the interception and springboot framework to generate a rapidly integrated start dependent package.
In this embodiment, the numerical key information of the ID generation includes, but is not limited to, generation time, service type, machine bit, and serial number.
In this embodiment, the optimal operation bit number is found by simulating bit operation, and a balanced QPS is provided in the two performance scenarios of maximum peak value and minimum granularity.
In this embodiment, the method includes packaging in combination with the spinbooth, and automatically performs distributed ID generation and assignment when table data is inserted, and may also be separately obtained by IdService.
In the embodiment, when the method is implemented, the API is provided with a generation ID, an inverse solution ID, a translation time and a translation machine bit. The generation time and the generation source are known according to the inverse solution ID, and the storage space occupied by the traditional field can be saved.
In this embodiment, when the local IP is obtained when the system service instance is started, the local port number and the IP are obtained, whether the IP and the port number are correct or not is determined, if the IP and the port number are abnormal, an abnormal alarm instruction is sent to notify the abnormality, and if the IP and the port number are correct, the local is continuously operated to generate the instance list file.
In this embodiment, in the process of starting the service instance, a generation thread is started, a preset number of IDs are generated in a preset memory, and after the service instance is started, the IDs are taken out from the preset memory, and the preset number of IDs are generated again in the emptied memory.
In this embodiment, the returned sequence number is divided into two sections, the first section serves the central server and the REST issue mode, multi-thread processing is used, and the second section serves the embedded issue mode; if one of the service modes is not available, fallback to the other publishing mode uses the locally provisioned machine ID to reach the maximum available service.
In this embodiment, the server machine serial number is divided into two segments, the first segment starts to go up at 0, the second segment starts to go down at 128, and a dynamic adjustment boundary can be performed, which satisfies scalability.
The server virtual cluster is authorized at regular time through the regular task to approve the time of the server, the slow-down time is not restarted, the vista throws out the abnormity, the ID is refused to be generated, the machine fast-down time is restarted, the ID is normally generated after adjustment, and no ID is generated in the adjustment time period.
The invention has the advantages that the generation of the integrated distributed ID of the distributed system is more efficient and convenient, the embedded system does not depend on central service when in use, the ID with global uniqueness can be generated, the problem of ID repetition is solved, the production time of the distributed ID and the service port information of the service can be received in real time through rich API interfaces, the storage space is saved, the performance is improved, and the requirement of high service growth speed is met. Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.
Claims (10)
1. A globally unique distributed ID generation method, comprising:
an embedded release mode is configured in advance, and a local IP is obtained when a service instance is started;
the method comprises the steps that an instance list file is generated in a local machine, the service instance is registered, and a machine serial number is returned after the registration; performing bit operation on the IP and the machine serial number to obtain a globally unique machine bit;
defining each item of displacement digit information and the maximum limiting sequence of the digit when the ID is generated through idMeta, carrying out right digit operation on the generated ID, and carrying out AND operation on the right digit and the maximum limiting sequence of the digit to obtain a generation time numerical value of the ID;
and performing left bit operation on the machine bit, the version number defined by the idMeta, the time type, the current time and the current machine serial number to obtain the unique distributed ID.
2. The globally unique distributed ID generation method according to claim 1, further comprising integrating a Spring frame and a Mybatis frame, developing an interceptor using a Mybatis frame interceptor interface, intercepting when data is inserted into the database, and generating a distributed ID for assignment if an ID field of a data entity is empty after intercepting a corresponding data entity; and then integrating the globally unique distributed ID generation method with the interception and springboot framework to generate a rapidly integrated starter dependent package.
3. The globally unique distributed ID generation method of claim 1, wherein said numerical key information at ID generation time includes but is not limited to generation time, traffic type, machine bit and sequence number.
4. The globally unique distributed ID generation method of claim 1, wherein the optimal number of operation bits is found by simulating bit operations, and there is a balanced QPS in both performance scenarios of maximum peak and minimum granularity.
5. The globally unique distributed ID generation method according to claim 1, wherein the method includes packaging in combination with SpingBoot, automatically performing distributed ID generation assignment when table data is inserted, and obtaining the assigned value separately by IdService.
6. The globally unique distributed ID generation method of claim 1, wherein said method is implemented with API set with production ID, inverse solution ID, translation time, translation machine bit.
7. The globally unique distributed ID generation method according to claim 1, wherein when acquiring a local IP when the system service instance is started, the method acquires a local port number and an IP, determines whether the IP and the port number are correct, sends an exception alarm instruction if it is determined that the IP and the port number are abnormal, notifies the exception, and continues to operate the local to generate the instance list file if it is determined that the IP and the port number are correct.
8. The globally unique distributed ID generation method according to claim 1, wherein in the service instance starting process, a generation thread is started, a preset number of IDs are generated in each preset memory, and after the service instance is started, the IDs are taken out from the corresponding preset memory, and the preset number of IDs are generated again in the emptied memory.
9. The globally unique distributed ID generation method of claim 1, wherein said returned sequence number is divided into two sections, the first section serving a central server and REST issue mode, using multi-threaded processing, the second section serving an embedded issue mode; if one of the service modes is not available, fallback to the other publishing mode uses the locally provisioned machine ID.
10. The globally unique distributed ID generation method of claim 1, wherein the server machine serial number is divided into two segments, a first segment starting at 0 up and a second segment starting at 128 down, and a dynamically adjustable demarcation line is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111421682.XA CN114116060A (en) | 2021-11-26 | 2021-11-26 | Globally unique distributed ID generation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111421682.XA CN114116060A (en) | 2021-11-26 | 2021-11-26 | Globally unique distributed ID generation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114116060A true CN114116060A (en) | 2022-03-01 |
Family
ID=80370290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111421682.XA Pending CN114116060A (en) | 2021-11-26 | 2021-11-26 | Globally unique distributed ID generation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114116060A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107864233A (en) * | 2017-11-08 | 2018-03-30 | 千寻位置网络有限公司 | Globally unique ID structures and generation method |
| CN108108252A (en) * | 2017-12-20 | 2018-06-01 | 马上消费金融股份有限公司 | A kind of globally unique ID generation methods, system and storage medium |
| CN109309736A (en) * | 2017-07-28 | 2019-02-05 | 北京京东尚科信息技术有限公司 | The generation method and generating means of globally unique ID |
| CN109327555A (en) * | 2018-08-09 | 2019-02-12 | 北京奇虎科技有限公司 | Distributed hair system, method and device |
| CN111930384A (en) * | 2020-07-02 | 2020-11-13 | 上海微亿智造科技有限公司 | Application method and system based on optimized snowflake algorithm in industrial big data |
| CN113420029A (en) * | 2021-06-25 | 2021-09-21 | 陕西浪潮英信科技有限公司 | Global ID generation method, device, equipment and medium for distributed system |
-
2021
- 2021-11-26 CN CN202111421682.XA patent/CN114116060A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109309736A (en) * | 2017-07-28 | 2019-02-05 | 北京京东尚科信息技术有限公司 | The generation method and generating means of globally unique ID |
| CN107864233A (en) * | 2017-11-08 | 2018-03-30 | 千寻位置网络有限公司 | Globally unique ID structures and generation method |
| CN108108252A (en) * | 2017-12-20 | 2018-06-01 | 马上消费金融股份有限公司 | A kind of globally unique ID generation methods, system and storage medium |
| CN109327555A (en) * | 2018-08-09 | 2019-02-12 | 北京奇虎科技有限公司 | Distributed hair system, method and device |
| CN111930384A (en) * | 2020-07-02 | 2020-11-13 | 上海微亿智造科技有限公司 | Application method and system based on optimized snowflake algorithm in industrial big data |
| CN113420029A (en) * | 2021-06-25 | 2021-09-21 | 陕西浪潮英信科技有限公司 | Global ID generation method, device, equipment and medium for distributed system |
Non-Patent Citations (3)
| Title |
|---|
| JAVA知音: "分布式全局唯一ID生成策略", vol. 1, pages 1 - 48, Retrieved from the Internet <URL:https://blog.51cto.com/u_12302929/3322784> * |
| 陈晨辰呀: "分布式id介绍及应用", Retrieved from the Internet <URL:https://blog.csdn.net/weixin_38004638/article/details/98661112> * |
| 陈秋歌: "如何设计一款多场景分布式发号器（Vesta）", Retrieved from the Internet <URL:https://blog.csdn.net/chenqiuge1984/article/details/80128916> * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112019475B (en) | Resource access method, device, system and storage medium under server-free architecture | |
| US11726984B2 (en) | Data redistribution method and apparatus, and database cluster | |
| US9952783B2 (en) | Data processing method and apparatus, and shared storage device | |
| US10534776B2 (en) | Proximity grids for an in-memory data grid | |
| EP3376403A1 (en) | Method of accessing distributed database and device providing distributed data service | |
| CN110888858B (en) | Database operation method and device, storage medium and electronic device | |
| CN110046062B (en) | Distributed data processing method and system | |
| CN111797121A (en) | Strong consistency query method, device and system for read-write separation architecture service system | |
| CN110730250B (en) | Information processing method and device, service system and storage medium | |
| CN112468601A (en) | Data synchronization method, access method and system of distributed storage system | |
| CN107370832A (en) | A kind of expansible method for refreshing for not restarting dynamic configuration parameters of application server | |
| CN110740155A (en) | Request processing method and device in distributed system | |
| WO2019037664A1 (en) | Address allocation | |
| US11018860B2 (en) | Highly available and reliable secret distribution infrastructure | |
| CN107277188B (en) | Method, client, server and service system for determining IP address attribution information | |
| WO2015192583A1 (en) | Internet protocol (ip) address allocation method and apparatus, server and terminal | |
| US20160050113A1 (en) | Methods for managing storage virtual machine configuration changes in a distributed storage system and devices thereof | |
| CN113420029B (en) | Global ID generation method, device, equipment and medium for distributed system | |
| CN103905201A (en) | Interaction method and device for master application and multiple slave applications | |
| US10749921B2 (en) | Techniques for warming up a node in a distributed data store | |
| WO2021147773A1 (en) | Data processing method and apparatus, electronic device and computer-readable storage medium | |
| CN114116060A (en) | Globally unique distributed ID generation method | |
| CN110113217B (en) | Micro-service management method, device, management platform and storage medium | |
| CN107038174B (en) | Data synchronization method and device for data system | |
| US10791088B1 (en) | Methods for disaggregating subscribers via DHCP address translation and devices thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |