CN118312518A

CN118312518A - Distributed ID generation method and system

Info

Publication number: CN118312518A
Application number: CN202410747708.7A
Authority: CN
Inventors: 朱洁; 黄毅坤; 张锦宇
Original assignee: Xiamen Zhiwen Technology Co ltd
Current assignee: Xiamen Zhiwen Technology Co ltd
Filing date: 2024-06-11
Publication date: 2024-07-09

Abstract

The invention discloses a distributed ID generation method and a system, comprising the following steps: selecting a designated machine node and a fragment value, performing time bit operation by utilizing time corresponding to the machine node and the fragment value, and obtaining a time bit operation binary result; performing machine bit operation by using the machine node, and obtaining a binary result of the machine bit operation; performing slicing bit operation by using the slicing value, and obtaining a binary result of the slicing bit operation; carrying out serialization bit operation by utilizing the time and obtaining a binary result of the serialization bit operation; performing bit operation by using the time bit operation binary result, the machine bit operation binary result, the slice bit operation binary result and the serialization bit operation binary result to obtain an ID value; and carrying out bit operation by using the ID value to obtain a table sequence number.

Description

Distributed ID generation method and system

Technical Field

The application relates to the technical field of software, in particular to a distributed ID generation method and system.

Background

In the prior art, a snowflake algorithm is adopted, so that the generation efficiency is high, but a library table where data are located cannot be located, and additional parameters are required to be added to assist in locating the data. Resulting in increased costs in parameter delivery (code writing, system operation, problem communication) in a split-library and split-table scenario.

The snowflake algorithm generates a unique ID of 8 bytes (64 bits in total) including a 1bit sign bit, a 41bit timestamp, a 10bit machine node, and a 12bit serial number.

41Bit timestamp: support to use for about 70 years;

10bit machine node: support 1024 machines;

sequence number of 12 bits: each machine is supported to create 4096 records per millisecond.

Therefore, there is a need for an algorithm that generates a distributed unique ID that solves the problems in the prior art.

Disclosure of Invention

Aiming at the technical problem that the database table where the data is located cannot be directly located from the ID in the prior art, the application provides a distributed ID generation method and system.

According to one aspect of the present invention, a method and a system for generating a distributed ID are provided, including:

S1, selecting a designated machine node and a fragment value, performing time bit operation by utilizing time corresponding to the machine node and the fragment value, and obtaining a binary result of the time bit operation;

S2, performing machine bit operation by using the machine node, and obtaining a binary result of the machine bit operation;

S3, performing slicing bit operation by using the slicing value, and obtaining a binary result of the slicing bit operation;

S4, carrying out serialization bit operation by utilizing the time in the step S1, and obtaining a binary result of the serialization bit operation;

s5, carrying out bit operation by using the time bit operation binary result, the machine bit operation binary result, the slice bit operation binary result and the serialization bit operation binary result to obtain an ID value;

S6, carrying out bit operation by using the ID value to obtain a table sequence number.

Further, the performing the time bit operation by using the time corresponding to the machine node and the fragment value includes: and taking a difference value between the time and the initial time, and performing bit data left shift by 23 bits, wherein the 23 bits comprise 8-bit machine nodes, 8-bit slice nodes and 7-bit serial number nodes.

Since the final value is a 64-bit binary component, the number of data bits of 3 block nodes is reserved at one time, so that bit operation is the highest-performance calculation mode in a computer.

Further, the performing the bit operation to obtain the ID value specifically includes performing an or operation on the time bit operation binary result, the machine bit operation binary result, the slice bit operation binary result, and the serialized bit operation binary result, where the final result is converted into a decimal, and the decimal is the ID value.

The calculated ID is used as a primary key value of data in a database table, and the monotonous increasing and uniqueness of the number is ensured.

Further, the obtaining the table sequence number by using the ID value for bit operation specifically includes:

And performing bit operation on the ID value, performing AND operation on the ID value and the fragment value to obtain a binary result, converting the binary result into decimal, and taking the remainder by using the decimal and the number of the tables to obtain the serial number of the final table.

The step of calculating the table can obtain the sequence number information of the table, and specific tables such as a user_0, a user_1 and a user_2 can be designated when the database is queried, and the user_1 (assumption) can know that the data exists in the table by calculating the table only without knowing which table the data exists in.

According to a second aspect of the present invention, a computer-readable storage medium is presented, on which one or more computer programs are stored which, when executed by a computer processor, implement the above-described method.

The above technical solutions in the embodiments of the present application have at least one of the following technical effects:

1) The development process does not need to additionally transmit the fragmentation parameters, so that the development efficiency is improved;

2) The system does not need to additionally transmit the fragmentation parameters in the running process, so that the execution efficiency is improved;

3) The number of tables can be transversely expanded, and the robustness of the system is improved.

Drawings

The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Many of the intended advantages of other embodiments and embodiments will be readily appreciated as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

FIG. 1 shows a schematic diagram of a distributed ID generation method according to one embodiment of the invention;

FIG. 2 illustrates a 64-bit binary representation pictorial view of a snowflake algorithm supporting a split-base split-table positioning, in accordance with a specific embodiment of the present invention;

FIG. 3 illustrates an ID generation flow chart diagram intent in accordance with a particular embodiment of the present invention;

FIG. 4 illustrates an ID routing to table flow diagram in accordance with a specific embodiment of the present invention;

Fig. 5 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.

Detailed Description

The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be noted that, for convenience of description, only the portions related to the present application are shown in the drawings.

It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.

Fig. 1 shows a schematic diagram of a distributed ID generation method according to one embodiment of the present invention, as shown in fig. 1,

Step one, ID generation: in this embodiment, 2024-03-27 09:36:30, machine node 2, and fragment 10 are combined with fig. 2, and fig. 3 illustrates the ID generation process:

The sequence number shown in fig. 2 is used to generate different IDs within the same millisecond, the time stamp is used to record the difference in the time stamp (relative to some fixed point in time), the sign bit is also called the first bit, which is always 0 because it is a positive integer, and the machine node is used to identify different work machines (e.g., different server instances).

The performing a time bit operation using the time includes: and taking a difference value between the time and the initial time, and performing bit data left shift by 23 bits, wherein the 23 bits comprise 8-bit machine nodes, 8-bit slice nodes and 7-bit serial number nodes.

In this embodiment, the time bit is calculated: obtaining the millisecond number of the current A city time, taking a difference value from the initial time 2022, 01 month and 01 day, and shifting left by 23 bits, wherein 8-bit machine nodes+8-bit slicing nodes+7-bit serial number nodes, and obtaining a binary result:

000010011111011100111011011010011011011100000000000000000000000。

supporting the use for about 35 years.

In this embodiment, the machine bit operation: according to the current machine 2, obtaining a value of 1, shifting 15 bits left, and obtaining a binary result: 00000001000000000000000, a maximum of 256 machines can be supported.

In this embodiment, the slice bit operation: and performing AND operation according to the input fragment values 10 and 255 (8 th power-1 of 2), and then shifting left by 7 bits to obtain a binary result: 000010100000000, 256 machines can be supported.

in this embodiment, the bit operation is serialized: the number of milliseconds obtained in S1 increases from 0, and the result 0000000 is obtained. 128 sequence numbers per each machine per slice are supported.

In this embodiment, the first 4 results are ored to obtain a binary result, as follows: the result of the conversion into decimal is 359053995408393472, wherein the calculation order starts from the first bit on the right and starts to the left:

1、000010011111011100111011011010011011011100000000000000 000000000

2、00000001000000000000000

3、000010100000000

4、0000000

results: 000010011111011100111011011010011011011100000001000 010100000000

Step two, referring to fig. 2 with ID359053995408393472, fig. 4 illustrates the process of routing from ID to table:

S6, carrying out bit operation by using the ID value to obtain a table sequence number, which specifically comprises the following steps:

In this embodiment, the ID value is shifted to the right by 7 bits (number of bits) to obtain a result 0000000000 0100111110111001110110110100110110111 00000001 00001010;

Obtaining a result 00000000000000000000000000000000 00000000000000000000000011111111 according to 8 bits of the fragmentation information;

Performing bit-wise AND operation on the two results to obtain 00001010, and restoring decimal system to obtain a final fragment value 10;

The bitwise AND operation is as follows:

0000000000010011111011100111011011010011011011100000001 00001010

00000000000000000000000000000000000000000000000000000000 11111111

results:

00001010

the remainder is taken from the fragment value 10 and the number of tables, and the result is the sequence number of the tables.

If the data storage of the system needs to be expanded, only the quantity needs to be modified, and the method specifically comprises the following steps:

There are 8 TABLEs TABLE_0, TABLE_1, TABLE_2, TABLE_3, TABLE_4, TABLE_5, TABLE_6, TABLE_7;

Because the sharded bits support a maximum value of 256 and the number of TABLEs is expanded to the power of 2, as long as the old TABLEs are copied in the way of n+8, the data manager copies and expands the TABLEs 8-15 according to the following mapping relation:

TABLE_0->TABLE_8

TABLE_1->TABLE_9

TABLE_2->TABLE_10

TABLE_3->TABLE_11

TABLE_4->TABLE_12

TABLE_5->TABLE_13

TABLE_6->TABLE_14

TABLE_7->TABLE_15

After the table data is ready, the number of the table in the code is updated, the table is changed from 8 to 16, the newly generated ID slicing bit is 16, the ID routing table is also changed from slicing bit% 8 to slicing bit% 16, and the table copy in the step 2 ensures that the old data can be queried in the 8-15 table, for example:

The slicing bit value is 5, the original 5%8 =5, and the capacity is expanded to 5% 16=5;

the slicing bit value is 11, the original 11%8=3, and the 11%16=11 after capacity expansion;

the slicing bit value is 8, the original 8%8 =0, and the 8% 16=8 after capacity expansion;

the slicing bit value is 16, the original 16%8=0, and the 16%16=0 after capacity expansion.

In summary, the invention provides a distributed ID generation method for realizing the functions of library and table division, which can be applied to a plurality of scenes, such as a distributed database and file system, a cache and message queue, an identity recognition and authentication system and the like;

In distributed databases and file systems, the distributed IDs are used to identify unique data entities, helping to quickly retrieve and manage data;

In the message cache queue, a distributed ID is used to uniquely identify the cache item and the message for transferring and sharing data between different nodes;

in identification and authentication systems, a distributed ID is used to uniquely identify a user or device for authentication and authorization.

The distributed ID generation supporting the database and table splitting mode can ensure the uniqueness and the traceability of the data entity and promote the reliability and the stability of a distributed system.

Referring now to FIG. 5, there is shown a schematic diagram of a computer system suitable for use in implementing an electronic device of an embodiment of the present application. The electronic device shown in fig. 5 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present application.

As shown in fig. 5, the computer system includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the system operation are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Liquid Crystal Display (LCD) or the like, a speaker or the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 501. The computer readable storage medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules involved in the embodiments of the present application may be implemented in software or in hardware.

As another aspect, the present application also provides a computer-readable storage medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable storage medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquiring a machine and a fragment value under a specified time, performing time bit operation by using the time, and acquiring a binary result; performing machine bit operation by using the machine, and obtaining a binary result; performing slicing bit operation by using the slicing value, and obtaining a binary result; carrying out serialization bit operation by using the appointed time, and obtaining a binary result; obtaining an ID value by using the binary results obtained in the four steps; and carrying out bit operation by using the ID value to obtain a table sequence number.

The above description is only illustrative of the preferred embodiments of the present application and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the application referred to in the present application is not limited to the specific combinations of the technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the inventive concept described above. Such as the above-mentioned features and the technical features disclosed in the present application (but not limited to) having similar functions are replaced with each other.

Claims

1. A distributed ID generation method, comprising the steps of:

2. The method according to claim 1, characterized in that: the time bit operation by utilizing the time corresponding to the machine node and the fragment value comprises the following steps: and taking a difference value between the time and the initial time, and performing bit data left shift by 23 bits, wherein the 23 bits comprise 8-bit machine nodes, 8-bit slice nodes and 7-bit serial number nodes.

3. The method according to claim 1, characterized in that: the bit operation is performed to obtain an ID value specifically including performing an or operation on the time bit operation binary result, the machine bit operation binary result, the slice bit operation binary result, and the serialization bit operation binary result, and converting the final result into a decimal, where the decimal is the ID value.

4. The method according to claim 1, characterized in that: the step of obtaining the table sequence number by using the ID value for bit operation specifically comprises the following steps:

And performing bit operation on the ID value, performing AND operation on the ID value and the fragment value to obtain a binary result, converting the binary result into decimal, and taking the remainder by utilizing the decimal and the number of the tables to obtain the serial number of the final table.

5. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1-4.

6. A computing system comprising a processor and a memory, the processor configured to perform the method of any of claims 1-4.