CN112199923A

CN112199923A - Identification generation method, system, device and medium based on distributed system

Info

Publication number: CN112199923A
Application number: CN202011122844.5A
Authority: CN
Inventors: 唐明东
Original assignee: Chongqing Unisinsight Technology Co Ltd
Current assignee: Chongqing Unisinsight Technology Co Ltd
Priority date: 2020-10-20
Filing date: 2020-10-20
Publication date: 2021-01-08

Abstract

The application provides an identification generation method, a system, equipment and a medium based on a distributed system, wherein the method comprises the following steps: acquiring one or more identification generation requests, wherein the identification generation requests comprise service KEY values; detecting whether the number of the marks left in the current cache meets the requirement of a mark generation request; when the number of the remaining identifiers in the cache meets the requirement of the identifier generation request, returning the identifiers to a requester; when the number of the remaining identifiers in the cache does not meet the requirement of the identifier generation request, calling a Redis INCR instruction to generate the identifiers to return to the cache so as to meet the identifier generation request; and utilizing the Redis INCR to acquire the maximum value of the sequence, and calculating the starting value in reverse to sequentially increase or decrease to generate the identifiers, wherein the generation number of the identifiers is determined according to the number of the remaining identifiers in the cache and the requirement of the identifier generation request. The method and the device improve the efficiency of the mark generation; the sequence ID generated in sequence avoids the ID repetition phenomenon, and is beneficial to subsequent capacity expansion.

Description

Identification generation method, system, device and medium based on distributed system

Technical Field

The present application relates to the field of computers, and in particular, to a method, system, device, and medium for generating an identifier based on a distributed system.

Background

In some scenarios of internet and software development, all data objects require a unique Identification (ID), and the service used to generate the ID is called an ID generator (or issuer). The inclusion of the ID generator issues a globally unique ID for a data object or other service. Data objects are assigned an ID before they can be stored and used.

Existing ID generators guarantee the uniqueness of data based on a self-increment sequence of a database or UUID (universally unique identification code). However, when the data reaches a certain magnitude, the system is upgraded to a distributed cluster architecture, on one hand, if a self-increment sequence mode is continuously adopted, the capacity expansion of the database nodes is limited, and the uniqueness of the ID is influenced; on the other hand, if the UUID mode is adopted, the generated IDs are unordered character strings, so that the query and storage efficiency is poor, the readability is poor, the trend is not ensured to be increased progressively, and the service sequence cannot be reflected; if the snowfall algorithm with the Twitter open source is adopted, because the snowfall algorithm strongly depends on a machine clock, when the clock dial-back condition occurs to the working equipment, the timestamp for generating the ID can be backed, and if the ID is generated according to the backed-back timestamp, the ID repetition problem can occur; meanwhile, the ID difference generated by multiple nodes at the same time is large, so that normal sequencing cannot be performed, and great waste of ID resources is easily caused.

Disclosure of Invention

In view of the above-mentioned shortcomings of the prior art, the present application aims to provide a method, a system, a device and a medium for generating an identifier based on a distributed system, so as to solve the problems of ID duplication and difficulty in extension easily caused by identifier generation in the prior art.

To achieve the above and other related objects, the present application provides an identifier generating method based on a distributed system, including:

acquiring one or more identification generation requests, wherein the identification generation requests comprise service KEY values;

detecting whether the number of the marks left in the current cache meets the requirement of a mark generation request;

when the number of the remaining identifiers in the cache meets the requirement of the identifier generation request, returning the identifiers to a requester;

when the number of the remaining identifiers in the cache does not meet the requirement of the identifier generation request, calling a Redis INCR instruction to generate the identifiers to return to the cache so as to meet the identifier generation request; and obtaining the maximum value of the sequence by utilizing the Redis INCR, and sequentially increasing or decreasing the starting value of the reverse calculation to generate the identifiers, wherein the generation number of the identifiers is determined according to the number of the remaining identifiers in the cache and the requirement of the identifier generation request.

Another object of the present application is to provide an identification generation system based on a distributed system, including:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring one or more identification generation requests, and the identification generation requests comprise service KEY values;

the detection module is used for detecting whether the number of the remaining identifiers in the current cache meets the requirement of the identifier generation request;

the first identification generation module is used for returning the identification to the requester when the number of the remaining identifications in the cache meets the requirement of the identification generation request;

the second identifier generation module is used for calling a Redis INCR instruction to generate an identifier and returning the generated identifier to the cache to meet the identifier generation request when the number of the remaining identifiers in the cache does not meet the requirement of the identifier generation request; and obtaining the maximum value of the sequence by utilizing the Redis INCR, and sequentially increasing or decreasing the starting value of the reverse calculation to generate the identifiers, wherein the generation number of the identifiers is determined according to the number of the remaining identifiers in the cache and the requirement of the identifier generation request.

Another object of the present application is to provide an electronic device, comprising:

one or more processing devices;

a memory for storing one or more programs; when the one or more programs are executed by the one or more processing devices, cause the one or more processing devices to perform the distributed system based identity generation method.

It is a further object of the present application to provide a computer-readable storage medium having stored thereon a computer program for causing the computer to execute the distributed system based identity generation method.

As described above, the identifier generation method, system, device and medium based on the distributed system according to the present application have the following advantages:

according to the method and the device, the cache is used for responding the identifier generation request in advance, and different identifier generation response modes are adopted according to the requirement of the identifier generation request, so that on one hand, the identifier generation efficiency is improved; on the other hand, the sequence IDs generated in sequence avoid the ID repetition phenomenon, and are beneficial to subsequent capacity expansion; in addition, the pure digital sequence ID meeting the distributed architecture system can be generated, the capability of rapidly generating the IDs in batches in a high-concurrency scene is realized, and the time sequence of the IDs can be still ensured in a cross-node same-service multi-copy scene.

Drawings

Fig. 1 is a flowchart illustrating an embodiment of an identifier generation method based on a distributed system according to the present application;

FIG. 2 is a flow chart illustrating a method for generating an identifier based on a distributed system according to another embodiment of the present disclosure;

FIG. 3 is a schematic block diagram of a distributed system-based identity generation system provided herein;

FIG. 4 is a block diagram illustrating a distributed system-based identifier generation system according to the present application;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation, and the type, number and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

Referring to fig. 1, a flowchart of an identifier generation method based on a distributed system according to the present application includes:

step S1, acquiring one or more identification generation requests, wherein the identification generation requests include service KEY values;

the ID serial numbers with different formats are generated according to different types of the service KEY values, for example, the KEY values can be determined according to form names, wherein pure digital ID serial numbers or ID serial numbers with service date information can be generated according to different service KEY values; meanwhile, an acquisition interface of a single ID serial number and a batch ID serial number can be formed through the identification generation request.

For example, when the ID serial number is a pure number, the range of the available value is 1 ~ 2⁶³-1, the sequence number is of non-fixed length, facilitating subsequent capacity expansion; when the ID serial number with service date information is used, the original KEY + date (yyyyMMdd) + fixed length of the fixed-length number sequence can be used.

Step S21, detecting whether the number of the remaining identifiers in the current cache meets the requirement of the identifier generation request;

step S22, when the number of the remaining identifiers in the cache meets the requirement of the identifier generation request, returning the identifiers to the requester;

step S23, when the number of the remaining identifiers in the cache does not meet the requirement of the identifier generation request, calling a Redis INCR instruction to generate the identifiers to return to the cache to meet the identifier generation request; and obtaining the maximum value of the sequence by utilizing the Redis INCR, and sequentially increasing or decreasing the starting value of the reverse calculation to generate the identifiers, wherein the generation number of the identifiers is determined according to the number of the remaining identifiers in the cache and the requirement of the identifier generation request.

The service system acquires the identifiers from the storage system, so that the situation that the time for reversely solving the ID acquired by the previous request is later than the time for reversely solving the ID acquired by the next request due to the fact that the time on the physical machine is inconsistent is avoided, and the ID is prevented from being not completely ordered.

Specifically, calculating a difference value between the required number of the identifier generation requests and the number of the identifiers remaining in the cache; moving the rest marks in the cache to a return result, and emptying the cache; the generation quantity of the identifiers is the sum of the maximum caching quantity of the cache and the difference value between the required quantity of the identifier generation request and the quantity of the remaining identifiers in the cache; under the condition that the caller does not use the KEY to obtain the ID, for example, when the condition that no service KEY value exists in the identification generation request is detected, the corresponding ID serial number is extracted from the cache by adopting a default service KEY value. Meanwhile, an ID cache pool is dynamically established according to the KEY, and the response speed is improved by locally caching a preset number of IDs.

For example, ID (identification) generation is realized by using an INCR instruction of Redis, and the idea is simple and practical. Firstly, a Redis INCR instruction belongs to atom increasing operation, and no matter how many client nodes, the ID result requested at the same time can be guaranteed to be a real sequential digital ID; secondly, the generated ID serial number can start from 1, so that the best use of the things can be achieved, and the ID resource is not wasted; moreover, different service tables can set corresponding KEY to generate relatively independent ID, and also can select to use default KEY to share the same group of ID resources, so that the flexibility and controllability are realized, and the subsequent capacity expansion of the identification is facilitated.

In this embodiment, the cache is used for responding to the identifier generation request in advance, and different identifier generation response modes are adopted according to the requirement of the identifier generation request, so that on one hand, the identifier generation efficiency is improved; on the other hand, the sequence IDs generated in sequence avoid the ID repetition phenomenon, and are beneficial to subsequent capacity expansion; in addition, the pure digital sequence ID meeting the distributed architecture system can be generated, the capability of rapidly generating the IDs in batches in a high-concurrency scene is realized, and the time sequence of the IDs can be still ensured in a cross-node same-service multi-copy scene.

Referring to fig. 2, a flowchart of another embodiment of a method for generating an identifier based on a distributed system according to the present application includes:

step S31, generating ID duplication prevention identification by using the main database and the standby database; receiving the identifier generation request, and detecting whether the main database is normal;

when the active database is normal in step S32, setting the active database identifier to a first value;

step S33, when the primary database is abnormal, switching to a standby database for use and setting the identifier of the standby database to be a second numerical value;

step S34, when it is detected that clock callback does not occur, setting the clock callback prevention flag bit to a third value;

step S35, generating an identifier based on the first value, the KEY value, the third value and the timestamp by using the primary database;

step S36, generating an identifier based on the second value, the KEY value, the third value and the timestamp by using the backup database.

In this embodiment, when the primary database goes offline or fails, the primary database is switched to the standby database, and the database state identifier is used for switching, so that the problem of ID duplication caused by the primary and standby consistency problem is prevented, and the uniqueness of the identifier (ID sequence number) is ensured.

The method effectively avoids the problem of repeatability of the identification, and the scheme is a perfect distributed ID generation scheme under the high-availability Redis cluster environment, so that the uniqueness of the identification is ensured.

In another embodiment, the identifiers sequentially generate identifiers according to the initial relevance of the service KEY values and the KEY values of the same type or similar types, and the associable identifiers are generated according to the relevance between the KEY values, so that the subsequent identifier association is facilitated, and the management control of the identifiers is facilitated.

Referring to fig. 3, a schematic framework diagram of an identification generation system based on a distributed system according to the present application includes:

1) a single ID and a batch ID acquisition interface are provided through an application program, so that a user can conveniently acquire an identification generation request;

2) connecting Redis, in preparation for operating Redis

3) When the ID acquisition interface is called, whether the entry parameter has a KEY is judged, and when the entry parameter does not have the KEY, the default KEY is used

4) Extracting ID cache pool from local cache through KEY

5) Judging whether the number of IDs in the cache pool meets the requirements of a calling party:

6) and when the cache number is enough, moving the corresponding number of IDs from the cache pool to a return result.

And when the number of the caches is not enough, all the residual cache IDs are moved to the returned result, and the cache pool is emptied. And calculating the number of IDs to be newly generated (the number of the IDs is equal to the demand number, the residual cache number and the maximum cache number), calling a Redis INCR instruction, and acquiring the maximum value of the number sequence. And calculating the initial value reversely, circularly increasing by 1, and adding to the return result until the required quantity is met. In the same manner, the remaining range values are filled into the cache pool.

The sequence with service date is similar to the above steps in logic processing, and the difference is that the KEY value is processed: original KEY + date (yyyyMMdd) + fixed-length number sequence; the identified digit sequence is of fixed length, such as 16 bits, and when the ID serial number does not meet the preset fixed length, zero complement length is adopted in front of the ID serial number; for example, the digital sequence part has the limitation of fixed length, the part which is not fixed length is filled with 0 to complement before the digit, and the abnormality is thrown out when the digit is over long; the result is returned as a character string in "date (yyyyMMdd) + fixed-length number sequence" format.

In another embodiment, a maven item is created; introducing spring redis dependence in a pom.xml file; configuring correct redis connection information;

the ID generator provides an interface method for generating a number sequence and a date sequence, provides an ID cache pool, and realizes ID generation logic.

When the application program generates a new ID sequence required by the service data, the ID generation method is called, the service KEY value is transmitted, and the corresponding ID serial number is obtained.

The ID generator preferentially extracts unused ID resources from the cache pool, and if the number of the resources does not meet the batch requirement of a calling party, an INCR request is sent to Redis, so that a new resource range is obtained. And then calculating and intercepting part of sufficient ID resources to be combined with the old resources, and returning the combined ID resources to the caller. And the rest resources are put into the cache pool for subsequent use.

Please refer to fig. 4, which is a block diagram of an identifier generation system based on a distributed system according to the present application; the method comprises the following steps:

the system comprises an acquisition module 1, a processing module and a processing module, wherein the acquisition module is used for acquiring one or more identifier generation requests, and the identifier generation requests comprise service KEY values;

the detection module 2 is used for detecting whether the number of the remaining identifiers in the current cache meets the requirement of the identifier generation request;

the first identifier generating module 3 is configured to, when the number of remaining identifiers in the cache meets the requirement of the identifier generating request, return the identifier to the requester;

the second identifier generating module 4 is configured to, when the number of remaining identifiers in the cache does not meet the requirement of the identifier generating request, invoke a Redis INCR instruction to generate an identifier and return the identifier to the cache to meet the identifier generating request; and obtaining the maximum value of the sequence by utilizing the Redis INCR, and sequentially increasing or decreasing the starting value of the reverse calculation to generate the identifiers, wherein the generation number of the identifiers is determined according to the number of the remaining identifiers in the cache and the requirement of the identifier generation request.

It should be further noted that the identifier generation system based on the distributed system and the identifier generation method based on the distributed system are in a one-to-one correspondence relationship, and here, technical details and technical effects related to each module and the above process steps are the same, and are not described herein one by one, please refer to the identifier generation method based on the distributed system.

Example 5

Referring now to fig. 5, a schematic diagram of an electronic device (e.g., a terminal device or server 700) suitable for implementing embodiments of the present disclosure is shown, where the terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc. the electronic device shown in fig. 5 is only one example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.

As shown in fig. 5, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 707 is also connected to bus 704.

Generally, the following devices may be connected to the I/O interface 707: input devices 707 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 5 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 708, or may be installed from the ROM 702. When executed by the processing device 701, performs the above-described functions defined in the methods of the embodiments of the present disclosure

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a data set, wherein the data set is a plurality of human images acquired by the same person under different scenes; distributing the portrait in the data set to corresponding bayonet equipment according to the number of the bayonet equipment, the distributed geographic position and a preset time period to form images which are arranged in a time sequence and represent the portrait track, and form images which are arranged in the time sequence and represent the portrait track; and clustering and archiving the image by using the platform to be tested, and calculating indexes of clustering and archiving.

Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In summary, according to the method and the device, the cache is used for responding to the identifier generation request in advance, and different identifier generation response modes are adopted according to the requirement of the identifier generation request, so that on one hand, the identifier generation efficiency is improved; on the other hand, the sequence IDs generated in sequence avoid the ID repetition phenomenon, and are beneficial to subsequent capacity expansion; in addition, the pure digital sequence ID meeting the distributed architecture system can be generated, the capability of rapidly generating the IDs in batches in a high-concurrency scene is realized, and the time sequence of the IDs can be still ensured in a cross-node same-service multi-copy scene.

The above embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.

Claims

1. An identification generation method based on a distributed system, which is characterized by comprising the following steps:

2. The distributed system based identity generation method of claim 1, further comprising: and generating ID serial numbers with different formats according to different types of the service KEY values, wherein the ID serial numbers with pure digital ID serial numbers or service date information can be generated according to different service KEY values.

3. The method according to claim 1 or 2, wherein the step of determining the number of generated tags according to the number of tags remaining in the cache and the requirement of the tag generation request comprises:

calculating the difference between the required number of the identifier generation requests and the number of the identifiers remained in the cache;

moving the rest marks in the cache to a return result, and emptying the cache;

the generation quantity of the identifiers is the sum of the maximum buffer quantity of the cache and the difference value between the required quantity of the identifier generation requests and the quantity of the identifiers remaining in the cache.

4. The distributed system-based identity generation method according to claim 1 or 2, further comprising: and when detecting that no service KEY value exists in the identifier generation request, adopting a default service KEY value to extract a corresponding ID serial number from a cache.

5. The distributed system-based identity generation method of claim 1, wherein the identity is of a non-fixed length and is preceded by a zero-padding length when the ID sequence number does not satisfy a preset fixed length.

6. The distributed system based identity generation method of claim 1, further comprising:

generating ID repetition prevention identification by using the main database and the standby database; receiving the identifier generation request, and detecting whether the main database is normal;

when the main database is normal, setting the main database identifier as a first numerical value;

when the main database is abnormal, switching to a standby database for use and setting the identifier of the standby database to be a second numerical value;

when the clock callback is detected not to occur, setting the clock callback prevention flag bit as a third numerical value;

generating an identifier by using the main database based on the first numerical value, the KEY value, the third numerical value and the timestamp;

generating, with the backup database, an identification based on the second value, the KEY value, the third value, and the timestamp.

7. The distributed system based identity generation method of claim 1, further comprising: and the identifiers generate identifiers in sequence according to the initial relevance of the service KEY values and the same type of KEY values or similar types of KEY values.

8. An identity generation system based on a distributed system, the system comprising:

9. An electronic device, characterized in that: the method comprises the following steps:

one or more processing devices;

a memory for storing one or more programs; when executed by the one or more processing devices, cause the one or more processing devices to implement the distributed system based identity generation method as described in any of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program for causing a computer to execute the method for generating an identity based on a distributed system according to any one of claims 1 to 7.