CN113553331A - Database-based multi-service inter-point log ID generation method - Google Patents

Database-based multi-service inter-point log ID generation method Download PDF

Info

Publication number
CN113553331A
CN113553331A CN202110859726.0A CN202110859726A CN113553331A CN 113553331 A CN113553331 A CN 113553331A CN 202110859726 A CN202110859726 A CN 202110859726A CN 113553331 A CN113553331 A CN 113553331A
Authority
CN
China
Prior art keywords
database
service
value
number segment
new
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110859726.0A
Other languages
Chinese (zh)
Inventor
陈碧勇
方敏
高永强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen Nanxun Co ltd
Original Assignee
Xiamen Nanxun Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen Nanxun Co ltd filed Critical Xiamen Nanxun Co ltd
Priority to CN202110859726.0A priority Critical patent/CN113553331A/en
Publication of CN113553331A publication Critical patent/CN113553331A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2264—Multidimensional index structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2474—Sequence data queries, e.g. querying versioned data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a database-based method for generating an ID (identity) of a multi-service integration log, which comprises the following steps: s1, generating configuration of the unified integral log ID in the ID database, wherein the generating configuration comprises a service identifier, a maximum ID value, an ID number segment length and a version number; s2, distributing corresponding service identification and ID number segment for each service in each service system according to the generation configuration; s3, the service system reads the current available ID value from the distributed ID number segment, if the current distributed ID number segment is consumed to the set threshold value, the next ID number segment is automatically loaded from the ID database to the memory of the service system. The invention can effectively reduce the system service blockage caused by reading and writing the database in the ID generation process every time, and improve the system response performance.

Description

Database-based multi-service inter-point log ID generation method
Technical Field
The invention relates to the technical field of computers, in particular to a database-based method for generating an ID (identity) of an inter-multi-service integral log.
Background
The method is characterized in that the unique ID identification generated for the integral log data of a plurality of data tables of a plurality of databases is a basic function of an integral system, and the unique ID is generated by utilizing the databases on the premise of reading and writing separation of the databases and architecture design of sub-tables of the databases.
The existing common scheme for generating the ID of the integral log is as follows: and a data table is commonly maintained in each database for ID generation, and the ID generation of each integral log database is set to be different step lengths, so that the situation that the integral log IDs are repeated in different systems is avoided. However, as the data volume increases, the number of databases inevitably increases, and the capacity expansion of the database in this scheme is difficult, after the database and the step size have been defined, if a new database needs to be added, the step size of the database needs to be readjusted, and a database read-write operation needs to be performed each time a new ID is acquired, and when the request volume for generating the integral log ID increases, the database read-write pressure is large, which easily causes system blocking.
Disclosure of Invention
In order to solve the problems, the invention provides a database-based method for generating the ID of the multi-service inter-point log.
The invention adopts the following technical scheme:
a database-based method for generating a multi-service score log ID, the method comprising:
s1, generating configuration of the unified integral log ID in the ID database, wherein the generating configuration comprises a service identifier, a maximum ID value, an ID number segment length and a version number;
s2, distributing corresponding service identification and ID number segment for each service in each service system according to the generation configuration;
s3, the service system reads the current available ID value from the distributed ID number segment, if the current distributed ID number segment is consumed to the set threshold value, the next ID number segment is automatically loaded from the ID database to the memory of the service system.
Further, the service identifier is used for service differentiation; the maximum ID value represents the maximum value of the batch ID to which the current service identification is allocated; the length of the ID number segment represents the number of IDs allocated at each time; the version number is used for confirming whether the current ID number segment is occupied or not.
Further, the value of the version number is a monotonically increasing positive integer, and the version number is updated each time an ID number segment is obtained from the ID database.
Further, the threshold is 20% -30%.
Further, the generating configuration further comprises a delta and a remainder, the delta being used to represent a step size between adjacent ID values; the remainder is used for representing the initial remainder of the ID value, and the ID value is calculated by the following method: IDn+1＝IDn+ increment, wherein, IDn+1Denotes the (n + 1) th ID value, IDnIndicating the nth ID value.
Further, the step S3 of automatically loading the next ID number segment from the ID database into the memory of the service system specifically includes the following steps:
s31, inquiring the current maximum ID value MaxId and the ID number segment length Step;
s32, calculating a new maximum ID value new _ maxld, where new _ maxld is maxld + Step;
s33, acquiring a new ID number segment and updating the maximum ID value in the ID database: comparing the current version number with the version number in the ID database, if the current version number is equal to the version number in the ID database, indicating that the new ID number segment is not occupied, acquiring the new ID number segment and updating the maximum ID value; if not, it indicates that the new ID number segment is occupied by other threads, fails to acquire the new ID number segment, and returns to step S31 for retry.
Further, the number of retries is set to 3-5 times.
After adopting the technical scheme, compared with the background technology, the invention has the following advantages:
1. the invention unifies the generation configuration of the integral log IDs generated by different databases, namely unifies the data sources generated by the IDs, can acquire new integral log IDs from the same data source, and ensures that the IDs are not repeated, thereby avoiding the problem that a plurality of data sources are required to be maintained to ensure that the IDs are not repeated when the quantity of the existing databases is expanded, and solving the problems of difficult expansion and difficult maintenance of the databases;
2. by acquiring a whole batch of IDs every time and caching the IDs into the memory of each service system, the number of database reading and writing times is obviously reduced, the pressure caused by frequent database reading and writing is reduced, the ID generation performance is improved, and the system throughput is improved;
3. by setting a reasonable threshold value and automatically loading the next batch of IDs, system service blockage caused by reading and writing a database in the process of generating the IDs each time is reduced, and the response performance of the system is improved.
Drawings
FIG. 1 is a block diagram of a generating configuration of the present invention;
FIG. 2 is an architecture diagram of the present invention for multi-service reading of different ID number segments;
FIG. 3 is a diagram illustrating a process of automatically loading a new ID number segment according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Examples
A database-based method for generating a multi-service score log ID, the method comprising:
s1, generating and configuring the unified integral log ID in the ID database;
as shown in fig. 1, the generation configuration includes a service identifier, a maximum ID value, an ID number segment length, and a version number;
the service identification (BizType) is used for service differentiation;
the maximum ID value (MaxId) represents the maximum value of the bulk ID to which the current service identity is assigned;
the length (Step) of the ID number segment represents the number of IDs allocated each time, the database is read and written in the prior art in a common method for acquiring the IDs each time, after the length of the ID number segment is set, the database is read and written once only when the whole ID segment is consumed soon, and the database reading and writing frequency can be effectively reduced to the original 1/ID segment length;
the Version number (Version) is used for confirming whether the current ID number segment is occupied, the value of the Version number is a monotonically increasing positive integer, and the Version number is updated once after one ID number segment is obtained from the ID database each time; the version number is an optimistic lock, and the version number is added every time of updating, so that the correctness of concurrent work is ensured, and no repeated ID number segment appears;
the generating configuration further includes an increment (Delta) for representing a step size between adjacent ID values and a Remainder (Remainder); the remainder is used for representing the initial remainder of the ID value, and the ID value is calculated by the following method: IDn+1＝IDn+ increment, wherein, IDn+1Denotes the (n + 1) th ID value, IDnIndicating the nth ID value.
S2, distributing corresponding service identification and ID number segment for each service in each service system according to the generation configuration; as shown in fig. 2, different services are distinguished by pointLog, pointcustomer, etc., and are ID number segments different from each other.
S3, the service system reads the current available ID value from the distributed ID number segment, if the current distributed ID number segment is consumed to the set threshold value, the next ID number segment is automatically loaded from the ID database to the memory of the service system. The threshold is set to 20%.
The step S3 of automatically loading the next ID number segment from the ID database into the memory of the service system specifically includes the following steps:
s31, inquiring the current maximum ID value MaxId and the ID number segment length Step;
s32, calculating a new maximum ID value new _ maxld, where new _ maxld is maxld + Step;
s33, acquiring a new ID number segment and updating the maximum ID value in the ID database: comparing the current version number with the version number in the ID database, if the current version number is equal to the version number in the ID database, indicating that the new ID number segment is not occupied, acquiring the new ID number segment and updating the maximum ID value; if not, it indicates that the new ID number segment is occupied by other threads, fails to acquire the new ID number segment, and returns to step S31 for retry. The number of retries is set to 3.
As shown in fig. 2, when a certain segment with the service identifier pointLog is consumed and other segments with the same service identifier are not updated, the segment (3000,4000) is unloaded, and the maxld of the data of the pointLog is updated to 4000.
If the existing scheme is adopted, the thread can be blocked during the period of taking a new number segment until the number segment is successfully updated. The scheme of the embodiment is as follows: as shown in fig. 3, when the number segment is consumed to 20% (i.e. to 200) of the preset threshold, an asynchronous thread loads the next number segment into the memory in advance, and there is no need to wait for the number segment to be used up and then update the number segment, thereby reducing the response slowness caused by the blocking. DB in the figure represents an ID database.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (7)

1. The method for generating the ID of the multi-service integration log based on the database is characterized in that: the method comprises the following steps:
s1, generating configuration of the unified integral log ID in the ID database, wherein the generating configuration comprises a service identifier, a maximum ID value, an ID number segment length and a version number;
s2, distributing corresponding service identification and ID number segment for each service in each service system according to the generation configuration;
s3, the service system reads the current available ID value from the distributed ID number segment, if the current distributed ID number segment is consumed to the set threshold value, the next ID number segment is automatically loaded from the ID database to the memory of the service system.
2. The database-based multi-service credit log ID generation method of claim 1, wherein: the service identification is used for service differentiation; the maximum ID value represents the maximum value of the batch ID to which the current service identification is allocated; the length of the ID number segment represents the number of IDs allocated at each time; the version number is used for confirming whether the current ID number segment is occupied or not.
3. The database-based multi-service credit log ID generation method of claim 2, wherein: the version number is a monotonically increasing positive integer, and the version number is updated each time an ID number segment is obtained from the ID database.
4. The database-based multi-service credit log ID generation method of claim 1, wherein: the threshold value is 20% -30%.
5. The database-based multi-service credit log ID generation method of claim 1, wherein: the generating configuration further includes a delta and a remainder, the delta representing a step size between adjacent ID values; the remainder is used for representing the initial remainder of the ID value, and the ID value is calculated by the following method: IDn+1＝IDn+ increment, wherein, IDn+1Denotes the (n + 1) th ID value, IDnIndicating the nth ID value.
6. The database-based multi-service credit log ID generation method of claim 3, wherein: the step S3 of automatically loading the next ID number segment from the ID database into the memory of the service system specifically includes the following steps:
s31, inquiring the current maximum ID value MaxId and the ID number segment length Step;
s32, calculating a new maximum ID value new _ maxld, where new _ maxld is maxld + Step;
s33, acquiring a new ID number segment and updating the maximum ID value in the ID database: comparing the current version number with the version number in the ID database, if the current version number is equal to the version number in the ID database, indicating that the new ID number segment is not occupied, acquiring the new ID number segment and updating the maximum ID value; if not, it indicates that the new ID number segment is occupied by other threads, fails to acquire the new ID number segment, and returns to step S31 for retry.
7. The database-based multi-service credit log ID generation method of claim 6, wherein: the number of retries is set to 3-5.
CN202110859726.0A 2021-07-28 2021-07-28 Database-based multi-service inter-point log ID generation method Pending CN113553331A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110859726.0A CN113553331A (en) 2021-07-28 2021-07-28 Database-based multi-service inter-point log ID generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110859726.0A CN113553331A (en) 2021-07-28 2021-07-28 Database-based multi-service inter-point log ID generation method

Publications (1)

Publication Number Publication Date
CN113553331A true CN113553331A (en) 2021-10-26

Family

ID=78104796

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110859726.0A Pending CN113553331A (en) 2021-07-28 2021-07-28 Database-based multi-service inter-point log ID generation method

Country Status (1)

Country Link
CN (1) CN113553331A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114283025A (en) * 2021-11-25 2022-04-05 国电南瑞科技股份有限公司 Equipment ID generation method and device suitable for power Internet of things and equipment access method
CN115687367A (en) * 2023-01-04 2023-02-03 成都精灵云科技有限公司 ID generation method and system based on ETCD key value version number

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114283025A (en) * 2021-11-25 2022-04-05 国电南瑞科技股份有限公司 Equipment ID generation method and device suitable for power Internet of things and equipment access method
CN115687367A (en) * 2023-01-04 2023-02-03 成都精灵云科技有限公司 ID generation method and system based on ETCD key value version number

Similar Documents

Publication Publication Date Title
CN113553331A (en) Database-based multi-service inter-point log ID generation method
US20100030994A1 (en) Methods, systems, and computer readable media for memory allocation and deallocation
WO2017050064A1 (en) Memory management method and device for shared memory database
CN112612799B (en) Data synchronization method and terminal
CN110727724B (en) Data extraction method and device, computer equipment and storage medium
CN111124474A (en) API version control method and device
CN112597153A (en) Data storage method and device based on block chain and storage medium
CN112181306A (en) Method for generating distributed ID (identity) in number segment mode
CN107704573A (en) A kind of intelligent buffer method coupled with business
US6691121B1 (en) Method and apparatus for online and dynamic extension of IMS data entry databases
CN110688357B (en) Method and device for reading log type data
CN112612773A (en) Database synchronization test method and device, computer equipment and storage medium
CN111143326A (en) Method and device for reducing database operation, computer equipment and storage medium
CN111797062B (en) Data processing method, device and distributed database system
CN112711627B (en) Data importing method, device and equipment of Greemplum database
CN110888929B (en) Data processing method, data processing device, data node and storage medium
CN113867649B (en) System and method for adaptive blockchain data storage plugin
CN114237507B (en) Method and device for managing data
US20050044090A1 (en) Computer system and program
CN116820548A (en) Method and device compatible with multiple hardware
CN113051274A (en) Mass tag storage system and method
CN115292416A (en) Data table synchronization method, device, equipment and storage medium
KR20000018875A (en) Backup method based on data grade in hlr system
CN115774727A (en) Data processing method, device, equipment and computer storage medium
CN115858244A (en) Database backup and recovery method, device, terminal and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination