CN112395313A

CN112395313A - Global sequence generation method and device, electronic equipment and computer storage medium

Info

Publication number: CN112395313A
Application number: CN201910760088.XA
Authority: CN
Inventors: 陈诗然
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-08-16
Filing date: 2019-08-16
Publication date: 2021-02-23

Abstract

The embodiment of the invention discloses a global sequence generation method, a device, electronic equipment and a computer storage medium, wherein the method comprises the following steps: acquiring global sequence source data; determining the number of data operation units, splitting the global sequence source data into global sequence source data segments according to the number of the data operation units, and establishing a mapping relation between the data operation units and the global sequence source data segments; and generating a global sequence corresponding to the data operation unit according to the mapping relation between the data operation unit and the global sequence source data segment. The technical scheme can generate a global sequence which is decentralized and supports a multi-unit environment, so that the requirement of global uniqueness can be met without synchronization among multiple units, network delay among available areas and possible equipment single-point faults can be avoided, the problem of network delay caused by crossing units or available areas is solved, and the global target of global uniqueness, high performance and high availability is further achieved.

Description

Global sequence generation method and device, electronic equipment and computer storage medium

Technical Field

The embodiment of the invention relates to the technical field of data processing, in particular to a global sequence generation method and device, electronic equipment and a computer storage medium.

Background

With the development of data technology, the application of data is more and more extensive. In order to provide richer and more extensive data services, there are many data systems in which multiple databases work in concert. In some application scenarios, multiple databases in a data system may be deployed on the same or different database instances, and the multiple database instances may also be in the same or different available areas, but in many cases, global uniqueness of a certain identification value needs to be ensured among the multiple databases, that is, a global sequence of a multi-unit environment is implemented on the premise of ensuring high availability, where the global sequence refers to a positive integer set that is automatically generated and guaranteed to be unique by the database system within a defined global scope, and the database instance refers to a logical entity that is used for running a database program, is capable of operating data, and is composed of components such as machine nodes, processes, and memories.

In a traditional single-machine database system, a global sequence is not supported, and only self-adding columns can be realized in a single table; or although the global sequence function is provided, the global sequence function is limited to be used in a single database, and the global uniqueness and high availability requirements of the global sequence in the multi-unit environment cannot be really met. In a novel distributed database system, the support of a global sequence or the single database is limited to be used in a single database and does not support a multi-unit environment; or the deployment of the multi-unit environment can be provided only by the distributed database system based on a distributed consistency protocol such as Paxos/Raft, and the like, although the scheme can ensure high availability, the data performance is lost due to factors such as performance loss caused by the distributed consistency protocol and network delay among cross-available areas, the request response time is increased, and the network traffic is increased; and the scheme also cannot balance stability, high availability and high performance due to the contradiction among the factors of stability, availability guarantee, network delay across available areas and the like of the external system.

Disclosure of Invention

The embodiment of the invention provides a global sequence generation method, a global sequence generation device, electronic equipment and a computer readable storage medium.

In a first aspect, an embodiment of the present invention provides a global sequence generation method.

Specifically, the global sequence generation method includes:

acquiring global sequence source data;

determining the number of data operation units, splitting the global sequence source data into global sequence source data segments according to the number of the data operation units, and establishing a mapping relation between the data operation units and the global sequence source data segments;

and generating a global sequence corresponding to the data operation unit according to the mapping relation between the data operation unit and the global sequence source data segment.

With reference to the first aspect, in a first implementation manner of the first aspect, the determining the number of the data operation units, splitting the global sequence source data into global sequence source data segments according to the number of the data operation units, and establishing a mapping relationship between the data operation units and the global sequence source data segments includes:

determining the number of data operation units and setting index values for the data operation units;

acquiring a data splitting step length;

and according to the number of the data operation units, the index value and the data splitting step length, sequentially splitting the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units to obtain a mapping relation between the data operation units and the global sequence source data segments, wherein the plurality of sub-global sequence source data segments corresponding to the data operation units are combined into the global sequence source data segment corresponding to the data operation units.

With reference to the first aspect and the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the generating a global sequence corresponding to the data operation unit according to a mapping relationship between the data operation unit and the global sequence source data segment includes:

acquiring a global sequence source data segment corresponding to the data operation unit according to the mapping relation between the data operation unit and the global sequence source data segment;

splitting the global sequence source data segment into two or more global sequence source data groups;

and generating a global sequence corresponding to the data operation unit according to the global sequence source data group.

With reference to the first aspect, the first implementation manner of the first aspect, and the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the obtaining, according to the mapping relationship between the data operation unit and the global sequence source data segment, the global sequence source data segment corresponding to the data operation unit includes:

determining a plurality of sub-global sequence source data segments corresponding to the data operation unit according to the index value of the data operation unit;

and combining the plurality of sub-global sequence source data segments corresponding to the data operation unit to obtain a global sequence source data segment corresponding to the data operation unit.

With reference to the first implementation manner of the first aspect, the second implementation manner of the first aspect, and the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the generating a global sequence corresponding to the data operation unit according to the global sequence source data group is implemented as:

in response to receiving a global sequence generation request, determining the working state of a global sequence source data cache corresponding to the data operation unit;

when the global sequence source data cache corresponding to the data operation unit is in an effective working state, reading a preset number of global sequence source data from the global sequence source data cache to generate a global sequence corresponding to the data operation unit;

when the global sequence source data cache corresponding to the data operation unit is in an invalid working state, creating the global sequence source data cache corresponding to the data operation unit according to the global sequence source data group, and reading a preset number of global sequence source data from the global sequence source data cache to generate a global sequence corresponding to the data operation unit.

With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, and the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the creating a global sequence source data cache corresponding to the data operation unit according to the global sequence source data group is implemented as:

determining the buffer capacity of the global sequence source data;

and selecting global sequence source data corresponding to the cache capacity of the global sequence source data from the global sequence source data group according to the cache capacity of the global sequence source data to create a global sequence source data cache corresponding to the data operation unit.

With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, the fourth implementation manner of the first aspect, and the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the selecting, from the global sequence source data group according to the global sequence source data cache capacity, global sequence source data corresponding to the global sequence source data cache capacity to create a global sequence source data cache corresponding to the data operation unit is implemented as:

selecting one global sequence source data group from the global sequence source data groups as a target global sequence source data group;

determining the working state of the target global sequence source data group;

when the target global sequence source data group is in an effective working state, selecting global sequence source data corresponding to the global sequence source data caching capacity from the global sequence source data group according to the global sequence source data caching capacity to create a global sequence source data cache corresponding to the data operation unit;

and when the target global sequence source data group is in an invalid working state, selecting another global sequence source data group from the global sequence source data groups as the target global sequence source data group until the target global sequence source data group is in an effective working state.

With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, the fourth implementation manner of the first aspect, the fifth implementation manner of the first aspect, and the sixth implementation manner of the first aspect, in a seventh implementation manner of the first aspect, after selecting, according to the global sequence source data cache capacity, global sequence source data corresponding to the global sequence source data cache capacity from the global sequence source data group to create a global sequence source data cache corresponding to the data operation unit, the method further includes:

and marking the selectable data of each global sequence source data group according to the cache capacity of the global sequence source data, the number of the global sequence source data groups and the identification information of the global sequence source data group where the selected data is located.

In a second aspect, an embodiment of the present invention provides a global sequence generation apparatus.

Specifically, the global sequence generating apparatus includes:

an acquisition module configured to acquire global sequence source data;

the splitting module is configured to determine the number of data operation units, split the global sequence source data into global sequence source data segments according to the number of the data operation units, and establish a mapping relation between the data operation units and the global sequence source data segments;

the generation module is configured to generate a global sequence corresponding to the data operation unit according to a mapping relation between the data operation unit and the global sequence source data segment.

With reference to the second aspect, in a first implementation manner of the second aspect, the splitting module includes:

the setting submodule is configured to determine the number of data operation units and set an index value for the data operation units;

a first obtaining submodule configured to obtain a data splitting step size;

the first splitting sub-module is configured to sequentially split the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units according to the number of the data operation units, the index value and the data splitting step length, so as to obtain a mapping relationship between the data operation units and the global sequence source data segments, wherein the plurality of sub-global sequence source data segments corresponding to the data operation units are combined into the global sequence source data segment corresponding to the data operation units.

With reference to the second aspect and the first implementation manner of the second aspect, in a second implementation manner of the second aspect, the generating module includes:

the second obtaining submodule is configured to obtain the global sequence source data segment corresponding to the data operation unit according to the mapping relation between the data operation unit and the global sequence source data segment;

a second splitting sub-module configured to split the global sequence source data segment into two or more global sequence source data groups;

a generation submodule configured to generate a global sequence corresponding to the data operation unit from the global sequence source data group.

With reference to the second aspect, the first implementation manner of the second aspect, and the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the second obtaining sub-module includes:

the determining submodule is configured to determine a plurality of sub-global sequence source data segments corresponding to the data operation unit according to the index value of the data operation unit;

and the combination submodule is configured to combine the plurality of sub-global sequence source data segments corresponding to the data operation unit to obtain a global sequence source data segment corresponding to the data operation unit.

With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, and the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the generation submodule is configured to:

With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, and the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the portion of the generation submodule that creates the global sequence source data cache corresponding to the data operation unit according to the global sequence source data group is configured to:

determining the buffer capacity of the global sequence source data;

With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, the fourth implementation manner of the second aspect, and the fifth implementation manner of the second aspect, in a sixth implementation manner of the second aspect, in the generation submodule, according to the global sequence source data cache capacity, global sequence source data corresponding to the global sequence source data cache capacity is selected from the global sequence source data group, and a global sequence source data cache corresponding to the data operation unit is created, and the generation submodule is configured to:

determining the working state of the target global sequence source data group;

With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, the fourth implementation manner of the second aspect, the fifth implementation manner of the second aspect, and the sixth implementation manner of the second aspect, in a seventh implementation manner of the second aspect, the generation submodule is further configured to:

In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory and a processor, where the memory is used to store one or more computer instructions that support a global sequence generation apparatus to execute the global sequence generation method in the first aspect, and the processor is configured to execute the computer instructions stored in the memory. The global sequence generating means may further comprise a communication interface for the global sequence generating means to communicate with other devices or a communication network.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer instructions for a global sequence generation apparatus, where the computer instructions include computer instructions for executing the global sequence generation method in the first aspect to the global sequence generation apparatus.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

according to the technical scheme, the global sequence source data are divided into the global sequence source data segments according to the number of the data operation units, and then the global sequence corresponding to the data operation units is generated according to the global sequence source data segments. The technical scheme can generate a global sequence which is decentralized and supports a multi-unit environment, so that the requirement of global uniqueness can be met without synchronization among multiple units, network delay among available areas and possible equipment single-point faults can be avoided, the problem of network delay caused by crossing units or available areas is solved, and the global target of global uniqueness, high performance and high availability is further achieved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the invention.

Drawings

Other features, objects and advantages of embodiments of the invention will become more apparent from the following detailed description of non-limiting embodiments thereof, when taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 illustrates a flow diagram of a global sequence generation method according to an embodiment of the invention;

FIG. 2 illustrates a multi-unit environment deployment architecture diagram;

FIG. 3 illustrates a global sequence source data split diagram according to an embodiment of the invention;

FIG. 4 shows a flow chart of step S102 of the global sequence generation method according to the embodiment shown in FIG. 1;

FIG. 5 illustrates a global sequence generation diagram according to an embodiment of the invention;

FIG. 6 shows a flow chart of step S103 of the global sequence generation method according to the embodiment shown in FIG. 1;

FIG. 7 shows a flowchart of step S601 of the global sequence generation method according to the embodiment shown in FIG. 6;

FIG. 8 is a block diagram illustrating a global sequence generation apparatus according to an embodiment of the present invention;

FIG. 9 is a block diagram of a splitting module 802 of the global sequence generation apparatus according to the embodiment shown in FIG. 8;

FIG. 10 is a block diagram illustrating the structure of a generation module 803 of the global sequence generation apparatus according to the embodiment shown in FIG. 8;

fig. 11 is a block diagram showing a second acquisition submodule 1001 of the global sequence generation apparatus according to the embodiment shown in fig. 10;

FIG. 12 shows a block diagram of an electronic device according to an embodiment of the invention;

FIG. 13 is a block diagram of a computer system suitable for use in implementing a global sequence generation method according to an embodiment of the present invention.

Detailed Description

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.

In the embodiments of the present invention, it is to be understood that terms such as "including" or "having", etc., are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may be present or added.

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. Embodiments of the present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

According to the technical scheme provided by the embodiment of the invention, the global sequence source data is divided into the global sequence source data segments according to the number of the data operation units, and then the global sequence corresponding to the data operation units is generated according to the global sequence source data segments. The technical scheme can generate a global sequence which is decentralized and supports a multi-unit environment, so that the requirement of global uniqueness can be met without synchronization among multiple units, network delay among available areas and possible equipment single-point faults can be avoided, the problem of network delay caused by crossing units or available areas is solved, and the global target of global uniqueness, high performance and high availability is further achieved.

Fig. 1 shows a flowchart of a global sequence generation method according to an embodiment of the present invention, and as shown in fig. 1, the global sequence generation method includes the following steps S101 to S103:

in step S101, global sequence source data is acquired;

in step S102, determining the number of data operation units, splitting the global sequence source data into global sequence source data segments according to the number of data operation units, and establishing a mapping relationship between the data operation units and the global sequence source data segments;

in step S103, a global sequence corresponding to the data operation unit is generated according to a mapping relationship between the data operation unit and the global sequence source data segment.

As mentioned above, as data technology has developed, the use of data has become more widespread. In order to provide richer and more extensive data services, there are many data systems in which multiple databases work in concert. In some application scenarios, multiple databases in a data system may be deployed on the same or different database instances, and the multiple database instances may also be in the same or different available areas, for example, as shown in fig. 2, a database system includes 4 databases A, B, C, D, where databases a and B are deployed on the same database instance X, databases C and D are deployed on different database instances Y and Z, respectively, and database instances X and Y are in the same available area 1, and database instance Z is in another available area 2. However, in many cases, it is necessary to ensure global uniqueness of a certain identification value among multiple databases, that is, to implement a global sequence of a multi-unit environment on the premise of ensuring high availability, where the global sequence refers to a positive integer set that is automatically generated and guaranteed uniqueness by a database system within a limited global scope, and the database instance refers to a logical entity that is used for running a database program, is capable of operating data, and is composed of components such as machine nodes, processes, and memories.

In view of the above problem, in this embodiment, a global sequence generation method is provided, in which global sequence source data is split into global sequence source data segments according to the number of data operation units, and a global sequence corresponding to the data operation unit is generated according to the global sequence source data segments. The technical scheme can generate a global sequence which is decentralized and supports a multi-unit environment, so that the requirement of global uniqueness can be met without synchronization among multiple units, network delay among available areas and possible equipment single-point faults can be avoided, the problem of network delay caused by crossing units or available areas is solved, and the global target of global uniqueness, high performance and high availability is further achieved.

The global sequence generation method can be implemented on a certain database node or a plurality of database nodes or even all database nodes. When the global sequence generation method is implemented on a certain database node, the database node is responsible for the distribution of all global sequence source data so as to generate a required global sequence, and when the global sequence generation method is implemented on a plurality of database nodes, each database node is responsible for the distribution of the global sequence source data which is managed and controlled by the database node in response to the received global sequence generation request, the database node corresponding to the global sequence source data needs to be obtained, and the database node corresponding to the distributed global sequence source data is found through routing.

In an embodiment of the present invention, the global sequence source data refers to source data that is obtained in advance and can be used as a basis for generating a subsequent global sequence, and has a data element identifier, for example, may be an unsigned positive integer sequence, and the like, where the data element identifier is used to identify each data in the global sequence source data, for example, if a certain global sequence source data has 90 data elements in total, which may also be called sequence values, then the identifiers of the 90 data elements, that is, sequence numbers of the sequence values, may be set to be from 1 to 90. The global sequence source data may be configured in advance by a user or other configuring party, or generated by a system syntax or based on other generation mechanisms, and the generation manner of the global sequence source data is not particularly limited in the present disclosure.

In an embodiment of the present invention, the data operation unit is configured to characterize a unit range of data operations, such as an available area, a database instance, or a database node. Wherein, a plurality of data operation units can form a complete data system.

In an embodiment of the present invention, after the global sequence source data is subjected to unitization processing by using one data operation unit as a unit, that is, after the global sequence source data is split into global sequence source data segments according to the number of the data operation units, a mapping relationship between the data operation units and the global sequence source data segments may be established, that is, the generated global sequence source data segments correspond to the data operation units, for example, as shown in fig. 3, if the number of the data operation units is 3, which are respectively marked as data operation unit 1, data operation unit 2 and data operation unit 3, the global sequence source data has 90 sequence values in total, and the sequence numbers of the sequence values are from 1 to 90, the global sequence source data may be equally divided into 3 global sequence source data segments, which are respectively: a global sequence source data segment 1 with sequence numbers from 1 to 30 corresponding to the data operation unit 1; a global sequence source data segment 2 with sequence numbers from 31 to 60 corresponding to the data operation unit 2; a global sequence source data segment 3 corresponding to the data manipulation unit 3 and having sequence numbers from 61 to 90. Certainly, when the global sequence source data is split into the global sequence source data segments, in addition to the average splitting method, other splitting methods may be adopted, for example, targeted splitting may be performed according to parameters such as actual requirements and data accommodation capacity of each data operation unit, and for example, a splitting step length may also be set to perform rolling step splitting.

That is, in an embodiment of the present invention, as shown in fig. 4, the step S102, namely, determining the number of data operation units, splitting the global sequence source data into global sequence source data segments according to the number of data operation units, and establishing a mapping relationship between the data operation units and the global sequence source data segments, includes the following steps S401 to S403:

in step S401, the number of data operation units is determined, and an index value is set for the data operation units;

in step S402, a data splitting step size is obtained;

in step S403, sequentially splitting the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units according to the number of the data operation units, the index value, and the data splitting step length, so as to obtain a mapping relationship between the data operation units and the global sequence source data segments, where the plurality of sub-global sequence source data segments corresponding to the data operation units are combined into the global sequence source data segment corresponding to the data operation unit.

In order to improve the effectiveness, reliability and anti-interference capability of data segmentation, in this embodiment, a stepping mechanism is used to split the global sequence source data.

Specifically, first, the number of data operation units is determined, and an index value is set for the data operation units, for example, as shown in the above example, if the number of data operation units is 3, the index values of the data operation units may be set to 1, 2, and 3.

And then acquiring a data splitting step size, namely a data operation range used when data splitting is performed for each round. The data splitting step length may be set according to factors such as a requirement of an actual application, a characteristic of the data operation unit, a capacity of the data operation unit, a characteristic of the global sequence source data, and a length of the global sequence source data, which is not specifically limited by the present disclosure.

Finally, according to the number of the data operation units, the index value and the data splitting step length, sequentially and rollably splitting the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units to obtain a mapping relation between the data operation units and the global sequence source data segments, and simultaneously enabling the combination of the plurality of sub-global sequence source data segments corresponding to the data operation units to be the global sequence source data segments corresponding to the data operation units, wherein each data operation unit has exclusive ownership of a part of source data in the global sequence source data, so that each subsequent data operation unit has exclusive ownership of data when generating a global sequence according to the global sequence source data segments corresponding to the subsequent data operation units, and does not generate data conflict with the global sequence generated by other units based on the same global sequence source data, meanwhile, each data operation unit only generates the global sequence belonging to the unit of the data operation unit, and does not need to interact with other data operation units, so that the uniqueness of the global sequence is ensured, unnecessary network communication is avoided, and further, data performance loss caused by network delay and distributed consistency protocol overhead is avoided.

For example, as shown in fig. 5, in the above example, if there are 90 sequence values in the global sequence source data, the sequence numbers of the sequence values are from 1 to 90, the data splitting step is set to 10, the number of the data operation units is 3, and the corresponding index values are 1, 2, and 3, respectively, then the data operation unit 1 is assigned with the sequence values with the sequence numbers from 1 to 10, the data operation unit 2 is assigned with the sequence values with the sequence numbers from 11 to 20, the data operation unit 3 is assigned with the sequence values with the sequence numbers from 21 to 30, then the data operation unit 1 is assigned with the sequence values with the sequence numbers from 31 to 40, the data operation unit 2 is assigned with the sequence values with the sequence numbers from 41 to 50, the data operation unit 3 is assigned with the sequence values with the sequence numbers from 51 to 60, finally the data operation unit 1 is assigned with the sequence values with the sequence numbers from 61 to 70, the data operation unit 2 is assigned with the sequence values with the sequence numbers from 71 to 80, the data operating unit 3 is assigned a sequence value of sequence numbers from 81 to 90. To this end, the global sequence source data may be sequentially and rollably divided into 3 global sequence source data segments, which are: the data operation unit comprises a global sequence source data segment 1 corresponding to the data operation unit 1, wherein the global sequence source data segment 1 comprises three sub-global sequence source data segments with sequence numbers from 1 to 10, 31 to 40, 61 to 70, a global sequence source data segment 2 corresponding to the data operation unit 2, wherein the global sequence source data segment 2 comprises three sub-global sequence source data segments with sequence numbers from 11 to 20, 41 to 50, 71 to 80, and a global sequence source data segment 3 corresponding to the data operation unit 3, and the global sequence source data segment 3 comprises three sub-global sequence source data segments with sequence numbers from 21 to 30, 51 to 60, 81 to 90.

In an embodiment of the present invention, as shown in fig. 6, the step S103 of generating a global sequence corresponding to the data operation unit according to a mapping relationship between the data operation unit and the global sequence source data segment includes the following steps S601-S603:

in step S601, obtaining a global sequence source data segment corresponding to the data operation unit according to a mapping relationship between the data operation unit and the global sequence source data segment;

in step S602, splitting the global sequence source data segment into two or more global sequence source data groups;

in step S603, a global sequence corresponding to the data operation unit is generated from the global sequence source data group.

In order to ensure high availability and high performance of the global sequence, in this embodiment, the global sequence source data segment corresponding to the data operation unit is firstly split into two or more global sequence source data groups, wherein each global sequence source data group has a respective independent data segment, and there is no intersection in data values between the data segments; and then generating a global sequence corresponding to the data operation unit according to the global sequence source data group obtained by splitting. The data grouping operation can effectively avoid or reduce the loss caused by single point of data failure, thereby further ensuring the high availability and high performance of the global sequence.

In an embodiment of the present invention, when the global sequence source data segment is divided into two or more global sequence source data groups, the number of the global sequence source data groups may be determined according to factors such as a requirement of an actual application, characteristics of the global sequence source data, a size of a subsequently used cache space for storing the source data, and a data stability requirement, which is not particularly limited in the present disclosure. For example, taking the data operation unit 1 as an example, the global sequence source data segment 1 corresponding to the sub-global sequence source data segment may be split into three global sequence source data groups, which are the global sequence source data group 1 with sequence number from 1 to 10, the global sequence source data group 2 with sequence number from 31 to 40, and the global sequence source data group 3 with sequence number from 61 to 70, respectively, according to the data range of the sub-global sequence source data segment.

In an embodiment of the present invention, as shown in fig. 7, the step S601, that is, the step of acquiring the global sequence source data segment corresponding to the data operation unit according to the mapping relationship between the data operation unit and the global sequence source data segment, includes the following steps S701 to S702:

in step S701, determining a plurality of sub-global sequence source data segments corresponding to the data operation units according to the index values of the data operation units;

in step S702, a plurality of sub-global sequence source data segments corresponding to the data operation unit are combined to obtain a global sequence source data segment corresponding to the data operation unit.

As mentioned above, the global sequence source data has been split into a plurality of global sequence source data segments having mapping relations respectively corresponding to the data operation units, so in this embodiment, when obtaining the global sequence source data segment corresponding to the data operation unit, a plurality of sub-global sequence source data segments corresponding to the data operation unit are first determined according to the index value of the data operation unit; and then combining the plurality of sub-global sequence source data segments corresponding to the data operation unit to obtain a global sequence source data segment corresponding to the data operation unit.

Still taking the above example as an example, combining the three sub-global sequence source data segments with the determined sequence numbers corresponding to the data operation units 1 from 1 to 10, 31 to 40, and 61 to 70 may obtain the global sequence source data segment 1 corresponding to the data operation unit 1, combining the three sub-global sequence source data segments with the determined sequence numbers corresponding to the data operation units 2 from 11 to 20, 41 to 50, and 71 to 80 may obtain the global sequence source data segment 2 corresponding to the data operation unit 2, and combining the three sub-global sequence source data segments with the determined sequence numbers corresponding to the data operation units 3 from 21 to 30, 51 to 60, and 81 to 90 may obtain the global sequence source data segment 3 corresponding to the data operation unit 3.

In an embodiment of the present invention, the step S603 of generating a global sequence corresponding to the data operation unit according to the global sequence source data group may be implemented as:

In order to improve the data reading speed, improve the data service quality, avoid frequent access to the bottom storage of the database and frequent network communication and remote call, and finally generate the required global sequence with high performance, in the embodiment, the source data is firstly stored in the cache, and then the data is read from the cache to generate the global sequence. In addition, considering that the state of the cache may change at any time, for example, working states such as non-creation, failure, exhausted data, failure exception, etc. may occur, when the source data is stored in the cache, it is necessary to determine the working state of the cache and then perform subsequent operations.

Specifically, firstly, in response to receiving a global sequence generation request, determining a working state of a global sequence source data cache corresponding to the data operation unit, if the global sequence source data cache corresponding to the data operation unit is in an effective working state, performing a normal data reading and storing process, and reading a preset number of global sequence source data from the global sequence source data cache to generate a global sequence corresponding to the data operation unit; if the global sequence source data cache corresponding to the data operation unit is in an invalid working state, the global sequence source data cache corresponding to the data operation unit is created according to the global sequence source data group, and then a preset number of global sequence source data are read from the global sequence source data cache to generate a global sequence corresponding to the data operation unit.

The effective working state of the global sequence source data cache means that the cache is available in the state, available data exist in the cache, and the data can be read; the working state of the cache invalidation of the global sequence source data refers to that in the working state, the cache is unavailable, no available data exists or the data is unreadable, such as the cache is not created, the cache is failed, the data in the cache is exhausted, the cache is in a fault exception and the like. When the invalid working state is that the data in the cache is exhausted, the creating of the global sequence source data cache corresponding to the data operation unit may be implemented to acquire new global sequence source data to fill in the cache space, or to use the new global sequence source data to cover old global sequence source data in the cache space.

The preset number can be set according to factors such as the requirement of practical application and the size of the cache space, and the preset number is not particularly limited by the disclosure.

In an embodiment of the present invention, the step of creating a global sequence source data cache corresponding to the data operation unit according to the global sequence source data group may be implemented as:

determining the buffer capacity of the global sequence source data;

The global sequence source data cache capacity refers to the amount of data that can be stored in the global sequence source data cache, and may be determined according to the actual application requirement, taking into consideration a plurality of factors such as a data reading speed, a total cache capacity, a data reading requirement of the global sequence, and the like, which is not specifically limited by the present disclosure.

For example, if the global sequence source data cache capacity is 5, 5 source data are selected from the global sequence source data group and stored in the cache each time, so as to form global sequence source data cache data corresponding to a certain data operation unit.

Further, in an embodiment of the present invention, the step of selecting, from the global sequence source data group, global sequence source data corresponding to the global sequence source data cache capacity according to the global sequence source data cache capacity to create a global sequence source data cache corresponding to the data operation unit may be implemented as:

determining the working state of the target global sequence source data group;

In order to further avoid or reduce the loss caused by single point of failure, high availability and high performance of the global sequence are realized, in the realization mode, the source of the cache data is not fixed, but the data to be stored in the cache is changeably selected from the global sequence source data group.

Specifically, one of the global sequence source data groups is selected from the global sequence source data groups as a target global sequence source data group, wherein the selection can be random selection or targeted selection generated according to the characteristics of the global sequence source data groups; then determining the working state of the selected target global sequence source data group; if the target global sequence source data group is in an effective working state, executing a normal process, selecting global sequence source data corresponding to the global sequence source data caching capacity from the global sequence source data group according to the global sequence source data caching capacity, and creating a global sequence source data cache corresponding to the data operation unit; and if the target global sequence source data group is in an invalid working state, selecting another global sequence source data group from the global sequence source data groups as the target global sequence source data group until the target global sequence source data group is in an effective working state.

The effective working state of the target global sequence source data group means that available data exist in the target global sequence source data group and the data can be read in the state; the invalid working state of the target global sequence source data group means that no available data exists in the target global sequence source data group or the data is unreadable in the state.

For example, if the global sequence source data cache capacity is set to 5, still taking the above example as an example, first randomly selecting the global sequence source data group 2 with sequence numbers from 31 to 40 from three global sequence source data groups as a target global sequence source data group, then judging the working state of the global sequence source data group 2, if the working state is valid, executing a normal flow, selecting 5 global sequence source data from the global sequence source data group 2, for example, selecting source data with sequence numbers from 31 to 35, and creating a global sequence source data cache corresponding to the data operation unit; if the target global sequence source data group is in the invalid working state, randomly selecting the global sequence source data group 1 with the serial numbers from 1 to 10 from the other two global sequence source data groups as a new target global sequence source data group, continuously judging the working state of the global sequence source data group 1, and repeating the steps until the determined target global sequence source data group is in the valid working state, so that a global sequence source data cache corresponding to the data operation unit can be created.

In an embodiment of the present invention, after the selecting, according to the global sequence source data cache capacity, global sequence source data corresponding to the global sequence source data cache capacity from the global sequence source data group to create a global sequence source data cache corresponding to the data operation unit, the method further includes:

In order to improve data reading and storage performance and enhance fault recovery capability, in this implementation manner, after a global sequence source data cache corresponding to a data operation unit is created, selectable data of each global sequence source data group is marked according to the capacity of the global sequence source data cache, the number of global sequence source data groups and identification information of the global sequence source data group in which selected data is located, so as to represent a usable data range of next selected data.

For example, if 5 source data with sequence numbers from 31 to 35 are selected from the global sequence source data group 2 to create a global sequence source data cache, the next data that can be selected in the global sequence source data group 2 is the 5 source data with sequence numbers from 36 to 40, and the 5 source data are marked, so that the speed of selecting data next time can be increased, and the fault occurrence position can be quickly determined after the data or the device has a fault, thereby enhancing the fault recovery capability.

The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention.

Fig. 8 is a block diagram illustrating a global sequence generating apparatus according to an embodiment of the present invention, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 8, the global sequence generating apparatus includes:

an obtaining module 801 configured to obtain global sequence source data;

a splitting module 802, configured to determine the number of data operation units, split the global sequence source data into global sequence source data segments according to the number of data operation units, and establish a mapping relationship between the data operation units and the global sequence source data segments;

a generating module 803, configured to generate a global sequence corresponding to the data operation unit according to a mapping relationship between the data operation unit and the global sequence source data segment.

In view of the above problem, in this embodiment, a global sequence generation apparatus is provided, which splits global sequence source data into global sequence source data segments according to the number of data operation units, and further generates a global sequence corresponding to the data operation unit according to the global sequence source data segments. The technical scheme can generate a global sequence which is decentralized and supports a multi-unit environment, so that the requirement of global uniqueness can be met without synchronization among multiple units, network delay among available areas and possible equipment single-point faults can be avoided, the problem of network delay caused by crossing units or available areas is solved, and the global target of global uniqueness, high performance and high availability is further achieved.

That is, in an embodiment of the present invention, as shown in fig. 9, the splitting module 802 includes:

a setting submodule 901 configured to determine the number of data operation units and set an index value for the data operation units;

a first obtaining submodule 902 configured to obtain a data splitting step size;

a first splitting sub-module 903, configured to sequentially split the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units according to the number of the data operation units, the index value, and the data splitting step length, so as to obtain a mapping relationship between the data operation units and the global sequence source data segments, where a plurality of sub-global sequence source data segments corresponding to the data operation units are combined into a global sequence source data segment corresponding to the data operation units.

Specifically, the setting sub-module 901 first determines the number of data operation units and sets index values for the data operation units, for example, as shown in the above example, if the number of data operation units is 3, the index values of the data operation units may be set to 1, 2, and 3.

The first obtaining sub-module 902 then obtains a data splitting step size, i.e., a data operation range used when performing each round of data splitting. The data splitting step length may be set according to factors such as a requirement of an actual application, a characteristic of the data operation unit, a capacity of the data operation unit, a characteristic of the global sequence source data, and a length of the global sequence source data, which is not specifically limited by the present disclosure.

Finally, the first splitting sub-module 903 sequentially and rollably splits the global sequence source data into a plurality of sub-global sequence source data segments corresponding to the data operation units according to the number of the data operation units, the index value and the data splitting step length to obtain a mapping relationship between the data operation units and the global sequence source data segments, and simultaneously, the combination of the plurality of sub-global sequence source data segments corresponding to the data operation units is made into the global sequence source data segment corresponding to the data operation units, and each data operation unit monopolizes a part of the source data in the global sequence source data, so that each subsequent data operation unit has data monopolarity when generating a global sequence according to the global sequence source data segment corresponding to the subsequent data operation unit, and does not generate data conflict with a global sequence generated by other units based on the same global sequence source data, meanwhile, each data operation unit only generates the global sequence belonging to the unit of the data operation unit, and does not need to interact with other data operation units, so that the uniqueness of the global sequence is ensured, unnecessary network communication is avoided, and further, data performance loss caused by network delay and distributed consistency protocol overhead is avoided.

In an embodiment of the present invention, as shown in fig. 10, the generating module 803 includes:

a second obtaining submodule 1001 configured to obtain a global sequence source data segment corresponding to the data operation unit;

a second splitting sub-module 1002 configured to split the global sequence source data segment into two or more global sequence source data groups;

a generating submodule 1003 configured to generate a global sequence corresponding to the data operation unit according to the global sequence source data group.

In order to ensure high availability and high performance of the global sequence, in this embodiment, the second splitting sub-module 1002 first splits the global sequence source data segment corresponding to the data operation unit into two or more global sequence source data groups, wherein each global sequence source data group has a respective independent data segment, and there is no intersection in data values between the data segments; then, the generating sub-module 1003 generates a global sequence corresponding to the data operation unit according to the global sequence source data group obtained by splitting. The data grouping operation can effectively avoid or reduce the loss caused by single point of data failure, thereby further ensuring the high availability and high performance of the global sequence.

In an embodiment of the present invention, when the second splitting sub-module 1002 splits the global sequence source data segment into two or more global sequence source data groups, the number of the global sequence source data groups may be determined according to factors such as requirements of practical applications, characteristics of the global sequence source data, a size of a subsequently used cache space for storing the source data, and data stability requirements, which are not particularly limited in this disclosure. For example, taking the data operation unit 1 as an example, the global sequence source data segment 1 corresponding to the sub-global sequence source data segment may be split into three global sequence source data groups, which are the global sequence source data group 1 with sequence number from 1 to 10, the global sequence source data group 2 with sequence number from 31 to 40, and the global sequence source data group 3 with sequence number from 61 to 70, respectively, according to the data range of the sub-global sequence source data segment.

In an embodiment of the present invention, as shown in fig. 11, the second obtaining sub-module 1001 includes:

a determining submodule 1101 configured to determine a plurality of sub-global sequence source data segments corresponding to the data operation unit according to the index value of the data operation unit;

and a combining submodule 1102 configured to combine the plurality of sub-global sequence source data segments corresponding to the data operation unit to obtain a global sequence source data segment corresponding to the data operation unit.

As mentioned above, the global sequence source data has been split into a plurality of global sequence source data segments having mapping relations respectively corresponding to the data operation units, so in this embodiment, when the second obtaining sub-module 1001 obtains the global sequence source data segments corresponding to the data operation units, the determining sub-module 1101 first determines a plurality of sub-global sequence source data segments corresponding to the data operation units according to the index values of the data operation units; then, the combining submodule 1102 combines the plurality of sub-global sequence source data segments corresponding to the data operation unit, so as to obtain a global sequence source data segment corresponding to the data operation unit.

In an embodiment of the present invention, the generating sub-module 1003 may be implemented as:

In order to increase the data reading speed, improve the data service quality, avoid frequent access to the bottom storage of the database and frequent network communication and remote invocation, and finally generate the required global sequence with high performance, in this embodiment, the generation submodule 1003 stores the source data into the cache, and then reads the data from the cache to generate the global sequence. In addition, considering that the state of the cache may change at any time, for example, working states such as non-creation, failure, exhausted data, failure exception, etc. may occur, when the source data is stored in the cache, it is necessary to determine the working state of the cache and then perform subsequent operations.

In an embodiment of the present invention, the part of the generation sub-module 1003 that creates the global sequence source data cache corresponding to the data operation unit according to the global sequence source data group may be configured to:

determining the buffer capacity of the global sequence source data;

Further, in an embodiment of the present invention, the generating sub-module 1003 selects, according to the global sequence source data cache capacity, a global sequence source data corresponding to the global sequence source data cache capacity from the global sequence source data group to create a portion of the global sequence source data cache corresponding to the data operation unit, and may be configured to:

determining the working state of the target global sequence source data group;

In an embodiment of the present invention, the generating sub-module 1003 is further configured to:

In order to improve data reading and storing performance and enhance fault recovery capability, in this implementation, the generating sub-module 1003 is further configured to, after creating a global sequence source data cache corresponding to the data operation unit, further mark selectable data of each global sequence source data group according to the global sequence source data cache capacity, the number of the global sequence source data groups, and identification information of the global sequence source data group where the selected data is located, so as to represent a data range usable by the next selected data.

Fig. 12 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 12, the electronic device 1200 includes a memory 1201 and a processor 1202; wherein the content of the first and second substances,

the memory 1201 is used to store one or more computer instructions, which are executed by the processor 1202 to implement any of the method steps described above.

FIG. 13 is a schematic block diagram of a computer system suitable for implementing a global sequence generation method according to an embodiment of the present invention.

As shown in fig. 13, the computer system 1300 includes a Central Processing Unit (CPU)1301 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)1302 or a program loaded from a storage portion 1308 into a Random Access Memory (RAM) 1303. In the RAM1303, various programs and data necessary for the operation of the system 1300 are also stored. The CPU1301, the ROM1302, and the RAM1303 are connected to each other via a bus 1304. An input/output (I/O) interface 1305 is also connected to bus 1304.

The following components are connected to the I/O interface 1305: an input portion 2406 including a keyboard, a mouse, and the like; an output section 1307 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1308 including a hard disk and the like; and a communication section 1309 including a network interface card such as a LAN card, a modem, or the like. The communication section 1309 performs communication processing via a network such as the internet. A drive 1310 is also connected to the I/O interface 1305 as needed. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1310 as necessary, so that a computer program read out therefrom is mounted into the storage portion 1308 as necessary.

In particular, the above described method may be implemented as a computer software program according to an embodiment of the present invention. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the global sequence generation method. In such embodiments, the computer program may be downloaded and installed from a network via communications component 1309 and/or installed from removable media 1311.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.

As another aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may be a computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present invention.

The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the embodiments of the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present invention are mutually replaced to form the technical solution.

Claims

1. A global sequence generation method, comprising:

acquiring global sequence source data;

2. The method according to claim 1, wherein the determining the number of data operation units, splitting the global sequence source data into global sequence source data segments according to the number of data operation units, and establishing a mapping relationship between the data operation units and the global sequence source data segments comprises:

acquiring a data splitting step length;

3. The method according to claim 2, wherein the generating a global sequence corresponding to the data operation unit according to the mapping relationship between the data operation unit and the global sequence source data segment includes:

4. The method according to claim 3, wherein the obtaining the global sequence source data segment corresponding to the data operation unit according to the mapping relationship between the data operation unit and the global sequence source data segment includes:

5. The method of claim 3, wherein the generating a global sequence corresponding to the data manipulation unit from the global sequence source data set is implemented as:

6. The method of claim 5, wherein the creating a global sequence source data cache corresponding to the data manipulation unit from the global sequence source data group is implemented as:

determining the buffer capacity of the global sequence source data;

7. The method of claim 6, wherein the selecting, from the global sequence source data group, the global sequence source data corresponding to the global sequence source data cache capacity according to the global sequence source data cache capacity to create the global sequence source data cache corresponding to the data manipulation unit is implemented as:

determining the working state of the target global sequence source data group;

8. The method of claim 6, wherein after selecting the global sequence source data corresponding to the global sequence source data cache size from the global sequence source data group according to the global sequence source data cache size to create the global sequence source data cache corresponding to the data operation unit, the method further comprises:

9. A global sequence generation apparatus, comprising:

an acquisition module configured to acquire global sequence source data;

10. The apparatus of claim 9, wherein the splitting module comprises:

a first obtaining submodule configured to obtain a data splitting step size;

11. The apparatus of claim 10, wherein the generating module comprises:

12. The apparatus of claim 11, wherein the second acquisition submodule comprises:

13. The apparatus of claim 11, wherein the generation submodule is configured to:

14. The apparatus of claim 13, wherein the portion of the generation submodule that creates a global sequence source data cache corresponding to the data manipulation unit from the global sequence source data group is configured to:

determining the buffer capacity of the global sequence source data;

15. The apparatus according to claim 14, wherein the generation sub-module selects, according to the global sequence source data cache capacity, global sequence source data corresponding to the global sequence source data cache capacity from the global sequence source data group to create a global sequence source data cache corresponding to the data operation unit, and is configured to:

determining the working state of the target global sequence source data group;

16. The apparatus of claim 14, wherein the generation submodule is further configured to:

17. An electronic device comprising a memory and a processor; wherein the content of the first and second substances,

the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any of claims 1-8.

18. A computer-readable storage medium having stored thereon computer instructions, characterized in that the computer instructions, when executed by a processor, carry out the method steps of any of claims 1-8.