CN117931811A

CN117931811A - Database processing method, device, computer equipment and storage medium

Info

Publication number: CN117931811A
Application number: CN202410333847.5A
Authority: CN
Inventors: 葛凯凯; 邬沛君; 李志阳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2024-03-22
Filing date: 2024-03-22
Publication date: 2024-04-26
Anticipated expiration: 2044-03-22
Also published as: CN117931811B

Abstract

The present application relates to a database processing method, apparatus, computer device, storage medium and computer program product. The embodiment of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the following steps: acquiring a table space expansion request aiming at a target database; the tablespace expansion request carries an expansion slice identifier corresponding to the expansion tablespace slice; determining a first storage unit group, determining a first storage partition from the first storage unit group, and generating physical storage mapping information corresponding to the expansion slice identifier; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units; physical storage mapping information is returned to the requestor to cause the requestor to store database data associated with the extended tablespace slice to a first storage partition in the first set of storage locations. The method can improve the utilization rate of the storage space.

Description

Database processing method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to a database processing method, apparatus, computer device, storage medium, and computer program product.

Background

With the development of computer technology, databases, which are warehouses that organize, store, and manage data by data structure, have emerged. Databases are widely used, for example, businesses use databases to manage business data and schools use databases to manage teaching data.

In the conventional technology, the table space of the database is usually expanded in units of small tables, and the table space is directly mapped to the physical storage units on the memory in units of small tables, which easily results in waste of the physical storage units.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a database processing method, apparatus, computer device, computer readable storage medium, and computer program product that can improve the utilization of storage space.

The application provides a database processing method, which comprises the following steps:

Acquiring a table space expansion request aiming at a target database; the tablespace expansion request carries an expansion slice identifier corresponding to an expansion tablespace slice; the tablespace of the target database is expanded by taking a tablespace slice as a unit;

Determining a first storage unit group corresponding to the extended slice identifier, and determining a first storage partition corresponding to the extended slice identifier from the first storage unit group; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group;

generating physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition;

And returning the physical storage mapping information to a requester corresponding to the tablespace expansion request, so that the requester stores database data associated with the expansion tablespace slice to the first storage partition in the first storage unit group based on the physical storage mapping information.

The application also provides a database processing device, which comprises:

The request acquisition module is used for acquiring a table space expansion request aiming at the target database; the tablespace expansion request carries an expansion slice identifier corresponding to an expansion tablespace slice; the tablespace of the target database is expanded by taking a tablespace slice as a unit;

The storage position determining module is used for determining a first storage unit group corresponding to the expansion slice identifier, and determining a first storage partition corresponding to the expansion slice identifier from the first storage unit group; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group;

The mapping information generation module is used for generating physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group and the first storage partition;

And the mapping information sending module is used for returning the physical storage mapping information to a requester corresponding to the tablespace expansion request so that the requester stores database data associated with the expansion tablespace slice into the first storage partition in the first storage unit group based on the physical storage mapping information.

Acquiring an expansion table space slice of a target database; the tablespace of the target database is expanded by taking a tablespace slice as a unit;

generating a tablespace expansion request carrying an expansion slice identifier corresponding to the expansion tablespace slice;

Transmitting the tablespace expansion request to an allocation node; the allocation node is configured to determine a first storage unit group corresponding to the extended slice identifier, determine a first storage partition corresponding to the extended slice identifier from the first storage unit group, and generate physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition corresponding to the extended slice identifier; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group;

and storing database data associated with the extended tablespace slice to the first storage partition in the first storage unit group based on the physical storage mapping information fed back by the allocation node.

The application also provides a database processing device, which comprises:

The slice acquisition module is used for acquiring the expansion table space slice of the target database; the tablespace of the target database is expanded by taking a tablespace slice as a unit;

The request generation module is used for generating a tablespace expansion request carrying an expansion slice identifier corresponding to the expansion tablespace slice;

A request sending module, configured to send the tablespace expansion request to an allocation node; the allocation node is configured to determine a first storage unit group corresponding to the extended slice identifier, determine a first storage partition corresponding to the extended slice identifier from the first storage unit group, and generate physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition corresponding to the extended slice identifier; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group;

And the data storage module is used for storing database data associated with the extended table space slice to the first storage partition in the first storage unit group based on the physical storage mapping information fed back by the distribution node.

Acquiring a database operation request aiming at a target database; the database operation request carries a data page identifier corresponding to a target data page to which operation data belong;

determining a target tablespace slice to which the target data page belongs in the tablespace of the target database based on the data page identification;

Generating a data search request carrying a target slice identifier corresponding to the target table space slice, and sending the data search request to an allocation node; the allocation node is used for inquiring a target storage unit group and a target storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier; the target storage unit group comprises physical storage units respectively provided by a plurality of memories, and the target storage partition comprises local storage spaces respectively corresponding to the physical storage units in the target storage unit group;

And searching operation data in the target data page from the target storage partition in the target storage unit group based on the target storage mapping information fed back by the distribution node.

The application also provides a database processing device, which comprises:

The database operation request acquisition module is used for acquiring a database operation request aiming at a target database; the database operation request carries a data page identifier corresponding to a target data page to which operation data belong;

a tablespace data determining module, configured to determine, based on the data page identifier, a target tablespace slice to which the target data page belongs in a tablespace of the target database;

The data search request sending module is used for generating a data search request carrying a target slice identifier corresponding to the target table space slice and sending the data search request to an allocation node; the allocation node is used for inquiring a target storage unit group and a target storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier; the target storage unit group comprises physical storage units respectively provided by a plurality of memories, and the target storage partition comprises local storage spaces respectively corresponding to the physical storage units in the target storage unit group;

An operation data searching module, configured to search operation data located in the target data page from the target storage partition in the target storage unit group based on the target storage mapping information fed back by the allocation node;

The processing result determining module is used for processing the searched operation data to obtain a processing result corresponding to the database operation request.

The application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the database processing method when executing the computer program.

The application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the database processing method described above.

The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the database processing method described above.

The database processing method, the database processing device, the computer equipment, the storage medium and the computer program product acquire a table space expansion request aiming at a target database; the tablespace expansion request carries an expansion slice identifier corresponding to the expansion tablespace slice; the tablespace of the target database is expanded by taking tablespace slices as units; determining a first storage unit group corresponding to the expansion slice identifier, and determining a first storage partition corresponding to the expansion slice identifier from the first storage unit group; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group; generating physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group and the first storage partition; and returning physical storage mapping information to a requester corresponding to the tablespace expansion request, so that the requester stores database data associated with the expanded tablespace slice to a first storage partition in a first storage unit group based on the physical storage mapping information. In this way, when the tablespace of the database is expanded, the tablespace slice is used as a unit to be expanded, the expanded tablespace slice is mapped to the first storage unit group, the expanded tablespace slice is further mapped to the first storage partition in the first storage unit group, the related database data of the expanded tablespace slice is used for the first storage partition stored in the first storage unit group, and the first storage partition comprises the local storage spaces of a plurality of physical storage units, so that the database data belonging to the same tablespace slice are scattered on different physical storage units, the database data can be distributed in the storage unit group in a partition discrete manner, the physical storage units are prevented from being exclusively used, and the storage space utilization rate is effectively improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.

FIG. 1 is a diagram of an application environment for a database processing method in one embodiment;

FIG. 2 is a flow diagram of a database processing method in one embodiment;

FIG. 3 is a diagram of partition group initialization in one embodiment;

FIG. 4 is a schematic diagram of a table space slice break up map in one embodiment;

FIG. 5 is a schematic diagram of a tablespace expansion in one embodiment;

FIG. 6 is a diagram of a partition group and partitions in one embodiment;

FIG. 7 is a flow chart of a database processing method in another embodiment;

FIG. 8 is a flow chart of a database processing method in another embodiment;

FIG. 9 is a flow diagram of a lookup database data in one embodiment;

FIG. 10 is a schematic diagram of multi-level data lookup in one embodiment;

FIG. 11 is a diagram of database data partition break up of a mysql database in one embodiment;

FIG. 12 is a block diagram of a database processing apparatus in one embodiment;

FIG. 13 is a block diagram of a database processing apparatus in another embodiment;

FIG. 14 is a block diagram of a database processing apparatus in another embodiment;

FIG. 15 is an internal block diagram of a computer device in one embodiment;

fig. 16 is an internal structural view of a computer device in another embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The embodiment of the application can be applied to various scenes, including but not limited to cloud technology, artificial intelligence, intelligent transportation, auxiliary driving and the like.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. Cloud technology relates to technical contents such as cloud computing, cloud storage, databases, big data and the like.

Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside. At present, the storage method of the storage system is as follows: when creating a logical volume, each logical volume is assigned a physical storage unit, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage unit of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object. The process of allocating physical storage units for the logical volumes by the storage system specifically includes: physical storage units are divided into stripes in advance according to the group of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of INDEPENDENT DISK), and a logical volume can be understood as a stripe, whereby physical storage units are allocated to the logical volume.

The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application. A Database MANAGEMENT SYSTEM (DBMS) is a computer software system designed for managing a Database, and generally has basic functions such as storage, interception, security, and backup. The database management system may classify according to the database model it supports, e.g., relational, XML (Extensible Markup Language ); or by the type of computer supported, e.g., server cluster, mobile phone; or by the query language used, such as SQL (structured query language (Structured Query Language), XQuery, or by the energy impact emphasis, such as maximum-scale, maximum-speed, or other classification means, regardless of which classification means is used, some DBMSs can cross-category, for example, while supporting multiple query languages.

The database processing method provided by the embodiment of the application can be applied to an application environment shown in figure 1. The database processing method is applied to a database system. The database system includes a compute node, an allocation node, and a storage node. The computing node is responsible for the computing portion of the database and the storage node is responsible for the storage portion of the database. The allocation node is configured to allocate corresponding storage resources for a tablespace of the database. The computing node, the storage node, and the distribution node may be servers, which may be implemented as a stand-alone server or as a server cluster or cloud server composed of multiple servers.

Specifically, the tablespace of the target database is expanded in tablespace slices. The computing node acquires an expansion tablespace slice of the target database, generates a tablespace expansion request carrying an expansion slice identifier corresponding to the expansion tablespace slice, and sends the tablespace expansion request to the distribution node. The distribution node determines a first storage unit group corresponding to the expansion slice identifier, determines a first storage partition corresponding to the expansion slice identifier from the first storage unit group, and generates physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group corresponding to the expansion slice identifier and the first storage partition. The first storage unit group comprises physical storage units respectively provided by a plurality of memories, the memories are provided by storage nodes, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group. The distribution node returns physical storage mapping information to the computing node. The computing node stores database data associated with the extended tablespace slice to a first storage partition in a first set of storage units based on the physical storage mapping information.

It can be understood that if the target database is a database integrated with the storage, the computing node, the allocation node and the storage node corresponding to the target database are the same node. If the target database is a database with separated storage and calculation, the calculation node, the distribution node and the storage node corresponding to the target database are different nodes.

In one embodiment, as shown in fig. 2, a database processing method is provided, and an example of application of the method to the allocation node in fig. 1 is described. Wherein:

Step S202, obtaining a table space expansion request aiming at a target database; the tablespace expansion request carries an expansion slice identifier corresponding to the expansion tablespace slice; the tablespace of the target database is extended in tablespace slices.

The target database refers to a database to be processed. The tablespace of a database refers to the logical storage space in which database data is stored. Logical storage space needs to be mapped to physical storage space. The logical storage space is used to represent the logical storage structure of the database data. The physical storage space is used to represent the physical storage structure of the database data. The logical storage space is managed based on a logical structure, so that a user can not need to care about the details of the underlying physical storage space; the physical storage space is directly associated with the hardware device of the computer, and the storage position of the data is limited by the hardware device.

As database data continues to increase, the tablespace increases adaptively. As database data continues to increase, it is necessary to expand the table space in time. The tablespace of the database is extended in tablespace slices. The tablespace expansion request is used for requesting physical storage mapping information corresponding to the tablespace blocks to be expanded. Extended tablespace slices refer to blocks of tablespace that need to be extended. The number of newly added tablespace slices required may be determined based on the amount of data the database needs to be increased.

The slice identifier is used to identify a tablespace slice, for example, the slice number of the tablespace slice may be used as the slice identifier of the tablespace slice. The extended slice identification is used to identify an extended tablespace slice.

Specifically, when the target database needs to expand the tablespace, the allocation node acquires a tablespace expansion request aiming at the target database, responds to the tablespace expansion request, and allocates corresponding storage resources for the expanded tablespace slice so as to store database data of the target database by using the corresponding storage resources.

In one embodiment, the target database is a separate-from-memory database, e.g., the target database is a separate-from-memory cloud database. When a computation node corresponding to a target database needs to expand a table space, a table space expansion request aiming at the target database is generated, the table space expansion request is sent to an allocation node, and the table space expansion request carries an expansion slice identifier corresponding to an expansion table space slice. And the allocation node responds to the table space expansion request, allocates corresponding storage resources for the expansion table space slice from the storage node cluster, and returns related information of the allocated storage resources to the computing node so that the computing node can store database data of the target database by utilizing the corresponding storage resources.

Step S204, a first storage unit group corresponding to the expansion slice identifier is determined, and a first storage partition corresponding to the expansion slice identifier is determined from the first storage unit group; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group.

The memory is a storage device, and for example, the memory may be a hard disk. The memory is provided with a physical space of a certain size for storing data, and the physical space can be divided into a plurality of physical storage units for storing and managing the data. The size of the physical storage unit may be set according to actual needs, for example, the size of the physical storage unit is 2GB. A memory cell group includes physical memory cells provided by a plurality of memories, respectively. In one memory cell group, one memory may provide at least one physical memory cell. For example, to achieve high concurrency, one memory cell group includes one physical memory cell provided by a plurality of memories, respectively.

It will be appreciated that the total space capacity of the set of storage units is greater than the slice capacity of the extended tablespace slice. The first set of memory cells is a set of memory cells allocated to an extended tablespace slice. The first memory partition is the physical memory space allocated to the extended tablespace slice in the first group of memory cells. The first storage partition comprises local storage spaces corresponding to all the physical storage units in the first storage unit group, wherein the local storage spaces corresponding to the physical storage units refer to part of the storage spaces of the physical storage units, namely, part of the storage spaces are acquired from all the physical storage units in the first storage unit group to form the first storage partition. Therefore, the expansion table space slice is scattered and mapped to each physical storage unit in the first storage unit group, so that the local storage space of a single physical storage unit in the first storage unit group is used for storing partial database data related to the expansion table space slice, and the database data belonging to the same table space slice are scattered on different physical storage units in the first storage unit group, so that the physical storage units can be effectively prevented from being exclusive, the hot spot phenomenon of the physical storage units is avoided, the utilization rate of the physical storage units is improved, and the data security is also improved. It is understood that in the first memory partition, the sizes of the local memory spaces corresponding to the physical memory units may be the same or different.

Specifically, the tablespace expansion request carries a slice identifier corresponding to the expanded tablespace slice, and the allocation node allocates a storage unit group and a storage partition for the expanded tablespace slice. The allocation node determines a first storage unit group corresponding to the extended tablespace slice, that is, determines a first storage unit group corresponding to the extended tablespace slice identifier, for example, acquires a storage unit group not allocated to other tablespace slices as the first storage unit group; acquiring a storage unit group which has been allocated to other tablespace slices but has free storage capacity as a first storage unit group; etc. Further, the allocation node determines, from the first storage unit group, a first storage partition corresponding to the extended table space slice, that is, determines a first storage partition corresponding to the extended slice identifier. For example, a free memory partition matching the slice capacity of the extended tablespace slice may be acquired from the first memory cell group as the first memory partition corresponding to the extended slice identifier. The first storage partition corresponding to the extended tablespace slice comprises local storage spaces respectively corresponding to all physical storage units in the first storage unit group, so that the monopolization of a single physical storage unit by a single tablespace slice can be effectively avoided, the waste of the physical storage unit caused under the condition that the tablespace slice is not filled with database data is avoided, and the utilization rate of the physical storage unit is improved.

Step S206, generating physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition.

Specifically, the allocation node generates physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition.

Step S208, the physical storage mapping information is returned to the requester corresponding to the tablespace expansion request, so that the requester stores the database data associated with the expanded tablespace slice to the first storage partition in the first storage unit group based on the physical storage mapping information.

The database data associated with the extended tablespace slice refers to database data which needs to be stored in the extended tablespace slice. It will be appreciated that the extended tablespace slice belongs to a logical storage space, the first storage partition in the first storage unit group belongs to a physical storage space where the database data ultimately needs to be stored. For example, when the current data of the database to be written into the target database is 2GB of service data, the slice capacity of the tablespace slice is 2GB, and the tablespace slice a is newly added for the service data, the service data belongs to the tablespace slice a, and after determining the first storage unit group and the first storage partition corresponding to the tablespace slice a, the computing node writes the service data into the first storage partition in the first storage unit group.

Specifically, the allocation node may return physical storage mapping information to a requestor corresponding to the tablespace expansion request. The requestor may determine a first storage unit group and a first storage partition corresponding to the extended tablespace slice based on the physical storage mapping information, and store database data associated with the extended tablespace slice to the first storage partition in the first storage unit group. It can be understood that the space capacity of the first storage partition is consistent with the slice capacity of the extended table space slice, the first storage partition comprises local storage spaces corresponding to the physical storage units in the first storage unit group, and database data associated with the extended table space slice can be distributed on the physical storage units in the first storage unit group in a scattered manner, so that the storage space utilization rate is effectively improved.

In one embodiment, the allocation node uses the location information of the first storage unit group and the first storage partition as physical storage mapping information corresponding to the extended slice identifier. The requestor may locate the first storage cell group and the first storage partition based on the location information to send corresponding database data to the first storage partition in the first storage cell group for storage.

In the database processing method, when the table space of the database is expanded, the table space slice is used as a unit for expanding, the expanded table space slice is mapped to the first storage unit group, the expanded table space slice is further mapped to the first storage partition in the first storage unit group, the related database data of the expanded table space slice is used for the first storage partition stored in the first storage unit group, and the first storage partition comprises the local storage spaces of a plurality of physical storage units, so that the database data belonging to the same table space slice are scattered on different physical storage units, the database data can be distributed in the storage unit group in a partitioning and discrete manner, the physical storage units are prevented from being exclusive, and the storage space utilization rate is effectively improved.

In one embodiment, determining a first set of storage units corresponding to an extended slice identifier includes:

Determining a first storage unit group corresponding to the expansion slice identifier from a candidate storage unit group set corresponding to the memory cluster based on the idle storage capacity corresponding to the candidate storage unit group; the candidate storage unit group set is obtained by grouping each physical storage unit provided by each storage unit in the storage cluster.

Wherein the memory cluster comprises a plurality of memories. The memory in the memory cluster may be provided by a single memory device or may be provided by a plurality of memory devices. A single memory provides multiple physical memory locations. Grouping each physical storage unit provided by each storage unit in the storage cluster to obtain a plurality of candidate storage unit groups, wherein each candidate storage unit group forms a candidate storage unit group set. The candidate memory cell group includes physical memory cells respectively provided by a plurality of memories.

The free storage capacity corresponding to the candidate storage unit group refers to the storage capacity of the remaining unallocated storage space in the candidate storage unit group.

Specifically, the allocation node may obtain a set of candidate storage unit groups corresponding to the memory cluster, and determine, from the set of candidate storage unit groups, a first storage unit group corresponding to the extended slice identifier based on a free storage capacity corresponding to the candidate storage unit group. For example, the candidate memory cell group having the largest free memory capacity may be regarded as the first memory cell group; one candidate memory cell group may be randomly selected from among the candidate memory cell groups having a free memory capacity greater than a preset capacity as the first memory cell group.

In the above embodiment, from the candidate storage unit group set corresponding to the memory cluster, the first storage unit group corresponding to the extended slice identifier is determined based on the free storage capacity corresponding to the candidate storage unit group, so that the candidate storage unit group with a larger free storage capacity can be obtained as the first storage unit group, so as to further improve the storage space utilization rate.

In one embodiment, the database processing method further comprises:

Acquiring a memory cluster; each memory in the memory cluster is provided with a plurality of physical memory units; grouping all memories in the memory cluster to obtain a plurality of memory groups; forming a candidate storage unit group by at least one physical storage unit respectively provided by each memory in the same memory group to obtain a plurality of candidate storage unit groups; different physical storage units are included among the candidate storage unit groups; and obtaining a candidate storage unit group set corresponding to the memory cluster based on each candidate storage unit group.

Wherein the memory group comprises a plurality of memories. The individual memories in a single memory bank may belong to the same memory device or may belong to different memory devices.

Specifically, the allocation node acquires a memory cluster, groups each memory in the memory cluster to obtain a plurality of memory groups, and each memory group comprises the same amount of memory. The allocation node may form the candidate storage unit group from at least one physical storage unit provided by each memory in the same memory group, to obtain a plurality of candidate storage unit groups. For example, one physical memory unit respectively provided by each memory in the same memory group is formed into a candidate memory unit group. It will be appreciated that different physical storage units are included between each candidate storage unit group to facilitate management of the physical storage units. Finally, the distribution node groups each candidate storage unit group into a candidate storage unit group set corresponding to the storage cluster.

In the above embodiment, the memories in the memory cluster are first grouped to obtain a plurality of memory groups, and then physical storage units provided by each memory in the same memory group are grouped to obtain a plurality of candidate storage unit groups, where each candidate storage unit group includes different physical storage units, so that the utilization rate of the physical storage units can be improved when the database data is stored, and the utilization rate of the memories is improved, and a candidate storage unit group set corresponding to the memory cluster is obtained based on each candidate storage unit group.

In one embodiment, the database processing method further comprises:

Acquiring a storage node set; each storage node in the storage node set is respectively provided with a plurality of memories; grouping each storage node in the storage node set based on a preset number of copies to obtain a plurality of storage node groups; taking the memories with matched positions respectively provided by all storage nodes in the same storage node group as a memory copy group to obtain a plurality of memory copy groups; each memory in the same set of memory copies is used to store the same database data; and respectively acquiring a memory from each memory copy group to form a memory cluster.

Wherein the storage node set comprises a plurality of storage nodes. And grouping each storage node in the storage node set to obtain a plurality of storage node groups. A single storage node group includes a plurality of storage nodes. The node number in the storage node group is a preset copy number. A single storage node provides multiple memories. And grouping the memories provided by the storage nodes in the single storage node group to obtain a plurality of memory copy groups. The single set of memory copies includes the location-matched memory provided by each storage node in the single set of storage nodes. The memory with matched positions between different storage nodes refers to the memory with the same slot positions on different nodes. Each memory in the memory copy group stores the same database data, so that the reliability and the availability of the data can be ensured, and when a single memory has a problem, other memories can still be used to ensure that the database is not interrupted due to a single point of failure.

Specifically, the distribution node acquires a storage node set, groups each storage node in the storage node set based on a preset number of copies, and obtains a plurality of storage node groups. Each storage node group comprises the same number of storage nodes, and the number of the storage nodes is the number of copies. For example, the preset number of copies is 3, and one storage node group includes 3 storage nodes. The distribution node takes the memories with matched positions respectively provided by all the storage nodes in the same storage node group as a memory copy group to obtain a plurality of memory copy groups corresponding to the same storage node group. And finally, obtaining a plurality of storage copy groups corresponding to the storage node groups. The distribution node respectively acquires a memory from each memory copy group to form a memory cluster, the acquired memory is used as a main memory, other memories belonging to the same memory copy group as the main memory are used as auxiliary memories, the main memory and the auxiliary memories store the same database data, and when the main memory fails, the auxiliary memories can ensure the continuity and stability of database service.

In one embodiment, when grouping each storage node in the storage node set, storage nodes that are closer may be grouped into storage node groups to improve data storage efficiency.

In one embodiment, when one memory component memory cluster is acquired from each memory copy group, the memory component memory cluster with the lowest historical failure rate may be acquired from each memory copy group, so as to improve data storage stability.

In the above embodiment, each storage node in the storage node set is grouped to obtain a plurality of storage node groups, the memories with matched positions provided by each storage node in the same storage node group are used as the memory copy groups to obtain a plurality of memory copy groups, and each memory in the same memory copy group is used for storing the same database data, so that the reliability and availability of the database can be improved, and even if a certain memory fails or data is lost, the copy memory can still be used.

In one embodiment, at the time of initializing the storage node cluster, the individual physical storage units (which may also be called a small table or a physical small table) provided by the storage node cluster are grouped into 8 groups (8 may be designated at the time of initializing or may be designated as other numbers such as 16), and all the physical storage units are divided into 8 groups of components. Also, in order for 8 physical storage units in a partition group to be concurrent, it is required that the 8 physical storage units within the partition group be distributed across different disks (i.e., memory). For example, a storage node cluster is composed of 12 machines, each machine has 12 disks, each disk has a capacity of 4TB (1800 small tables can be stored), the number of copies is 3, and the 144 disks can be divided into 48 copy groups according to 3 copies. The storage node cluster is divided into 6 partition copy groups according to 8 partition copy groups, each group comprises 8 disks, each disk comprises 1800 small tables, and therefore one storage node cluster comprises 10800 partition groups.

Referring to fig. 3, 12 machines are grouped into 3 machine groups, which can be divided into 4 groups. The disk with the same slot position of three machines in one machine group is used as a copy group, one machine group is provided with 12 copy groups, and all the copy groups are numbered in sequence, namely from 1 to 48. Every 8 replica groups are used as a partition replica group to create a partition group. The disks of each partition copy group are provided with 1800 small tables, the small tables are numbered in sequence, the small table numbers of the first copy group are from 1 to 1800, the small table numbers of the second copy group are 1801 to 3600, and the like, and the small table numbers of the copy group 48 are 47 x 1800+1 to 48 x 1800. Thus, the 8 tabs contained in the first partition group are tab 1 of copy group 1, tab 1801 of copy group 2, tab 3601 of copy group 3, tab 5401 of copy group 4, and so on, with the last tab being tab 12601 of copy group 8. While the small table contained within the second partition group starts from replica group 9 to replica group 16 and is the first numbered small table within each replica group. By analogy, by the time of the seventh partition group, the small table contained within the group is the second numbered small table within replica group 1 through replica group 8. In this way, the allocation node initializes all partition groups when initializing the storage node cluster, and sequentially numbers the partition groups, and when expanding the following table space, the allocation node selects and schedules the partition groups with the largest residual free storage capacity according to the residual free storage capacity of the partition groups.

In one embodiment, determining a first storage partition corresponding to an extended slice identifier from a first storage unit group includes:

Acquiring the identifiers of all the table space blocks corresponding to the extended table space slices; the tablespace block identifier is used for identifying a tablespace block obtained by splitting an extended tablespace slice; taking each physical storage unit in the first storage unit group as a target storage unit respectively corresponding to each table space block identifier in turn according to the sequence; taking the free storage space matched with the block capacity of the table space block in the target storage unit as a local storage space corresponding to the table space block identifier; and obtaining a first storage partition corresponding to the extended slice identifier based on the local storage spaces respectively corresponding to the table space block identifiers.

And splitting the extended tablespace slice to obtain a plurality of tablespace blocks corresponding to the extended tablespace slice. For example, the slice capacity of an extended tablespace slice is 2GB, and the extended tablespace slice is split by 1MB to obtain a plurality of tablespace blocks, and the block capacity of one tablespace block is 1MB. The tablespace block identification is used to identify tablespace blocks. The target storage unit is a physical storage unit corresponding to the table space block identifier in the first storage unit group.

Specifically, after determining the first storage unit group corresponding to the extended tablespace slice, the extended tablespace slice may be mapped to each physical storage unit in the first storage unit group in a scattered manner. The allocation node acquires each tablespace block identifier corresponding to the extended tablespace slice, and takes each physical storage unit in the first storage unit group as a target storage unit respectively corresponding to each tablespace block identifier in turn according to a certain sequence. For example, the first storage unit group includes 8 physical storage units, respectively physical storage units a-H, the extended tablespace slice includes 16 tablespace blocks, respectively tablespace blocks 1-16, physical storage unit a is the target storage unit corresponding to tablespace block 1, physical storage unit B is the target storage unit corresponding to tablespace block 2, physical storage unit C is the target storage unit corresponding to tablespace block 3, and so on, physical storage unit H is the target storage unit corresponding to tablespace block 8, physical storage unit a is the target storage unit corresponding to tablespace block 9, physical storage unit B is the target storage unit corresponding to tablespace block 10, and so on, physical storage unit H is the target storage unit corresponding to tablespace block 16. It is understood that the ordering among the physical storage units in the first storage unit group may be random ordering or may be ordered according to a certain rule.

After determining the target storage unit corresponding to the tablespace block, the allocation node takes the free storage space matched with the block capacity of the tablespace block in the target storage unit as the local storage space corresponding to the tablespace block, namely the local storage space corresponding to the tablespace block identifier. And the distribution node forms the local storage spaces corresponding to the table space block identifiers into a first storage partition corresponding to the expansion slice identifier.

For example, the tablespace is extended at a slice (tablespace slice) granularity. The granularity of slice may be specified at initialization, for example, may be set to 512MB, 1GB, etc. It will be appreciated that slice is smaller than a small table. After the slice allocation maps to the partition group, in order to eliminate the hot spot slice, the data inside the slice is scattered with granularity of 1MB, and the 1MB size is called block. One partition group includes 8 small tables, 8 small tables being located on different disks. Here the scattering is done by a polling algorithm, as shown in fig. 4, the first block of slice maps to table 1 in the partition group, the second block maps to table 2 in the partition group, and so on, the ninth block maps to table 1 in the partition group. This distributes the slice's data across the physical tables in the partition group.

In one embodiment, the physical storage units in the first storage unit group are ordered from large to small according to the free storage capacity, and the physical storage units in the first storage unit group are alternately used as target storage units respectively corresponding to the table space block identifiers according to the ordering result, so that the physical storage units with large space storage capacity are preferentially allocated for the table space slices, and the utilization rate of the physical storage units is further improved.

In the above embodiment, each physical storage unit in the first storage unit group is sequentially and alternately used as the target storage unit corresponding to each table space block identifier, so that the utilization rate of the physical storage units can be improved. The same physical storage unit can be used for a plurality of databases or a plurality of table space slices, the utilization rate of the physical storage unit can be further improved, the free storage space matched with the block capacity of the table space block in the target storage unit is used as the local storage space corresponding to the table space block identifier, and the first storage partition corresponding to the extended slice identifier is obtained based on the local storage space respectively corresponding to each table space block identifier.

In one embodiment, returning physical storage mapping information to a requestor corresponding to a tablespace expansion request, such that the requestor stores database data associated with an expanded tablespace slice to a first storage partition in a first set of storage locations based on the physical storage mapping information, comprises:

and returning physical storage mapping information to a requester corresponding to the tablespace expansion request, so that the requester stores database data associated with the tablespace blocks of the expanded tablespace slice into corresponding local storage spaces in a first storage partition in the first storage unit group based on the physical storage mapping information.

Specifically, the physical storage mapping information includes a first storage unit group corresponding to the extended tablespace slice and related information of a first storage partition, and the first storage partition includes local storage spaces respectively corresponding to respective tablespace blocks of the extended tablespace slice. The requester can acquire the position information of the local storage spaces corresponding to the table space blocks of the extended table space slice based on the physical storage mapping information, and further can store the database data associated with the table space blocks into the corresponding local storage spaces in the first storage partition in the first storage unit group.

In the above embodiment, the first storage partition includes the corresponding local storage spaces of each tablespace block of the extended tablespace slice in the corresponding target storage unit, and stores the database data belonging to the tablespace block into the corresponding local storage space, so that the database data belonging to the extended tablespace slice can be distributed on each physical storage unit in the first storage unit group in a dispersed manner, and the utilization rate of the physical storage units is effectively improved.

In one embodiment, the database processing method further comprises:

Acquiring a data searching request aiming at a target database and sent by a requester; the data search request carries a target slice identifier corresponding to the target table space slice; inquiring a second storage unit group and a second storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier; the target storage mapping information is returned to the requestor such that the requestor looks up database data associated with the target tablespace slice from the second storage partition in the second storage unit group based on the target storage mapping information.

The data searching request is used for requesting physical storage mapping information corresponding to the table space slice to which the database data to be searched belongs. The data search request carries a target slice identifier corresponding to the target table space slice. The target tablespace slice refers to the tablespace slice to which the database data to be looked up belongs. The target slice identifier refers to a slice identifier corresponding to a target table space slice. The target storage mapping information refers to physical storage mapping information corresponding to the target table space slice. The second set of memory locations is a set of memory locations allocated to the target tablespace slice. The second memory partition is the physical memory space allocated to the target tablespace slice in the second memory cell group.

Specifically, all database data of the target database are stored in physical storage units of a plurality of memories in a scattered manner, when the database data needs to be searched, a requester of the tablespace expansion request needs to acquire physical storage mapping information corresponding to tablespace slices to which the database data to be searched belongs, and therefore the requester can send a data search request aiming at the target database to an allocation node, and the data search request carries a target slice identifier corresponding to the tablespace slices to which the database data to be searched belongs. The allocation node may query, based on the target slice identifier, a second storage unit group and a second storage partition allocated for the target table space slice, to obtain target storage mapping information corresponding to the target slice identifier. The distribution node returns target storage mapping information to the requestor. The requester searches database data associated with the target tablespace slice from the second storage partition in the second storage unit group based on the target storage mapping information, and acquires the required database data from the database data associated with the target tablespace slice.

In the above embodiment, when the database data in the target tablespace slice needs to be searched, the allocation node queries the target storage mapping information corresponding to the target tablespace slice, and based on the target storage mapping information, obtains the physical storage space corresponding to the target tablespace slice, and can quickly find the required database data from the physical storage space corresponding to the target tablespace slice.

In a specific embodiment, referring to fig. 5, in the conventional art, the tablespace of the database is directly mapped to a physical small tablespace, typically 2GB, and the database can only be expanded in 2GB of space. In the case of a smaller database instance, such as a maximum of 10GB in a table space, the database instance may be divided into 5 small tables according to the small table division of 2GB, and the storage cluster may generally include a large number of disks, which may cause that most of the disks are idle, data of the database is only distributed on a small part of the disks of the storage cluster, and read-write traffic is only on these disks, so that poor performance of the small database instance is easy to occur. In addition, in the scene of sequential data filling of the database, one 2GB needs to be written out and the next 2GB is written, and when one 2GB is written, all flow of the database instance is on a disk corresponding to the 2GB, so that the phenomenon of hotspots of the database instance small table easily occurs. In addition, the table space is expanded according to the physical small table 2GB of the storage cluster, namely the physical small table 2GB space is exclusive, and when the use space is less than 2GB, for example, only a few bytes are used in extreme cases, the rest space of the physical small table cannot be used by other table spaces, and the problem of storage space waste exists.

However, referring to FIG. 5, in the method of the present application, the tablespace is extended at the slice granularity, mapping slices to local storage space in the physically small tablespace. The granularity of slice may be specified at the time of storage node cluster initialization, e.g., may be set to 512MB, 1GB, etc., and slice is smaller than the physical small table. The table space expansion at slice granularity may be more flexible because the granularity is small.

Multiple database instances may be created based on one distributed storage cluster, e.g., a cloud service provider may provide cloud database services to the outside based on the distributed storage cluster, and multiple users may build their own cloud databases using the cloud database services. The distribution node (which may be referred to as dbmaster) globally controls the mapping of slice when the table space expands. To break up the slice space into multiple physical small table spaces, a partition group is formed by 8 physical small tables, and is a broken management unit. To map slice data onto physical tables within a partition group requires two layers of mapping: 1. mapping from slice to partition group; 2. mapping of slice data blocks to physical wavelets

1. Slice to partition group mapping

When partition mapping is needed for a slice, a partition group with the largest residual capacity is selected from all partition groups (mainly for balancing the space utilization rate of all small tables as much as possible, so that the data rate of a disk is balanced as much as possible), wherein the partition number of the partition group is related to the slice size, and if the slice size is 1GB, one partition group is 8 physical small tables, each small table can occupy one partition with 128MB data of the slice, and the partition group can have 16 partitions (2 GB/128 MB).

When there is a database instance to expand the table space, such as database instance 1 expansion slice a and database instance 2 expansion slice B in fig. 6, a request is sent to dbmaster to find the partition group with the most remaining capacity. dbmaster, after obtaining the partition group, map the slice to be extended to the partition group (where mapping is recording a correspondence), and allocate an idle partition to the slice, for example, in fig. 6, idle partition 1 in the partition group is allocated to slice a of database instance 1, and idle partition 2 is allocated to slice B of database instance 2. It can also be seen here that the physical small tablespace is not exclusive and can be allocated to tablespaces of different database instances, thereby making more efficient use of physical storage space.

2. Mapping of slice data blocks to physical wavelets

To avoid hot-spot slices, slice space needs to be broken up. And scattering the data in the slice onto each physical small table in the partition group through a certain algorithm, wherein the data which belong to the same slice on each physical small table are scattered to form a partition. For example, referring to fig. 4, the break up may be performed by a polling algorithm.

The application provides a partition balanced database table space expansion method, wherein the table space of a database is not directly mapped to a physical small table space any more, a logic mapping layer slice is added, so that the table space is firstly mapped to the slice, then mapped to the physical small table space from the slice, and the data of the table space are subjected to partition discrete distribution, so that the data belonging to the same logic table space are scattered on different physical small table spaces, and the expansion and the distribution of the database data in a distributed block storage system are better.

In one embodiment, as shown in fig. 7, a database processing method is provided, and an example of application of the method to the computing node in fig. 1 is described. Wherein:

step S702, obtaining an expansion table space slice of a target database; the tablespace of the target database is extended in tablespace slices.

Step S704, a tablespace expansion request carrying an expansion slice identifier corresponding to the expansion tablespace slice is generated.

The target database refers to a database to be processed. The tablespace of the target database is extended in tablespace slices. Extended tablespace slices refer to blocks of tablespace that need to be extended. The tablespace expansion request is used for requesting physical storage mapping information corresponding to the tablespace blocks to be expanded.

Specifically, when the target database needs to expand the tablespace, the computing node determines an expansion tablespace slice of the target database and an expansion slice identifier corresponding to the expansion tablespace slice, generates a tablespace expansion request carrying the expansion slice identifier, and sends the tablespace expansion request to the allocation node, so that the allocation node allocates corresponding storage resources for the expansion tablespace slice, and the computing node can utilize the allocated storage resources to store database data associated with the expansion tablespace slice.

Step S706, a tablespace expansion request is sent to an allocation node; the allocation node is used for determining a first storage unit group corresponding to the expansion slice identifier, determining a first storage partition corresponding to the expansion slice identifier from the first storage unit group, and generating physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group corresponding to the expansion slice identifier and the first storage partition; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group.

Wherein the first set of memory cells is a set of memory cells allocated to an extended tablespace slice. The first memory partition is the physical memory space allocated to the extended tablespace slice in the first group of memory cells. The first storage partition comprises local storage spaces corresponding to all the physical storage units in the first storage unit group, wherein the local storage spaces corresponding to the physical storage units refer to part of the storage spaces of the physical storage units, namely, part of the storage spaces are acquired from all the physical storage units in the first storage unit group to form the first storage partition.

Specifically, the computing node sends a tablespace expansion request to the allocation node, the allocation node allocates storage unit groups and storage partitions for the expanded tablespace slices, determines a first storage unit group corresponding to the expanded slice identifiers, and determines a first storage partition corresponding to the expanded slice identifiers from the first storage unit group. The distribution node generates physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group and the first storage partition, and returns the physical storage mapping information corresponding to the expansion slice identifier to the calculation node.

It will be appreciated that the data processing procedure of the allocation node may refer to the content of the foregoing embodiments, and will not be described herein.

Step S708, based on the physical storage mapping information fed back by the allocation node, stores the database data associated with the extended tablespace slice to the first storage partition in the first storage unit group.

Specifically, the computing node may determine, based on the physical storage mapping information, a first storage unit group and a first storage partition corresponding to the extended tablespace slice, and store database data associated with the extended tablespace slice to the first storage partition in the first storage unit group.

According to the database processing method, when the table space of the database is expanded, the table space slice is used as a unit for expanding, the expanded table space slice is mapped to the first storage unit group, the expanded table space slice is further mapped to the first storage partition in the first storage unit group, the related database data of the expanded table space slice is used for the first storage partition stored in the first storage unit group, and the first storage partition comprises the local storage spaces of the physical storage units, so that the database data belonging to the same table space slice are scattered on different physical storage units, the database data can be distributed in the storage unit group in a partition discrete mode, the physical storage units are prevented from being exclusive, and the storage space utilization rate is effectively improved.

In one embodiment, obtaining an extended tablespace slice of a target database includes:

Acquiring database data to be written into a target database; and determining the extended tablespace slice of the target database based on the data amount of the database data to be written into the target database and the slice capacity of the tablespace slice corresponding to the target database.

Wherein the database data to be written into the target database includes at least one of data table data (page data) and log data. The data table data is data recorded in a data table in a database. The log data is used to record various operations and transaction information that occur in the database. The data table data is data actually stored in the database, and the log data is log information for ensuring database persistence and consistency.

In particular, the compute node may determine the tablespace slice that needs to be expanded based on the amount of data that needs to be written to the database. The computing node acquires database data to be written into the target database, and determines an extended tablespace slice of the target database based on the data amount of the database data to be written into the target database and the slice capacity of the tablespace slice corresponding to the target database. For example, if an existing tablespace slice is already filled with data, then the amount of database data to be written to the target database may be divided by the slice capacity of the tablespace slice corresponding to the target database to obtain the spatial extent that requires several expanded tablespace slices and each expanded tablespace slice. If the existing tablespace slice is not filled with data, a part of database data to be written into the target database can be written into a storage partition corresponding to the existing tablespace slice, and then the space range which needs a plurality of expansion tablespace slices and corresponds to each expansion tablespace slice can be obtained based on the data amount of the database data of the residual target database to be written into the tablespace slice corresponding to the target database.

It will be appreciated that taking a size of 2GB as an example, a first tablespace slice of the database has a spatial range of 0-2 GB, a second tablespace slice has a spatial range of 2-4 GB, a third tablespace slice has a spatial range of 4-6 GB, and so on.

In the above embodiment, the extended tablespace slice of the target database is determined based on the data amount of the database data to be written into the target database and the slice capacity of the tablespace slice corresponding to the target database, so that the tablespace can be extended according to the granularity of the tablespace slice when the database data is gradually increased, and the tablespace can be extended as required, thereby avoiding the waste of storage resources.

In one embodiment, the first memory partition includes local memory spaces respectively corresponding to respective tablespace block identifiers corresponding to the extended tablespace slices, the tablespace block identifiers identifying tablespace blocks resulting from splitting the extended tablespace slices.

Storing database data associated with the extended tablespace slice to a first storage partition in a first storage unit group based on physical storage mapping information fed back by the allocation node, comprising:

And storing the database data associated with the tablespace blocks of the expanded tablespace slice into the corresponding local storage space in the first storage partition in the first storage unit group based on the physical storage mapping information fed back by the allocation node.

And splitting the extended tablespace slice to obtain a plurality of tablespace blocks corresponding to the extended tablespace slice. The tablespace block identification is used to identify tablespace blocks. The target storage unit is a physical storage unit corresponding to the table space block identifier in the first storage unit group.

Specifically, after determining the first storage unit group corresponding to the extended tablespace slice, the allocation node may scatter-map the extended tablespace slice onto each physical storage unit in the first storage unit group. The allocation node acquires each tablespace block identifier corresponding to the extended tablespace slice, takes each physical storage unit in the first storage unit group as a target storage unit corresponding to each tablespace block identifier in turn according to a certain sequence, takes the free storage space matched with the block capacity of the tablespace block in the target storage unit as a local storage space corresponding to the tablespace block identifier, and forms a first storage partition corresponding to the extended slice identifier by the local storage space corresponding to each tablespace block identifier.

The computing node acquires the physical storage mapping information returned by the distribution node, determines a first storage unit group and a first storage partition corresponding to the expansion table space slice based on the physical storage mapping information, and determines the corresponding local storage space of each table space block of the expansion table space slice in the first storage partition based on the first storage partition. The computing node stores database data associated with the tablespace blocks of the expanded tablespace slice to corresponding local storage space within a first storage partition in the first group of storage units.

In the above embodiment, the expanded tablespace slice is scattered and mapped to each physical storage unit in the first storage unit group with the tablespace block as granularity, and when writing data to the database, the database data belonging to the tablespace block is stored in the corresponding local storage space in the corresponding physical storage unit in the first storage unit group, and the database data is scattered and stored on each physical storage unit, so that the utilization rate of the physical storage units can be effectively improved.

In one embodiment, as shown in fig. 8, a database processing method is provided, and an example of application of the method to the computing node in fig. 1 is described. Wherein:

step S802, acquiring a database operation request aiming at a target database; the database operation request carries a data page identifier corresponding to a target data page to which the operation data belongs.

Step S804, based on the data page identification, determines the target tablespace slice to which the target data page belongs in the tablespace of the target database.

Wherein the database operation request is for requesting an operation on data in the database. For example, the operations may be add, delete, modify, search, etc. In one embodiment, database operation requests are divided into page requests and log requests. The page request is used to find page data. The log request is used to find log data.

The operation data refers to database data to be operated. When a line is required to be read from the data table, the data page where the line is located is found first, then the physical storage space where the data page is located is found, and data is read from the corresponding physical storage space. A data page (page) is a basic unit of data storage in a database, typically 16KB in size. The data page identification is used to identify the data page, for example, a data page number may be used as the data page identification. The target data page is a data page to which database data to be operated belongs.

In one embodiment, the page data consists of 16KB units of data pages, each data page having a globally unique number (called page number). log data records modifications made to database data pages by database transaction operations. Page data and log data corresponding to the same data page are stored in the same physical small table.

Specifically, when a user operates a target database, the computing node generates a database operation request for the target database, where the database operation request carries a data page identifier corresponding to a target data page to which operation data belongs. The computing node determines, based on the data page identification, a target tablespace slice to which the target data page belongs in the tablespace of the target database. It will be appreciated that the size of the tablespace is known, the size of the data pages is known, the ordering among the data pages is such that the location of the data pages in the tablespace is determinable, the size of the tablespace is known, the size of the tablespace slices is such that the location of the tablespace slices in the tablespace is determinable, and the combination of the locations of the data pages in the tablespace and the locations of the tablespace slices in the tablespace can determine the target tablespace slice to which the target data pages belong in the tablespace of the target database.

Step S806, a data searching request carrying a target slice identifier corresponding to the target table space slice is generated, and the data searching request is sent to the distribution node; the distribution node is used for inquiring the target storage unit group and the target storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier; the target storage unit group comprises physical storage units respectively provided by a plurality of memories, and the target storage partition comprises local storage spaces respectively corresponding to the physical storage units in the target storage unit group.

Step S808, based on the target storage mapping information fed back by the allocation node, searches the operation data located in the target data page from the target storage partition in the target storage unit group.

Wherein the target set of storage locations is a set of storage locations allocated to the target tablespace slice. The target storage partition is the physical storage space allocated to the target tablespace slice in the target storage unit group. The target storage partition comprises local storage spaces corresponding to all physical storage units in the target storage unit group respectively, wherein the local storage spaces corresponding to the physical storage units refer to part of storage spaces of the physical storage units, namely, part of storage spaces are acquired from all the physical storage units in the target storage unit group respectively to form the target storage partition. .

Specifically, the computing node generates a data search request carrying a target slice identifier corresponding to the target tablespace slice, and sends the data search request to the allocation node, so that the allocation node queries the storage resources allocated for the target tablespace slice, and the computing node can search the operation data from the corresponding storage resources. The distribution node inquires the target storage unit group and the target storage partition corresponding to the target slice identifier, obtains target storage mapping information corresponding to the target slice identifier, and returns the target storage mapping information corresponding to the target slice identifier to the calculation node. For example, the tablespace of the target database includes 3 tablespace slices, the allocation node has allocated respective sets of storage units and storage partitions for each tablespace slice, and if the target tablespace slice is the 2 nd tablespace slice, the allocation node queries the set of storage units and storage partitions allocated for the 2 nd tablespace slice as the target set of storage units and the target storage partition.

The compute node searches for operational data located in the target data page from the target storage partition in the target storage unit group based on the target storage mapping information.

It may be appreciated that the data processing procedure of allocating storage resources for the target tablespace slice by the allocation node may refer to the content of the foregoing embodiments, which is not described herein.

In one embodiment, the computing node does not need to send the page data to the storage node for storage, the computing node selects to send the log data to the storage node, and the storage node performs log playback based on the log data to obtain the page data, so that the page data is stored. For example, the first log data is the original page data, the target log data is recorded with the modified content of the page data, and the modified content of the target log data is applied to the original page data to obtain new page data.

Step S810, processing the searched operation data to obtain a processing result corresponding to the database operation request.

Specifically, after the computing node finds the operation data located in the target data page from the target storage partition in the target storage unit group, the computing node can process the found operation data to obtain a processing result corresponding to the database operation request. It will be appreciated that the specific manner of processing may be determined according to actual needs. For example, if the database operation request is a data search request, the searched operation data may be used as a data query result, and the data query result may be returned to the requester; if the database operation request is a data modification request, the searched operation data can be modified to obtain a data modification result, the data modification result is returned to the requester, if the database operation request carries a modification scheme, the searched operation data can be modified based on the modification scheme to obtain the data modification result, and the data modification result can be used for indicating whether the modification is successful or not.

According to the database processing method, the related database data of the target tablespace slice are used for the target storage partition stored in the target storage unit group, and the target storage partition comprises the local storage spaces of the physical storage units, so that the database data belonging to the same tablespace slice are scattered on different physical storage units, the database data can be distributed in the storage unit group in a partitioned and discrete manner, the physical storage units are prevented from being monopolized, and the storage space utilization rate is effectively improved. The computing node can accurately search the required operation data from the corresponding physical storage control based on the target storage mapping information returned by the distribution node.

In one embodiment, determining, based on the data page identification, a target tablespace slice to which the target data page belongs in a tablespace of the target database, includes:

Based on the data page identification, determining a corresponding space address of the target data page in the tablespace of the target database, and based on the space address and the slice capacity of the tablespace slice of the target database, determining the target tablespace slice to which the target data page belongs in the tablespace.

The corresponding space address of the target data page in the table space of the target database refers to the position of the target data page in the table space of the target database. For example, the space range of the table space is 0-4T, and the target data page is 1 GB-1 GB+16 KB in the table space. The tablespace of the target database is extended in tablespace slices.

In particular, the size of the tablespace is known, the size of the data pages is known, and the data pages are ordered between each other, so that the computing node can determine the corresponding spatial address of the target data page in the tablespace of the target database based on the data page identification of the target data page. For example, the space range of the table space is 0-4 t, the size of the data page is 16KB, and if the target data page is the first data page, the space address corresponding to the target data page is 0-16 KB. The compute node further determines a target tablespace slice to which the target data page belongs in the tablespace of the target database based on the space address and the slice capacity of the tablespace slice of the target database. For example, the slice capacity of the tablespace slice is 2GB, and the target data page is 1 GB-1 GB+16 KB in the tablespace, then the target tablespace slice to which the target data page belongs is the first tablespace slice of the tablespace.

In the above embodiment, the space address corresponding to the target data page in the tablespace of the target database is determined based on the data page identification, and the target tablespace slice to which the target data page belongs in the tablespace can be quickly determined based on the space address and the slice capacity of the tablespace slice of the target database.

In one embodiment, as shown in fig. 9, generating a data search request carrying a target slice identifier corresponding to a target tablespace slice, and sending the data search request to an allocation node includes:

step S902, determining a target tablespace block to which the target data page belongs in the target tablespace slice based on the data page identification.

Step S904, generating a data search request carrying a target slice identifier corresponding to a target tablespace slice and a target space block identifier corresponding to a target tablespace block, and sending the data search request to an allocation node; the allocation node is used for inquiring the target storage unit group and the target storage partition corresponding to the target slice identifier, inquiring the local storage space corresponding to the target space block identifier in the target storage partition, and obtaining target storage mapping information corresponding to the target space block identifier.

Searching operation data in a target data page from a target storage partition in a target storage unit group based on target storage mapping information fed back by an allocation node, wherein the method comprises the following steps:

Step S906, based on the target storage mapping information fed back by the allocation node, searches the operation data located in the target data page from the corresponding local storage space in the target storage partition in the target storage unit group.

Wherein the target tablespace block is the tablespace block to which the target data page belongs in the target tablespace slice. For example, the size of the tablespace block is 1M, the target tablespace slice is 0-2 GB in the tablespace, the target data page is 1 GB-1 GB+16 KB in the tablespace, and then the target tablespace block to which the target data page belongs in the target tablespace slice is the 1085 th tablespace block in the target tablespace slice.

Specifically, the tablespace slice is scattered and mapped into corresponding storage unit groups by taking the tablespace block as a unit. To further narrow the lookup range, the compute node may determine a target tablespace block to which the target data page belongs in the target tablespace slice based on the data page identification. For example, a target tablespace block to which the target data page belongs in the target tablespace slice is determined based on a spatial address corresponding to the target data page in the tablespace of the target database. The computing node generates a data searching request carrying a target slice identifier corresponding to the target tablespace slice and a target space block identifier corresponding to the target tablespace block, and sends the data searching request to the distribution node. And inquiring the physical storage mapping information corresponding to the target table space block from the distribution node.

The allocation node inquires a target storage unit group and a target storage partition corresponding to the target slice identifier, inquires a local storage space corresponding to the target space block identifier in the target storage partition, and obtains target storage mapping information corresponding to the target space block identifier. And the distribution node returns target storage mapping information corresponding to the target space block identification to the calculation node.

The computing node searches operation data in a target data page from corresponding local storage spaces in a target storage partition in a target storage unit group based on target storage mapping information corresponding to the target space block identification.

In the above embodiment, the computing node may further determine a target tablespace block corresponding to the target data page to which the operation data belongs in the target tablespace slice to which the operation data belongs, so as to further narrow the search range, generate a data search request carrying a target slice identifier corresponding to the target tablespace slice and a target space block identifier corresponding to the target tablespace block, send the data search request to the allocation node to query the target storage mapping information corresponding to the target space block identifier, and the computing node may quickly find the operation data from the physical storage space with a smaller range based on the target storage mapping information returned by the allocation node.

In a specific embodiment, the data in the table space is partitioned, and then multiple levels of mapping transformation are needed when the data distribution searching is performed. Referring to fig. 10, a total of 5 levels of lookups are passed. Each page is 16KB, the page request carries a page number, and the linear address of the table space where the page corresponding to the page number is located can be calculated through the page number. And calculating which slice the page is located in according to the slice size and the page linear address, and belonging to which block in the slice. And (3) finding out the partition group to which the slice belongs through dbmaster, and calculating which small table in the partition group to which the block belongs according to a polling data scattering algorithm. The duplicate disk set may be found from the small table number. In the same small table where the page data and the corresponding log data of one page are located, both the page request and the log request have the page number, so the log data can also use the multiple stages to search the data.

In a specific embodiment, the method of the application may be applied to a pool of isolated mysql databases. Referring to FIG. 11, to separate computation and storage, typically using distributed storage as the data base, the compute node is responsible for the compute portion of the mysql database and the storage node is responsible for the storage portion of the mysql database. In a scenario where distributed block storage is used, a storage node cluster includes multiple storage nodes, one storage node includes multiple disks, and one disk includes multiple physical storage units (i.e., small tables, physical small tables).

Instead of mapping the tablespace of the mysql database directly to a physically small tablespace, a logical mapping layer is added to divide the tablespace into individual slice granularities, and the granularity of a slice can be specified at the time of initialization, for example, can be set to 512MB, 1GB, and the like. Every 8 physical small tables in the distributed block storage cluster form a partition group for storing slice data, and the 8 physical small tables in the partition group are located on different magnetic disks. Each slice data is mapped on the small tables in the partition group according to a scattering algorithm, and the scattered data belonging to the same slice on each small table form a partition. Each small table in the partition group corresponds to three disks of 2GB of physical space, one of the three disks is a disk, and the other is a slave disk. The tablespace is typically extended up to 128T.

When writing data to the mysql database, the computing node sends a tablespace expansion request to the distribution node through the block storage client, and the distribution node distributes a corresponding partition group and a partition corresponding to the partition group for the slice needing to be expanded. And the distribution node returns the distributed mapping information to the calculation node, and the calculation node writes data to the corresponding partition in the corresponding partition group in the block storage server based on the mapping information. When data is read from the mysql database, the computing node inquires the partition group and the partition to which the data to be read belongs from the distribution node through the block storage client, and reads the data from the corresponding partition in the corresponding partition group in the block storage server based on the information returned by the distribution node.

According to the method, the logical mapping layer is added between the database table space and the small table of the distributed block storage cluster, and meanwhile, the table space data are subjected to partition discrete distribution, so that the data belonging to the same logical mapping space are distributed on different physical small table spaces, and the mysql data are expanded and distributed better in the distributed block storage system.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a database processing device for realizing the above related database processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the database processing device provided below may refer to the limitation of the database processing method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 12, there is provided a database processing apparatus 1200 comprising: a request acquisition module 1202, a storage location determination module 1204, a mapping information generation module 1206, and a mapping information transmission module 1208, wherein:

A request acquisition module 1202 for acquiring a tablespace expansion request for a target database; the tablespace expansion request carries an expansion slice identifier corresponding to the expansion tablespace slice; the tablespace of the target database is extended in tablespace slices.

The storage location determining module 1204 is configured to determine a first storage unit group corresponding to the extended slice identifier, and determine a first storage partition corresponding to the extended slice identifier from the first storage unit group; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group.

The mapping information generating module 1206 is configured to generate physical storage mapping information corresponding to the extended slice identifier based on the first storage unit group and the first storage partition.

The mapping information sending module 1208 is configured to return physical storage mapping information to a requester corresponding to the tablespace expansion request, so that the requester stores database data associated with the expanded tablespace slice to the first storage partition in the first storage unit group based on the physical storage mapping information.

In one embodiment, the storage location determination module 1204 is further to:

Acquiring a memory cluster; each memory in the memory cluster is provided with a plurality of physical memory units;

grouping all memories in the memory cluster to obtain a plurality of memory groups;

forming a candidate storage unit group by at least one physical storage unit respectively provided by each memory in the same memory group to obtain a plurality of candidate storage unit groups; different physical storage units are included among the candidate storage unit groups;

and obtaining a candidate storage unit group set corresponding to the memory cluster based on each candidate storage unit group.

Acquiring a storage node set; each storage node in the storage node set is respectively provided with a plurality of memories;

grouping each storage node in the storage node set based on a preset number of copies to obtain a plurality of storage node groups;

Taking the memories with matched positions respectively provided by all storage nodes in the same storage node group as a memory copy group to obtain a plurality of memory copy groups; each memory in the same set of memory copies is used to store the same database data;

and respectively acquiring a memory from each memory copy group to form a memory cluster.

Acquiring the identifiers of all the table space blocks corresponding to the extended table space slices; the tablespace block identifier is used for identifying a tablespace block obtained by splitting an extended tablespace slice;

Taking each physical storage unit in the first storage unit group as a target storage unit respectively corresponding to each table space block identifier in turn according to the sequence;

Taking the free storage space matched with the block capacity of the table space block in the target storage unit as a local storage space corresponding to the table space block identifier;

And obtaining a first storage partition corresponding to the extended slice identifier based on the local storage spaces respectively corresponding to the table space block identifiers.

In one embodiment, the mapping information sending module 1208 is further configured to:

In one embodiment, the database processing apparatus 1200 is further configured to:

Acquiring a data searching request aiming at a target database and sent by a requester; the data search request carries a target slice identifier corresponding to the target table space slice;

Inquiring a second storage unit group and a second storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier;

the target storage mapping information is returned to the requestor such that the requestor looks up database data associated with the target tablespace slice from the second storage partition in the second storage unit group based on the target storage mapping information.

In one embodiment, as shown in fig. 13, there is provided a database processing apparatus 1300 comprising: m module, N module and L module, wherein:

A slice acquisition module 1302, configured to acquire an extended table space slice of the target database; the tablespace of the target database is extended in tablespace slices.

The request generation module 1304 is configured to generate a tablespace expansion request that carries an expansion slice identifier corresponding to an expansion tablespace slice.

A request sending module 1306, configured to send a tablespace expansion request to an allocation node; the allocation node is used for determining a first storage unit group corresponding to the expansion slice identifier, determining a first storage partition corresponding to the expansion slice identifier from the first storage unit group, and generating physical storage mapping information corresponding to the expansion slice identifier based on the first storage unit group corresponding to the expansion slice identifier and the first storage partition; the first storage unit group comprises physical storage units respectively provided by a plurality of memories, and the first storage partition comprises local storage spaces respectively corresponding to the physical storage units in the first storage unit group.

The data storage module 1308 is configured to store database data associated with the extended tablespace slice to the first storage partition in the first storage unit group based on the physical storage mapping information fed back by the allocation node.

In one embodiment, the slice acquisition module 1302 is further configured to:

Acquiring database data to be written into a target database;

and determining the extended tablespace slice of the target database based on the data amount of the database data to be written into the target database and the slice capacity of the tablespace slice corresponding to the target database.

In one embodiment, the first memory partition includes local memory spaces respectively corresponding to respective tablespace block identifiers corresponding to the extended tablespace slices, the tablespace block identifiers identifying tablespace blocks resulting from splitting the extended tablespace slices. The data storage module 1308 is further configured to:

In one embodiment, as shown in FIG. 14, there is provided a database processing apparatus 1400 comprising: a database operation request acquisition module 1402, a tablespace data determination module 1404, a data lookup request transmission module 1406, an operation data lookup module 1408, and a processing result determination module 1410, wherein:

A database operation request acquisition module 1402 configured to acquire a database operation request for a target database; the database operation request carries a data page identifier corresponding to a target data page to which the operation data belongs.

The tablespace data determination module 1404 is configured to determine, based on the data page identification, a target tablespace slice to which the target data page belongs in the tablespace of the target database.

A data search request sending module 1406, configured to generate a data search request carrying a target slice identifier corresponding to a target table space slice, and send the data search request to an allocation node; the distribution node is used for inquiring the target storage unit group and the target storage partition corresponding to the target slice identifier to obtain target storage mapping information corresponding to the target slice identifier; the target storage unit group comprises physical storage units respectively provided by a plurality of memories, and the target storage partition comprises local storage spaces respectively corresponding to the physical storage units in the target storage unit group.

An operation data searching module 1408 is configured to search operation data located in a target data page from a target storage partition in the target storage unit group based on the target storage mapping information fed back by the allocation node.

The processing result determining module 1410 is configured to process the searched operation data to obtain a processing result corresponding to the database operation request.

In one embodiment, the tablespace data determination module 1404 is also to:

In one embodiment, the data lookup request sending module 1406 is further to:

determining a target tablespace block to which a target data page belongs in a target tablespace slice based on the data page identification;

Generating a data search request carrying a target slice identifier corresponding to the target table space slice and a target space block identifier corresponding to the target table space block, and sending the data search request to the allocation node; the allocation node is used for inquiring the target storage unit group and the target storage partition corresponding to the target slice identifier, inquiring the local storage space corresponding to the target space block identifier in the target storage partition, and obtaining target storage mapping information corresponding to the target space block identifier.

The operational data lookup module 1408 is also for:

and searching operation data in the target data page from the corresponding local storage space in the target storage partition in the target storage unit group based on the target storage mapping information fed back by the allocation node.

According to the database processing device, when the table space of the database is expanded, the table space slices are used as units for expanding, the expanded table space slices are mapped to the first storage unit group, the expanded table space slices are further mapped to the first storage partition in the first storage unit group, the related database data of the expanded table space slices are used for the first storage partition stored in the first storage unit group, and the first storage partition comprises the local storage spaces of the physical storage units, so that the database data belonging to the same table space slice are scattered on different physical storage units, the database data can be distributed in the storage unit group in a partition discrete mode, the physical storage units are prevented from being exclusive, and the storage space utilization rate is effectively improved.

The various modules in the database processing apparatus described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 15. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data such as physical storage mapping information corresponding to each tablespace slice. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a database processing method.

In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 16. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a database processing method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by persons skilled in the art that the structures shown in fig. 15 and 16 are merely block diagrams of portions of structures associated with aspects of the present application and are not intended to limit the computer apparatus to which aspects of the present application may be applied, and that a particular computer apparatus may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A database processing method, the method comprising:

2. The method of claim 1, wherein the determining the first set of storage units to which the extended slice identifier corresponds comprises:

3. The method according to claim 2, wherein the method further comprises:

Acquiring a memory cluster; each memory in the memory cluster is respectively provided with a plurality of physical memory units;

grouping each memory in the memory cluster to obtain a plurality of memory groups;

And obtaining a candidate storage unit group set corresponding to the memory cluster based on the candidate storage unit groups.

4. The method according to claim 2, wherein the method further comprises:

5. The method of claim 1, wherein the determining, from the first set of storage units, a first storage partition to which the extended slice identifier corresponds comprises:

Acquiring each tablespace block identifier corresponding to the extended tablespace slice; the tablespace block identifier is used for identifying a tablespace block obtained by splitting the extended tablespace slice;

6. The method of claim 5, wherein the returning the physical storage mapping information to the requestor corresponding to the tablespace expansion request causes the requestor to store database data associated with the expanded tablespace slice to the first storage partition in the first group of storage locations based on the physical storage mapping information, comprising:

And returning the physical storage mapping information to a requester corresponding to the tablespace expansion request, so that the requester stores database data associated with the tablespace blocks of the expanded tablespace slice into the corresponding local storage space in the first storage partition in the first storage unit group based on the physical storage mapping information.

7. The method according to any one of claims 1 to 6, further comprising:

acquiring a data searching request aiming at the target database and sent by the requester; the data search request carries a target slice identifier corresponding to the target table space slice;

Returning the target storage mapping information to the requestor to cause the requestor to lookup database data associated with the target tablespace slice from the second storage partition in the second storage unit group based on the target storage mapping information.

8. A database processing method, the method comprising:

9. The method of claim 8, wherein the obtaining an extended tablespace slice of the target database comprises:

Acquiring database data to be written into a target database;

and determining the expansion tablespace slice of the target database based on the data volume of the database data to be written into the target database and the slice capacity of the tablespace slice corresponding to the target database.

10. The method of claim 8, wherein the first storage partition includes local storage spaces respectively corresponding to respective tablespace block identifiers corresponding to the extended tablespace slices, the tablespace block identifiers identifying tablespace blocks resulting from splitting the extended tablespace slices;

the storing, based on the physical storage mapping information fed back by the allocation node, database data associated with the extended tablespace slice into the first storage partition in the first storage unit group includes:

And based on the physical storage mapping information fed back by the allocation node, storing database data associated with the tablespace blocks of the extended tablespace slice into corresponding local storage spaces in the first storage partition in the first storage unit group.

11. A database processing method, the method comprising:

Searching operation data in the target data page from the target storage partition in the target storage unit group based on the target storage mapping information fed back by the distribution node;

And processing the searched operation data to obtain a processing result corresponding to the database operation request.

12. The method of claim 11, wherein the determining, based on the data page identification, a target tablespace slice to which the target data page belongs in a tablespace of the target database comprises:

And determining a space address corresponding to the target data page in a table space of the target database based on the data page identification, and determining a target table space slice of the target data page in the table space based on the space address and the slice capacity of the table space slice of the target database.

13. The method of claim 11, wherein generating a data lookup request carrying a target slice identifier corresponding to the target tablespace slice, and sending the data lookup request to an allocation node comprises:

Determining a target tablespace block to which the target data page belongs in the target tablespace slice based on the data page identification;

Generating a data search request carrying a target slice identifier corresponding to the target table space slice and a target space block identifier corresponding to the target table space block, and sending the data search request to an allocation node; the allocation node is used for inquiring a target storage unit group and a target storage partition corresponding to the target slice identifier, inquiring a local storage space corresponding to the target space block identifier in the target storage partition, and obtaining target storage mapping information corresponding to the target space block identifier;

The searching operation data in the target data page from the target storage partition in the target storage unit group based on the target storage mapping information fed back by the allocation node includes:

And searching operation data in the target data page from the corresponding local storage space in the target storage partition in the target storage unit group based on the target storage mapping information fed back by the distribution node.

14. A database processing apparatus, the apparatus comprising:

15. A database processing apparatus, the apparatus comprising:

16. A database processing apparatus, the apparatus comprising:

and the processing result determining module is used for processing the searched operation data to obtain a processing result corresponding to the database operation request.

17. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 13 when the computer program is executed.

18. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 13.

19. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 13.