CN114265828A - Line migration elimination method and device, computer equipment and storage medium - Google Patents
Line migration elimination method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114265828A CN114265828A CN202111560676.2A CN202111560676A CN114265828A CN 114265828 A CN114265828 A CN 114265828A CN 202111560676 A CN202111560676 A CN 202111560676A CN 114265828 A CN114265828 A CN 114265828A
- Authority
- CN
- China
- Prior art keywords
- row
- data block
- record
- migration
- original data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application relates to the technical field of cloud data, and provides a line migration elimination method, a line migration elimination device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: detecting a record row with row migration, determining a row identifier corresponding to the record row, searching a current data block where the record row is located after the row migration and an original data block where the record row is located before the row migration based on the row identifier, determining a remaining free space of the original data block and a storage space required to be occupied by the record row, migrating the record row in the current data block to the original data block when the remaining free space is not smaller than the storage space, and deleting a pointer in the original data block, which points to the current data block, so that extra I/O operations of a database are reduced, and the data processing performance of the database is improved as a whole.
Description
Technical Field
The present application relates to the field of cloud data technologies, and in particular, to a method and an apparatus for eliminating row migration, a computer device, a storage medium, and a computer program product.
Background
When a non-row-linked row is modified in an oracle (relational database management system), the length of the modified row is larger than that of the row before modification, and when the data block cannot accommodate the modified row, the data of the whole row is migrated to another data block, and a pointer pointing to a new position of the row is kept in the original data block, which is the case of row migration. The performance of the database may be degraded when the data for which the row migration occurs is queried, and additional I/O operations may be generated.
The conventional scheme is to store data with row migration into a temporary table, delete the data with row migration from the original table, take out the data of the temporary table and re-insert the data into the original table, and delete the temporary table. However, this processing method does not consider the case of the foreign key constraint, which may result in that the data row with the row migration and the foreign key constraint cannot be deleted, and the database performance is affected.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus, a computer device, a computer readable storage medium, and a computer program product for eliminating row migration, which can improve the performance of a database.
In a first aspect, the present application provides a row migration elimination method. The method comprises the following steps:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line;
based on the row identification, searching a current data block where the record row is located after the record row is migrated and an original data block where the record row is located before the record row is migrated;
determining the residual free space of the original data block and the storage space required to be occupied by the recording line;
and when the residual free space is not less than the storage space, migrating the record row in the current data block to the original data block, and deleting a pointer pointing to the current data block in the original data block.
In one embodiment, the determining the remaining free space of the original data block includes:
acquiring a data block identifier corresponding to the original data block;
positioning the record information corresponding to the original data block in a data table by taking the data block identifier as an index condition;
and obtaining the residual free space of the original data block from the recording information.
In one embodiment, the obtaining the remaining free space of the original data block from the recording information includes:
acquiring the total storage space of the original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information;
accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block;
and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
In one embodiment, the detecting that there is a record line with line migration and determining a line identifier corresponding to the record line includes:
screening out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior;
and determining a record row with row migration and a row identifier corresponding to the record row based on the target information table.
In one embodiment, the determining, based on the target information table, that there is a record row with row migration and a row identifier corresponding to the record row includes:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table;
and traversing the data dictionary table, and determining the record rows with row migration and the row identifiers corresponding to the record rows.
In one embodiment, the method further comprises:
triggering a row migration elimination task aiming at the recording row according to a preset period;
and when the residual free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the residual free space exists, ending the line migration elimination task in the current period.
In a second aspect, the present application further provides a row migration elimination apparatus. The device comprises:
the detection module is used for detecting the record rows with row migration and determining the row identifiers corresponding to the record rows;
the database searching module is used for searching the current data block where the record row is located after the record row is migrated and the original data block where the record row is located before the record row is migrated based on the row identification;
a space determining module, configured to determine a remaining free space of the original data block and a storage space required to be occupied by the record line;
and the line migration eliminating module is used for migrating the recording line in the current data block to the original data block and deleting the pointer which points to the current data block in the original data block when the residual free space is not smaller than the storage space.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line;
based on the row identification, searching a current data block where the record row is located after the record row is migrated and an original data block where the record row is located before the record row is migrated;
determining the residual free space of the original data block and the storage space required to be occupied by the recording line;
and when the residual free space is not less than the storage space, migrating the record row in the current data block to the original data block, and deleting a pointer pointing to the current data block in the original data block.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line;
based on the row identification, searching a current data block where the record row is located after the record row is migrated and an original data block where the record row is located before the record row is migrated;
determining the residual free space of the original data block and the storage space required to be occupied by the recording line;
and when the residual free space is not less than the storage space, migrating the record row in the current data block to the original data block, and deleting a pointer pointing to the current data block in the original data block.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line;
based on the row identification, searching a current data block where the record row is located after the record row is migrated and an original data block where the record row is located before the record row is migrated;
determining the residual free space of the original data block and the storage space required to be occupied by the recording line;
and when the residual free space is not less than the storage space, migrating the record row in the current data block to the original data block, and deleting a pointer pointing to the current data block in the original data block.
The above-described line migration elimination method, apparatus, computer device, storage medium, and computer program product, the method comprises the steps of determining a line identifier corresponding to a recording line by detecting the recording line with line migration, facilitating the positioning of the recording line, searching a current data block where the recording line is located after the line migration and an original data block where the recording line is located before the line migration based on the line identifier, determining the positions of the recording line before and after the line migration, determining the remaining free space of the original data block and the storage space occupied by the recording line so as to compare the space size, when the remaining free space is not less than the storage space, the record line in the current data block is migrated to the original data block, and the pointer in the original data block to the current data block is deleted, therefore, row migration is eliminated, additional I/O operations are reduced, and the data processing performance of the database is improved on the whole.
Drawings
FIG. 1 is a diagram of an application environment for a row migration elimination method in one embodiment;
FIG. 2 is a flow diagram illustrating a row migration elimination method according to an embodiment;
FIG. 3 is a flow chart illustrating a row migration elimination method in another embodiment;
FIG. 4 is a flow chart illustrating a row migration elimination method in yet another embodiment;
FIG. 5 is a block diagram showing the structure of a row migration elimination apparatus according to one embodiment;
FIG. 6 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The row migration elimination method provided by the embodiment of the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 may be configured to configure a triggering condition of a row migration elimination event, and when the server detects that the triggering condition is met, the server triggers the row migration elimination event to perform row migration elimination, where the processing performed by the server 104 specifically includes: detecting a recording line with line migration, determining a line identifier corresponding to the recording line, searching a current data block where the recording line is located after the line migration and an original data block where the recording line is located before the line migration based on the line identifier, determining a residual free space of the original data block and a storage space required to be occupied by the recording line, migrating the recording line in the current data block to the original data block when the residual free space is not less than the storage space, and deleting a pointer pointing to the current data block in the original data block, thereby eliminating the line migration of the recording line.
The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), and a big data and artificial intelligence platform.
In one embodiment, as shown in fig. 2, a row migration elimination method is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
Where line migration means that a line of data is relocated to another block, which is not suitable for placing the current block, but a pointer is kept in the original block, the line id of the index still points to the original location. Specifically, when a record row not linked with a row is modified in the oracle database, the length of the modified record row is greater than that of the record row before modification, and when the free space in the data block is small and cannot accommodate the modified record row, the oracle database migrates the data of the whole row to another data block, and a pointer pointing to a new position of the row is reserved in an original data block, wherein the oracle database may be a cloud database stored in a cloud.
The record row is a unit of data for data recording by row in the oracle database. Unlike line linking, which refers to a way of storing data using multiple data blocks divided into slices when a line of data is too large to fit in a single data block, data corresponding to a recorded line is recorded in the same data block. For example, when an oracle data block size of 4kb is used and 8k is needed to insert a row of data, the oracle database needs to store the data by dividing 3 data blocks into pieces.
The row identifier refers to a unique identifier corresponding to each row of data in a table of the oracle database, and may be called rowid, and the rowid is usually used to access data inside the oracle. In particular, rowid requires 10 bytes of storage space and is displayed with a string that indicates the physical specific location of the row in the oracle database. The character string includes a data object number, a corresponding file number, a block number, and a line number. The data object number indicates the number of the database object to which the row belongs, each data object is uniquely assigned a number at the time of database establishment, and the number is unique. The corresponding file number indicates the number of the file in which the row is located, and each file label of the table space is unique. The block number indicates the location of the block of the file in which the line resides, and the line number indicates the specific location of the line in the line directory. The row identifier can represent the internal address of the row data in the physical table, and comprises two addresses, one is the address pointing to the data file stored in the block containing the row in the data table, and the other is the address of the row in the data block which can be directly positioned to the data row.
Specifically, for the record row migration, the oracle database migrates the entire row of data into a data block capable of storing the next entire row of data, the migrated original pointer points to a new data block storing a new row of data, and Rowid is unchanged. When a data line migrates, accessing it will cause a reduction in I/O performance because the oracle must access more data blocks to obtain the data of the data line. The server responds to the trigger of the row migration elimination event, starts a detection process of the record rows with the row migration, detects and positions the record rows with the row migration, and stores rowids of the record rows.
And step 204, based on the row identification, searching the current data block where the record row is located after the row migration and the original data block where the record row is located before the row migration.
Wherein, the data blocks of the recording rows before and after the row migration are different. And the data block where the record row is located after the row migration is the current data block, and the data block where the record row is located before the row migration is the original data block. The pointer pointing to the storage position of the recording line in the current data block is stored in the original data block, so that when data search is performed on the recording line, the data block where the recording line is located currently can be determined based on the pointer stored in the original data block, and data corresponding to the recording line is acquired.
Specifically, the original data block is used as the data block where the recording line is located when the line migration does not occur before the modification, and after the line migration occurs, a pointer of the data block where the current recording line is located is stored in the original data block. The server determines a current data block where the recording line is located currently and a pointer which points to the current data block and corresponds to the recording line based on the line identification, and the server realizes the positioning of the original data block corresponding to the recording line by positioning the data block stored with the corresponding pointer.
In step 206, the remaining free space of the original data block and the storage space required to be occupied by the recording line are determined.
The remaining free space is the sum of the space sizes of the fragmented free spaces in the data block, in which no data is stored. It should be noted that the remaining free space is not required to be a continuous storage space, and may be composed of a plurality of fragmented storage spaces. The storage space required by the recording lines refers to the space required by the recording lines for storing in the data blocks, the storage space required by the recording lines is related to the number of the data contents corresponding to the recording lines, the more the data contents of the recording lines, the larger the storage space required by the recording lines, the less the data contents of the recording lines, and the smaller the storage space required by the recording lines.
Specifically, the server may obtain storage information of the record row through table lookup, where the storage information is a size of a storage space occupied by the record row in the data block, and determine the storage space required to be occupied by the record row by obtaining the storage information. The server can locate the position of the original data block through the data block identifier of the original data block and check the remaining free space of the original data block.
And step 208, when the remaining free space is not less than the storage space, migrating the record line in the current data block to the original data block, and deleting the pointer pointing to the current data block in the original data block.
After the line migration occurs, the recording line is stored in the current data block, and a pointer pointing to the current data block is stored in the original data block. And the server compares the size of the storage space required to be occupied by the recording line with the size of the residual free space of the original data block to obtain a comparison result, and executes further data processing operation according to the comparison result. The comparison result includes two conditions that the remaining free space of the original data block is greater than or equal to the size of the storage space required to be occupied by the recording line and the remaining free space of the original data block is less than the size of the storage space required to be occupied by the recording line, different comparison results correspond to different data processing operations, and the association relationship between the specific data processing operation and the comparison result can be preset.
Specifically, if the remaining free space of the original data block is greater than or equal to the size of the storage space required to be occupied by the recording line, the representation may migrate the recording line from the current data block to the original data block, so as to eliminate the line migration. In order to eliminate the line migration, two steps need to be completed, the first is to migrate the recording line in the current data block to the original data block to release the space in the current data block, so as to realize the migration of data from the current data block to the original data block, and the second is to delete the pointer pointing to the current data block in the original data block, so as to release the association between the two data blocks.
The line migration eliminating method determines the line identification corresponding to the record line by detecting the record line with line migration, is convenient for positioning the record line, searches the current data block where the record line is located after the line migration and the original data block where the record line is located before the line migration based on the line identification, determines the positions of the record line before and after the line migration, determines the residual free space of the original data block and the storage space occupied by the record line so as to compare the space size, when the remaining free space is not less than the storage space, the record row in the current data block is migrated to the original data block, and the pointer pointing to the current data block in the original data block is deleted, so that the problem that the record with the external key constraint and line migration cannot be deleted in the traditional scheme is solved, and a large amount of time is not consumed in maintaining the balance of the index tree. Due to the fact that a large amount of table space fragments can occur in the long-term operation process of the database, the recording rows are migrated to the original data blocks capable of accommodating the recording rows, pointers of the original data blocks are deleted, and the space of the current data blocks is released. On one hand, the row migration phenomenon is eliminated, extra I/O operation is reduced, on the other hand, the free fragment space of the tablespace is filled, the table space fragmentation phenomenon is relieved, the space of the current data block is released, and the data processing performance of the database is improved on the whole.
In one embodiment, determining the remaining free space of the original data block comprises: acquiring a data block identifier corresponding to an original data block; positioning the record information corresponding to the original data block in the data table by taking the data block identifier as an index condition; the remaining free space of the original data block is derived from the recording information.
The data block identification is a unique identifier corresponding to each data block in the oracle database, and different data blocks have different data block identifications. The data table is a table used for recording corresponding information of each data block in an oracle database. The recording information is attribute information of the data block, and includes a size of a total storage space of the data block, a size of a used storage space or a size of a remaining free space of the data block, and the like.
Specifically, the server locates the record information corresponding to the original data block from the data table by acquiring the data block identifier corresponding to the original data block and taking the data block identifier of the original data block as an index condition, and queries the storage space information of the original data block from the record information corresponding to the original data block according to the keyword corresponding to the storage space, so as to obtain the remaining free space of the original data block.
In this embodiment, the recording information of the original data block is located by using the data block identifier corresponding to the original data block as an index condition, so as to obtain the remaining free space of the original data block, thereby achieving rapid and accurate acquisition of the remaining free space and improving the data processing speed.
In one embodiment, deriving the remaining free space of the original data block from the recording information comprises: acquiring the total storage space of an original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information; accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block; and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
The total storage space refers to a storage space that can be provided by a data block before any recording line is not stored, the total storage spaces of different data blocks may be the same or different, and the total storage space of a data block may be specifically configured according to an actual application scenario. The occupied storage space refers to the storage space occupied by each currently stored recording line, the accumulated occupied storage space refers to the sum of the storage spaces occupied by the currently stored recording lines, and the accumulated occupied storage space comprises the space for storing the data corresponding to the recording lines and the space for storing the pointers corresponding to the recording lines.
Specifically, when the server obtains the storage information from the record information in the data table as the used storage space of the data block, the remaining storage space of the original data block is obtained by calculating the difference between the size of the total storage space of the data block and the size of the used storage space. In a specific application, the storage information obtained by the server from the recording information in the data table may be an occupied storage space corresponding to each stored recording line, and since the occupied storage space corresponding to each recording line may be discontinuous, the server obtains the size of the occupied storage space corresponding to the data block by accumulating the size of the occupied storage space corresponding to each recording line in the data block, and then obtains the remaining storage space of the original data block by calculating the difference between the size of the total storage space of the data block and the size of the used storage space.
When the server obtains the storage information from the record information in the data table as the size of the remaining storage space, the remaining free space of the original data block can be directly obtained. The remaining free space of the original data block may be obtained by counting the data block every time the data storage is completed, so that the server can obtain the remaining free space of the original data block quickly.
In this embodiment, by obtaining the total storage space of the original data block from the recording information and calculating the cumulative occupied storage space of each recording line, the remaining free space of the original data block can be accurately obtained based on the total storage space and the cumulative occupied storage space by analyzing the recording lines with discontinuous occupied storage spaces, so as to improve the accuracy of the result of performing the space size comparison subsequently.
In one embodiment, as shown in fig. 3, detecting a record line with line migration, and determining a line identifier corresponding to the record line includes:
and 302, screening out a target information table representing the row migration behavior from the information table based on the data storage mode corresponding to the row migration behavior.
The data storage mode comprises a storage mode of storing data in the same data block, a storage mode of storing data in different data blocks through line linkage, and a storage mode of migrating the data to another data block through line migration and reserving a pointer in the data block before migration. The information table is a data table for recording the storage information corresponding to the rows, the data storage mode corresponding to each recording row can be determined based on the storage information recorded in the information table, and the row identifier corresponding to the corresponding recording row is recorded in the information table.
Specifically, the server screens out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior, so as to screen the record rows with the row migration behavior, and the server determines the record rows with the row migration and the row identifiers corresponding to the record rows based on the row identifiers recorded in the target information table.
In this embodiment, the server screens out a target information table representing the row migration behavior from the information table according to a data storage manner specific to the row migration behavior, and based on the target information table, can quickly and accurately locate the record row in which the row migration exists and determine the row identifier corresponding to the record row, thereby implementing accurate identification of the record row in which the row migration exists.
In one embodiment, determining, based on the target information table, a record row for which row migration exists and a row identifier corresponding to the record row includes:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table; and traversing the data dictionary table, and determining the record rows with the row migration and the row identifications corresponding to the record rows.
The data dictionary table is created by a script UTLCHAIN. SQL or a script UTLCHN1.SQL and is used for receiving data results for analyzing the information table. The server may query the data result of analyzing the information table by traversing the data dictionary table, i.e. whether there is a record row for row migration and a corresponding row identifier when there is a record row for row migration.
Specifically, the server screens a target information table with row migration by acquiring an information table including recorded row information and based on a data storage mode corresponding to row migration behaviors, records row identifications corresponding to record rows with row migration into a data dictionary table by creating the data dictionary table and analyzing the table with row migration, and can query the row identifications corresponding to the record rows with row migration through the data dictionary table.
In this embodiment, the server records the row identifier corresponding to the record row generating the row migration behavior by creating the data dictionary table, so that the row identifier corresponding to the record row having the row migration behavior can be conveniently and quickly acquired, and the data processing efficiency is improved.
In one embodiment, the method further comprises: triggering a row migration elimination task aiming at the recording row according to a preset period; and when the remaining free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the remaining free space exists, ending the line migration elimination task in the current period.
The preset period refers to a trigger period of a row migration elimination task for a recording row, and can be set according to actual scene needs.
Specifically, the server tries to perform line migration elimination on all record lines with line migration behaviors in a preset period, when the remaining free space is larger than or equal to the storage space of the current record line, the server migrates the current record line to the original data block, deletes a pointer pointing to the current data volume in the original data block, and when the remaining free space is smaller than the storage space of the current record line, the server skips the record line, and performs the same processing on the next unprocessed record line as the current record line until no record line with the storage space smaller than or equal to the remaining free space exists, thereby ending the line migration elimination task of the current period, and waiting for triggering of the line migration elimination task of the record line of the next period.
In this embodiment, by setting a trigger cycle of a row migration elimination task for a record row and executing the row migration elimination task for the record row on a periodic basis, it is possible to adaptively eliminate row migration automatically in response to a change in storage space in a data block, thereby improving data processing performance of a database.
In a specific application, as shown in fig. 4, a row migration elimination method is provided, which specifically includes the following processing procedures:
when a non-row-linked row is modified in an oracle, the length of the modified row is larger than that of the row before modification, and when the free space in the data block is small and cannot accommodate the modified row, the oracle will migrate the data of the whole row to another data block, and keep a pointer pointing to a new position of the row in the original data block, which is the case of row migration. When data subjected to row migration is queried, performance of the database is lowered, and because the rowid of the row subjected to row migration cannot be changed, 2 data blocks need to be accessed when the data row subjected to row migration is accessed, and additional I/O operation is generated.
The conventional scheme is to store the data with row migration into a temporary table (or called an intermediate table), delete the data with row migration from the original table, take out the data of the temporary table and insert the data into the original table, and delete the temporary table. Since the case of the foreign key constraint is not considered in this processing method, the data line with the line migration and the foreign key constraint cannot be deleted. And this approach does not process the index at the time of insertion and deletion, which results in a significant amount of time spent on maintaining the balance of the index tree.
Because the use scene of the traditional scheme is limited and the database generates table space fragmentation after running for a period of time, the method provides that the record rows with row migration are collected, the residual free space of the original data blocks of the record rows is compared with the size of the record rows, when the original data blocks can contain the record rows, the record rows are migrated into the original data blocks, the pointers of the original data blocks are deleted, and the space of the current data blocks is released.
The specific steps for eliminating the line migration are as follows:
the first step is as follows: detecting and collecting rowids with row migration;
the server collects rowids for the record lines by locating the record lines for which the line migration occurred. Specifically, the server acquires a table including recorded row information, screens the table with row migration based on a data storage mode corresponding to row migration behaviors, creates a data dictionary table, records rowid information generating row migration into the data dictionary table by analyzing the table with row migration, and can query the rowid with row migration in the data dictionary table.
The second step is that: inquiring the size of the record line;
specifically, the server obtains the storage information of the record row through table query, the storage information refers to the size of the storage space occupied by the record row in the data block, and the size of the space of the record row is determined through obtaining the storage information so as to be used for comparing with the size of the remaining free space of the original data block.
The third step: finding the original data block where the record row is located;
specifically, the server determines a current data block where the recording line is currently located and a pointer, corresponding to the recording line, pointing to the current data block based on the line identifier, and the server locates the original data block corresponding to the recording line by locating the data block in which the corresponding pointer is stored.
The fourth step: inquiring the spare space of the residual fragments of the original data block;
specifically, the server locates the record information corresponding to the original data block from the data table by acquiring the data block identifier corresponding to the original data block and taking the data block identifier of the original data block as an index condition, and obtains the remaining storage space of the original data block by calculating the difference between the size of the total storage space of the data block and the size of the used storage space when the server acquires the used storage space of which the storage information is the data block from the record information in the data table according to the keyword corresponding to the storage space. In a specific application, the storage information obtained by the server from the recording information in the data table may be an occupied storage space corresponding to each stored recording line, and since the occupied storage space corresponding to each recording line may be discontinuous, the server obtains the size of the occupied storage space corresponding to the data block by accumulating the size of the occupied storage space corresponding to each recording line in the data block, and then obtains the remaining storage space of the original data block by calculating the difference between the size of the total storage space of the data block and the size of the used storage space.
The fifth step: judging whether the spare space of the residual fragments of the original data block is larger than the size of the recording line or not; if yes, entering the sixth step, and if not, entering the eighth step;
sixthly, migrating the record row in the current data block to the original data block;
skipping the recording line, and eliminating line migration when the free space of the residual fragments of the original data block is larger than the size of the recording line;
eighth step: the pointer in the original data block to the current data block is deleted and the space in the current data block is released.
Specifically, if the remaining free space of the original data block is greater than or equal to the size of the storage space required to be occupied by the recording line, the representation may migrate the recording line from the current data block to the original data block, so as to eliminate the line migration. In order to eliminate the line migration, the server needs to complete two steps, the first is to migrate the recording line in the current data block to the original data block to release the space in the current data block, so as to realize the migration of data from the current data block to the original data block, and the second is to delete the pointer pointing to the current data block in the original data block, so as to release the association between the two data blocks.
The line migration elimination method solves the problem that the traditional scheme can not delete the records with the line migration and the external key constraint, and does not need to consume a large amount of time on maintaining the balance of the index tree. Because a large amount of table space fragments can occur in the long-term operation process of the database, based on the point, the scheme migrates the recording rows into the original data blocks capable of accommodating the recording rows, deletes the pointers of the original data blocks and releases the space of the current data blocks. On one hand, the row migration phenomenon is eliminated, extra I/O operation is reduced, on the other hand, the free fragment space of the tablespace is filled, the fragmentation phenomenon of the tablespace is relieved, the data block space is released, and the data processing performance of the database is improved on the whole.
It should be understood that, although the steps in the flowcharts related to the embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides a row migration elimination apparatus for implementing the row migration elimination method mentioned above. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the row migration elimination apparatus provided below may refer to the limitations on the row migration elimination method in the foregoing, and details are not described here.
In one embodiment, as shown in fig. 5, there is provided a row migration eliminating apparatus including: a detection module 502, a database lookup module 504, a space determination module 506, and a row migration elimination module 508, wherein:
a detection module 502, configured to detect a record row with row migration and determine a row identifier corresponding to the record row;
a database searching module 504, configured to search, based on the row identifier, a current data block where the record row is located after the row migration and an original data block where the record row is located before the row migration;
a space determining module 506, configured to determine a remaining free space of the original data block and a storage space required to be occupied by the recording line;
and the line migration eliminating module 508 is configured to, when the remaining free space is not less than the storage space, migrate a recording line in the current data block to the original data block, and delete a pointer in the original data block that points to the current data block.
In one embodiment, the space determining module is further configured to obtain a data block identifier corresponding to the original data block; positioning the record information corresponding to the original data block in the data table by taking the data block identifier as an index condition; the remaining free space of the original data block is derived from the recording information.
In one embodiment, the space determining module is further configured to obtain, from the recording information, a total storage space of the original data block and a space size of an occupied storage space corresponding to each recording line in the original data block; accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block; and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
In one embodiment, the detection module is further configured to screen out a target information table representing the row migration behavior from the information table based on a data storage manner corresponding to the row migration behavior; and determining the record rows with the row migration and the row identifications corresponding to the record rows based on the target information table.
In one embodiment, the detection module is further configured to record, based on the target information table, a row identifier corresponding to a record row generating a row migration behavior into the data dictionary table; and traversing the data dictionary table, and determining the record rows with the row migration and the row identifications corresponding to the record rows.
In one embodiment, the row migration elimination apparatus further includes a task triggering module, configured to trigger a row migration elimination task for the recording row according to a preset period; and when the remaining free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the remaining free space exists, ending the line migration elimination task in the current period.
According to the row migration eliminating device, the row identification corresponding to the recording row is determined by detecting the existence of the recording row of the row migration, so that the positioning of the recording row is facilitated, the current data block where the recording row is located after the row migration and the original data block where the recording row is located before the row migration are searched based on the row identification, the positions of the recording row before and after the row migration are determined, the remaining free space of the original data block and the storage space occupied by the recording row are determined, so that the comparison of the space size is facilitated, when the remaining free space is not smaller than the storage space, the recording row in the current data block is migrated into the original data block, and the pointer pointing to the current data block in the original data block is deleted, so that the row migration is eliminated, the additional I/O operation is reduced, and the data processing performance of the database is improved on the whole.
The respective modules in the line migration elimination apparatus may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 6. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a line migration elimination method.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line; based on the row identification, searching a current data block where the record row is located after the row migration and an original data block where the record row is located before the row migration; determining the residual free space of the original data block and the storage space occupied by the recording line; and when the residual free space is not less than the storage space, migrating the record line in the current data block to the original data block, and deleting the pointer pointing to the current data block in the original data block.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring a data block identifier corresponding to an original data block; positioning the record information corresponding to the original data block in the data table by taking the data block identifier as an index condition; the remaining free space of the original data block is derived from the recording information.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring the total storage space of an original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information; accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block; and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
screening out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior; and determining the record rows with the row migration and the row identifications corresponding to the record rows based on the target information table.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table; and traversing the data dictionary table, and determining the record rows with the row migration and the row identifications corresponding to the record rows.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
triggering a row migration elimination task aiming at the recording row according to a preset period; and when the remaining free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the remaining free space exists, ending the line migration elimination task in the current period.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line; based on the row identification, searching a current data block where the record row is located after the row migration and an original data block where the record row is located before the row migration; determining the residual free space of the original data block and the storage space occupied by the recording line; and when the residual free space is not less than the storage space, migrating the record line in the current data block to the original data block, and deleting the pointer pointing to the current data block in the original data block.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a data block identifier corresponding to an original data block; positioning the record information corresponding to the original data block in the data table by taking the data block identifier as an index condition; the remaining free space of the original data block is derived from the recording information.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring the total storage space of an original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information; accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block; and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
In one embodiment, the computer program when executed by the processor further performs the steps of:
screening out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior; and determining the record rows with the row migration and the row identifications corresponding to the record rows based on the target information table.
In one embodiment, the computer program when executed by the processor further performs the steps of:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table; and traversing the data dictionary table, and determining the record rows with the row migration and the row identifications corresponding to the record rows.
In one embodiment, the computer program when executed by the processor further performs the steps of:
triggering a row migration elimination task aiming at the recording row according to a preset period; and when the remaining free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the remaining free space exists, ending the line migration elimination task in the current period.
In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line; based on the row identification, searching a current data block where the record row is located after the row migration and an original data block where the record row is located before the row migration; determining the residual free space of the original data block and the storage space occupied by the recording line; and when the residual free space is not less than the storage space, migrating the record line in the current data block to the original data block, and deleting the pointer pointing to the current data block in the original data block.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a data block identifier corresponding to an original data block; positioning the record information corresponding to the original data block in the data table by taking the data block identifier as an index condition; the remaining free space of the original data block is derived from the recording information.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring the total storage space of an original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information; accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block; and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
In one embodiment, the computer program when executed by the processor further performs the steps of:
screening out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior; and determining the record rows with the row migration and the row identifications corresponding to the record rows based on the target information table.
In one embodiment, the computer program when executed by the processor further performs the steps of:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table; and traversing the data dictionary table, and determining the record rows with the row migration and the row identifications corresponding to the record rows.
In one embodiment, the computer program when executed by the processor further performs the steps of:
triggering a row migration elimination task aiming at the recording row according to a preset period; and when the remaining free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the remaining free space exists, ending the line migration elimination task in the current period.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A method for eliminating row migration, the method comprising:
detecting a recording line with line migration, and determining a line identifier corresponding to the recording line;
based on the row identification, searching a current data block where the record row is located after the record row is migrated and an original data block where the record row is located before the record row is migrated;
determining the residual free space of the original data block and the storage space required to be occupied by the recording line;
and when the residual free space is not less than the storage space, migrating the record row in the current data block to the original data block, and deleting a pointer pointing to the current data block in the original data block.
2. The method of claim 1, wherein the determining the remaining free space of the original data block comprises:
acquiring a data block identifier corresponding to the original data block;
positioning the record information corresponding to the original data block in a data table by taking the data block identifier as an index condition;
and obtaining the residual free space of the original data block from the recording information.
3. The method of claim 2, wherein the deriving the remaining free space of the original data block from the recording information comprises:
acquiring the total storage space of the original data block and the space size of occupied storage space corresponding to each recording line in the original data block from the recording information;
accumulating the space size of the occupied storage space corresponding to the recording line to obtain the accumulated occupied storage space corresponding to the original data block;
and obtaining the residual free space of the original data block based on the total storage space corresponding to the original data block and the accumulated occupied storage space.
4. The method according to claim 1, wherein the detecting that there is a record row with row migration and determining a row identifier corresponding to the record row comprises:
screening out a target information table representing the row migration behavior from the information table based on a data storage mode corresponding to the row migration behavior;
and determining a record row with row migration and a row identifier corresponding to the record row based on the target information table.
5. The method according to claim 4, wherein the determining, based on the target information table, that there is a record row of row migration and a row identifier corresponding to the record row comprises:
recording the row identification corresponding to the record row generating the row migration behavior to a data dictionary table based on the target information table;
and traversing the data dictionary table, and determining the record rows with row migration and the row identifiers corresponding to the record rows.
6. The method of claim 1, further comprising:
triggering a row migration elimination task aiming at the recording row according to a preset period;
and when the residual free space is smaller than the storage space of the current recording line and no recording line with the storage space smaller than or equal to the residual free space exists, ending the line migration elimination task in the current period.
7. A row migration elimination apparatus, comprising:
the detection module is used for detecting the record rows with row migration and determining the row identifiers corresponding to the record rows;
the database searching module is used for searching the current data block where the record row is located after the record row is migrated and the original data block where the record row is located before the record row is migrated based on the row identification;
a space determining module, configured to determine a remaining free space of the original data block and a storage space required to be occupied by the record line;
and the line migration eliminating module is used for migrating the recording line in the current data block to the original data block and deleting the pointer which points to the current data block in the original data block when the residual free space is not smaller than the storage space.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111560676.2A CN114265828A (en) | 2021-12-20 | 2021-12-20 | Line migration elimination method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111560676.2A CN114265828A (en) | 2021-12-20 | 2021-12-20 | Line migration elimination method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114265828A true CN114265828A (en) | 2022-04-01 |
Family
ID=80828003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111560676.2A Pending CN114265828A (en) | 2021-12-20 | 2021-12-20 | Line migration elimination method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114265828A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114840497A (en) * | 2022-05-31 | 2022-08-02 | 苏州浪潮智能科技有限公司 | Method, system and device for preprocessing row migration of database and storage medium |
-
2021
- 2021-12-20 CN CN202111560676.2A patent/CN114265828A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114840497A (en) * | 2022-05-31 | 2022-08-02 | 苏州浪潮智能科技有限公司 | Method, system and device for preprocessing row migration of database and storage medium |
| CN114840497B (en) * | 2022-05-31 | 2024-01-12 | 苏州浪潮智能科技有限公司 | Line migration preprocessing method, system and device for database and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10642515B2 (en) | Data storage method, electronic device, and computer non-volatile storage medium | |
| US10552402B2 (en) | Database lockless index for accessing multi-version concurrency control data | |
| US10127260B2 (en) | In-memory database system providing lockless read and write operations for OLAP and OLTP transactions | |
| US9043334B2 (en) | Method and system for accessing files on a storage system | |
| US9149054B2 (en) | Prefix-based leaf node storage for database system | |
| US8290991B2 (en) | Atomic deletion of database data categories | |
| US9514211B2 (en) | High throughput data modifications using blind update operations | |
| KR102031588B1 (en) | Method and system for implementing index when saving file | |
| US20160283501A1 (en) | Posix-compatible file system, method of creating a file list and storage device | |
| CN102231168B (en) | Method for quickly retrieving resume from resume database | |
| CN110888837B (en) | Object storage small file merging method and device | |
| US11777983B2 (en) | Systems and methods for rapidly generating security ratings | |
| CN111104377B (en) | File management method, electronic device and computer readable storage medium | |
| Li et al. | Database management strategy and recovery methods of Android | |
| CN113094374A (en) | Distributed storage and retrieval method and device and computer equipment | |
| CN112416880A (en) | Method and device for optimizing storage performance of mass small files based on real-time merging | |
| CN114265828A (en) | Line migration elimination method and device, computer equipment and storage medium | |
| CN109947730A (en) | Metadata restoration methods, device, distributed file system and readable storage medium storing program for executing | |
| Zhang et al. | Recovering SQLite data from fragmented flash pages | |
| CN116303267A (en) | Data access method, device, equipment and storage medium | |
| CN115858471A (en) | Service data change recording method, device, computer equipment and medium | |
| CN115185778A (en) | Database monitoring method and device | |
| CN114416741A (en) | KV data writing and reading method and device based on multi-level index and storage medium | |
| CN114416676A (en) | Data processing method, device, equipment and storage medium | |
| US7565217B2 (en) | Traversal of empty regions in a searchable data structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |