CN114328526A - Data processing method and device, electronic equipment and computer readable storage medium - Google Patents
Data processing method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114328526A CN114328526A CN202111621112.5A CN202111621112A CN114328526A CN 114328526 A CN114328526 A CN 114328526A CN 202111621112 A CN202111621112 A CN 202111621112A CN 114328526 A CN114328526 A CN 114328526A
- Authority
- CN
- China
- Prior art keywords
- index page
- index
- storage space
- database
- byte number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 39
- 239000012634 fragment Substances 0.000 abstract description 75
- 238000005516 engineering process Methods 0.000 abstract description 5
- 239000002699 waste material Substances 0.000 abstract description 3
- 238000004140 cleaning Methods 0.000 description 28
- 238000013467 fragmentation Methods 0.000 description 9
- 238000006062 fragmentation reaction Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method, a data processing device, electronic equipment and a computer readable storage medium. Wherein, the method comprises the following steps: acquiring an index page in a database; inquiring the initial byte number corresponding to the index page in a database; inquiring the target byte number corresponding to the index page in the database in a preset period; determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number; and in the case that the ratio is larger than the preset threshold value, processing is carried out on the basis of the index page so as to release the additionally occupied storage space of the index page. The invention solves the technical problems of time and labor waste and low efficiency of manual processing of fragment information in a database table in the related technology.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a data processing method, an apparatus, an electronic device, and a computer-readable storage medium.
Background
In a database, frequent database operation statements, for example, insert/update/delete operations on database tables and other database operation DML statements, cause gaps in index pages, splitting data rows, and so on, and thus generate a large amount of fragments, that is, the database operation statements are executed in a page with the same content, for example, after updating, the page occupies more storage space, which causes additional page reading during index scanning, and reduces the efficiency of index lookup. The traditional index fragment cleaning mode needs database maintenance personnel to periodically or manually inquire the fragment information of a database table, judges whether the fragment needs to be cleaned according to the fragment information, is low in repeated work and efficiency, cannot timely find a high-rate table containing the fragment, timely cleans the fragment in the table, and possibly causes slow user query and poor experience.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a data processing method and device, electronic equipment and a computer readable storage medium, which are used for at least solving the technical problems of time and labor waste and low efficiency caused by manual processing of fragment information in a database table in the related technology.
According to an aspect of an embodiment of the present invention, there is provided a data processing method including: acquiring an index page in a database; inquiring an initial byte number corresponding to the index page in the database; inquiring a target byte number corresponding to the index page in the database at a preset period; determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number; and if the ratio is larger than a preset threshold value, processing based on the index page to release the additionally occupied storage space of the index page.
Optionally, the obtaining an index page in the database includes: determining a database table according to the database; determining an index built in the database table; and acquiring an index page corresponding to the database table according to the database table and the index.
Optionally, the determining, according to the initial number of bytes and the target number of bytes, a proportion of an extra occupied storage space of the index page includes: determining a difference between the target byte number and the initial byte number; and determining the proportion of the difference value to the target byte number, and taking the proportion as the proportion of the extra occupied storage space of the index page.
Optionally, the processing based on the index page to release the storage space additionally occupied by the index page includes: sending alarm information to a terminal, wherein the alarm information carries the proportion of the index page additionally occupying the storage space and is used for reminding processing based on the index page; receiving a processing request sent by the terminal; and responding to the processing request, and processing based on the index page to release the additionally occupied storage space of the index page.
Optionally, the processing based on the index page to release the storage space additionally occupied by the index page includes: and processing based on the index page at a preset time to reduce the additionally occupied storage space of the index page.
Optionally, the processing based on the index page to release the storage space additionally occupied by the index page includes: deleting the index corresponding to the index page; and establishing a target index to release the additionally occupied storage space of the index page.
Optionally, after the creating the target index to release the storage space additionally occupied by the index page, the method further includes: and acquiring a target index page according to the target index, wherein the content displayed by the target index page is the same as the content displayed by the index page, and the proportion of the extra storage space occupied by the target index page is different from the proportion of the extra storage space occupied by the index page.
According to an aspect of an embodiment of the present invention, there is provided a data processing apparatus including: the acquisition module is used for acquiring an index page in a database; the first query module is used for querying the initial byte number corresponding to the index page in the database; the second query module is used for querying the target byte number corresponding to the index page in the database at a preset period; the determining module is used for determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number; and the processing module is used for processing based on the index page to release the additionally occupied storage space of the index page under the condition that the ratio is greater than a preset threshold value.
According to an aspect of an embodiment of the present invention, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data processing method of any one of the above.
According to an aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, wherein instructions of the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform any one of the data processing methods described above.
In the embodiment of the invention, by determining the index page in the database, the initial byte number corresponding to the index page, that is, the initially occupied storage space, is queried, and the target byte number corresponding to the index page, that is, the occupied storage space in the predetermined period, is queried in the database in the predetermined period. The proportion of the extra storage space occupied by the index page is determined according to the initial byte number and the target byte number, because the index page is the same index page, the target byte number should be the same as the initial byte number under the condition of not changing the content, and when the target byte number is greater than the initial byte number, the current index page can be considered to additionally occupy the storage space. Therefore, the proportion of the storage space additionally occupied by the index page is determined. When the ratio is larger than the preset threshold value, the extra occupied storage space is considered to be too large, and the operation of the database is influenced, so that the processing is carried out based on the index page, and the extra occupied storage space of the index page is released. The target byte number is automatically acquired according to the preset period, and when the ratio obtained by the target byte number is larger than the preset threshold value, the processing is carried out based on the index page, so that the purpose of releasing the additionally occupied storage space of the index page is achieved, and the automatic and efficient processing is carried out. The technical problems that manual processing of fragment information in a database table is time-consuming and labor-consuming and low in efficiency in the correlation technology are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of a data processing method according to an embodiment of the invention;
FIG. 2 is a flow chart of a conventional index shard cleaning method in the related art;
FIG. 3 is a flow diagram of a method for autonomously cleaning index shards in accordance with an alternative embodiment of the present invention;
FIG. 4a is a flow chart of a manual cleaning method provided by an alternative embodiment of the present invention;
FIG. 4b is a flow chart of a method for automatic cleaning of a system provided in an alternative embodiment of the present invention;
fig. 5 is a block diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided an embodiment of a data processing method, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than that herein.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention, as shown in fig. 1, the method including the steps of:
step S102, obtaining an index page in a database;
step S104, inquiring the initial byte number corresponding to the index page in a database;
step S106, inquiring the target byte number corresponding to the index page in the database in a preset period;
step S108, determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number;
and step S110, under the condition that the proportion is larger than a preset threshold value, processing is carried out based on the index page so as to release the additionally occupied storage space of the index page.
Through the steps, the index page in the database is determined, the initial byte number corresponding to the index page, namely the initially occupied storage space, is inquired, and the target byte number corresponding to the index page, namely the occupied storage space in the predetermined period, is inquired in the database in the predetermined period. The proportion of the extra storage space occupied by the index page is determined according to the initial byte number and the target byte number, because the index page is the same index page, the target byte number should be the same as the initial byte number under the condition of not changing the content, and when the target byte number is greater than the initial byte number, the current index page can be considered to additionally occupy the storage space. Therefore, the proportion of the storage space additionally occupied by the index page is determined. When the ratio is larger than the preset threshold value, the extra occupied storage space is considered to be too large, and the operation of the database is influenced, so that the processing is carried out based on the index page, and the extra occupied storage space of the index page is released. Because the target byte number is automatically acquired according to the preset period, and the processing is performed based on the index page when the ratio obtained by the target byte number is greater than the preset threshold value, the purpose of releasing the additionally occupied storage space of the index page can be achieved, and the automatic and efficient processing is performed. The technical problems that manual processing of fragment information in a database table is time-consuming and labor-consuming and low in efficiency in the correlation technology are solved.
As an alternative embodiment, when a plurality of index pages in the database are obtained, the database table may be determined according to the database, and the index established in the database table may be determined. An index is a single, physical storage structure that sorts one or more columns of values in a database table, and is a collection of one or more columns of values in a table and a corresponding list of logical pointers to data pages in the table that physically identify the values. The index is equivalent to the directory of the book, and the required content can be quickly found according to the page number in the directory. Therefore, according to the database table and the index, the corresponding index page can be quickly found, and the content which is in line with the expectation of the user can be found.
It should be noted that, as can be seen from the knowledge of the database, the index needs to occupy a certain storage space. When database operation statements, such as insert/update/delete, are frequently executed on database tables, DML statement operations cause gaps in index pages, split data rows, and so on, thereby generating a large amount of fragmentation, i.e., extra storage space occupied by index pages. For example, the index data of a table has 100 pages, but the fragmentation rate is 25%, so that only 75 pages of data in the 100 replaced pages are valid. That is, the index page should only occupy 75% of the storage space of the data, but actually occupies 25% more storage space, that is, the above-mentioned additional storage space occupied by the index page, and the fragmentation rate, so that the efficiency of the index is very low when the fragmentation rate of the index is very high. Therefore, it is necessary to process the index fragment when the index fragment rate reaches a certain value. So as to release the extra occupied storage space and accelerate the indexing efficiency.
As an alternative embodiment, the database is queried for an initial byte number corresponding to the index page, that is, the database is queried for the storage space actually occupied by the index page, that is, the storage space occupied by useful data. And querying the database for the target byte number corresponding to the index page in a predetermined period, because in the predetermined period, a database operation statement, such as insert/update/delete, may occur for the database table, and after the operation statement is executed, the storage space occupied by the index page becomes large, and there is useless storage space occupied by data, so that the target byte number is obtained. And determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number. When the ratio is determined, a difference between the target byte number and the initial byte number may be determined, where the difference is a storage space occupied by unused data, that is, a storage space occupied by the index fragments. Determining the ratio of the difference to the target byte number, which may also be referred to as a fragmentation rate, i.e., determining the ratio of the unused data to the total data, which is the ratio of the extra storage space occupied by the index page. When the ratio is small, the index fragments are considered to be less, the influence on operations such as retrieval of a database table is less, and a small amount of storage space is occupied, so that the method is acceptable. However, when the ratio is large, it is considered that the index fragments are more, which may affect the retrieval of the database table, cause bad experience to the user, and additionally occupy a large amount of storage space, which may cause waste of resources. Therefore, in the case that the ratio is larger than the predetermined threshold, processing is performed based on the index page to release the storage space additionally occupied by the index page.
As an alternative embodiment, when the index page is processed, a plurality of processing modes are included. Two processing modes, one is a manual processing mode, and the other is a system automatic processing mode, are explained below. The two methods can be applied to the same group of databases, respective operations are executed in different time periods, and the method provided by the optional embodiment can achieve better effects by combining the two methods.
And (I) sending alarm information to the terminal when the proportion is greater than a preset threshold value under the condition that the database is frequently used, wherein the alarm information carries the proportion of the index page additionally occupying the storage space and is used for reminding processing based on the index page. The terminal can be a terminal used by a database manager, so that the database manager can timely acquire the information with excessive index fragments to process the information. And then, the system receives and responds to the processing request sent by the terminal, and the processing is carried out based on the index page so as to release the storage space additionally occupied by the index page, clear the index fragments in time and improve the index retrieval efficiency.
And (II) the method is suitable for processing based on the index page at a preset moment when the use of the database is idle and the proportion is larger than a preset threshold value so as to reduce the extra occupied storage space of the index page. The storage space additionally occupied by the index page is released, the index fragments are timely cleaned, and the index retrieval efficiency is improved. At this time, since the cleaning is automatically performed at a predetermined timing, a complete cleaning log of the index shard, including the database name, the table name, the index name, the shard information, the cleaning time, and the like, can be provided. So that database management personnel can consult in time.
As an alternative embodiment, when the processing is performed based on the index page to release the storage space additionally occupied by the index page, there may be a plurality of ways, for example, the index corresponding to the index page may be deleted, and an index, referred to as a target index herein, may be reestablished to release the storage space additionally occupied by the index page. And acquiring a target index page according to the target index, wherein the content displayed by the target index page is the same as the content displayed by the index page, and the proportion of the extra storage space occupied by the target index page is different from the proportion of the extra storage space occupied by the index page. Namely, the target index page occupies a low proportion of the storage space additionally, and the index page occupies a high proportion of the storage space additionally, so that the purpose of storing the storage space additionally occupied by the index page is achieved.
Based on the above embodiments and alternative embodiments, an alternative implementation is provided, which is described in detail below.
To illustrate how the index shards are generated, the following is exemplified:
creating a test table u _ test with a table structure of u _ id, u _ name and u _ extra; u _ id occupies 4Bytes, u _ name occupies 999Bytes, and u _ extra occupies 400 Bytes. Wherein u _ id and u _ name occupy 1003Bytes together, and 8 pieces of simulation data containing u _ id and u _ name can fill up one page, namely 1003 × 8 ═ 8024 Bytes. By commanding the verified data index information (sp _ used 'u _ test'), index _ size is found to be 8 KB.
Executing a database operation statement, executing an update operation, assuming that the original name of u _ name is 'jerry', modifying u _ name with u _ id of 2 from 'jerry' to 'jerry 2', after the modification is completed, inquiring the index information again, and finding that the index _ size is changed from 8KB to 16KB, and 2 pages of storage space are needed, so that an index fragment is generated.
Fig. 2 is a flowchart of a conventional index fragment cleaning method in the related art, and as shown in fig. 2, the conventional index fragment cleaning method needs a database maintenance worker to periodically or manually query fragment information of a database table, and determines whether to clean the fragment according to the fragment information, so that the conventional index fragment cleaning method is low in efficiency, cannot find a table with a high fragment rate containing the fragment in time, and cleans the fragment in the table in time, which may cause slow query and poor experience of a user.
In view of this, an optional embodiment of the present invention provides a method for autonomously cleaning index fragments based on a structured query language, which can timely obtain fragment information in an index page of a database table, analyze a fragment rate of the fragment information in the index page, and clean an index page with a high fragment rate. When the database is busy, the system can give an alarm to inform database management personnel of manual judgment, and then index fragments are cleared according to the judgment; when the database is idle, the system can realize the autonomous cleaning of the index fragments. Both index fragment cleaning mechanisms ensure normal operation of programs and databases. FIG. 3 is a flowchart of a method for autonomously cleaning index shards according to an alternative embodiment of the present invention, which is shown in FIG. 3 and described in detail below:
s1, configuring the database:
the database can be normally accessed, and the database authority is normal. One or more databases may be configured.
S2, acquiring the information in the database:
the method comprises the steps of acquiring all tables in a database and all index information in the tables, storing database names, table names and index names into a memory, and updating database information by a system every morning to ensure that a complete database table can be covered.
S3, utilizing the fragment query component to query fragments of the database information stored in the memory:
the system is internally provided with a plurality of fragment query components, namely a component I, which can query all indexed fragment information in a database through the transmitted database name; the second component can inquire fragment information of all indexes of the specified table through the incoming database name and the table name; and a third component, wherein the fragment information of the specified index can be inquired through the incoming database name, the table name and the index name. Specific components can be selected for query according to the actual application scene and requirements.
S4, manual cleaning, and automatic alarm notification:
fig. 4a is a flowchart of a manual cleaning method according to an alternative embodiment of the present invention, as shown in fig. 4a, the system periodically obtains index fragment information of the database table at 12 o/20 o' clock of 1/3/5 every week, analyzes a fragment rate of index fragment information in the table, determines that a large number of index fragments exist when the fragment rate is greater than a predetermined threshold, stores the index information with the large number of index fragments in the index fragment table, and notifies the database administrator of an alarm, so that the database administrator can know that an index with the large number of fragments exists at the first time, and the database administrator determines whether to clean the index fragment, and when the cleaning is confirmed, cleans the index fragment, and improves the index retrieval efficiency.
By way of example, in the above example of how the index shard is generated, the logical shard (shard rate) is 50% and the total number of pages is 2 pages by looking up the shard detail information. And judging whether the index fragment needs to be cleaned up or not through the queried logical fragment (avg _ fragmentation _ in _ percentage). The value of avg _ fragmentation _ in _ percent should be as close to zero as possible. However, values from 0% to 10% are acceptable. The lower the value the higher the indexing efficiency.
Therefore, in the case that the fragmentation rate is greater than the predetermined threshold, for example, 30%, 50% is greater than 30%, it is considered that there are a large number of index fragments, the index information of the large number of index fragments is put into the index fragment table, and an alarm is given to the database administrator, so that the database administrator can know that there are indexes of the large number of fragments at the first time and clean the index fragments. And executing the reconstruction index command to complete fragment cleaning.
S5, autonomous idle cleaning:
fig. 4b is a flow chart of an automatic cleaning method for a system according to an alternative embodiment of the present invention, as shown in fig. 4b, the system is timed to a predetermined time of day, in case it is ensured that the database is free, for example, in the early morning. Uniformly acquiring index fragment information of a database table, analyzing the fragment rate of the index fragment information in the table, considering that a large number of index fragments exist under the condition that the fragment rate is greater than a preset threshold value, cleaning the index fragments, and providing a complete cleaning log of the index fragments, wherein the cleaning log comprises database names, table names, index names, fragment information, cleaning time and the like.
For example, as in the operation in S4, in the continuing operation of how the above-described exemplary index shard is generated, by looking up the shard detail information, it can be seen that the logical shard (shard rate) is 50%, and the total number of pages is 2 pages. Setting the predetermined threshold value to be 30%, and setting 50% to be more than 30%, clearing the index shards, and providing a complete clearing log of the index shards, wherein the clearing log comprises database names, table names, index names, shard information, clearing time and the like.
The method in step S4 and the method in step S5 do not interfere with each other and cooperate with each other; s4, when the database is used frequently, the system will alarm to inform the database manager, and the manager will judge if the index fragment needs to be cleaned; s5 is a process for autonomously performing index fragmentation cleaning when the database is free. The two are mutually matched and cooperated, so that a better effect of clearly indexing the fragments can be achieved.
Through the above alternative embodiment, at least the following advantages can be achieved:
(1) frequent manual inquiry of index fragments by maintenance personnel is not needed, and whether the index fragments need to be cleaned or not is judged;
(2) the system realizes autonomous alarm notification and autonomous idle cleaning, greatly improves the working efficiency of maintenance personnel, can clean indexes with high fragment in time and improve the index retrieval efficiency;
(3) the system provides two index fragment cleaning mechanisms to ensure the normal operation of the program and the database.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
According to an embodiment of the present invention, there is also provided an apparatus for implementing the data processing method, and fig. 5 is a block diagram of a structure of the data processing apparatus according to the embodiment of the present invention, as shown in fig. 5, the apparatus includes: an obtaining module 502, a first querying module 504, a second querying module 506, a determining module 508 and a processing module 510, which are described in detail below.
An obtaining module 502, configured to obtain an index page in a database; a first query module 504, connected to the obtaining module 502, configured to query the database for an initial byte number corresponding to the index page; a second query module 506, connected to the first query module 504, for querying a database for a target number of bytes corresponding to the index page at a predetermined period; a determining module 508, connected to the second querying module 506, for determining a ratio of the storage space additionally occupied by the index page according to the initial byte number and the target byte number; and a processing module 510, connected to the determining module 508, configured to, if the ratio is greater than the predetermined threshold, perform processing based on the index page to release the storage space additionally occupied by the index page.
It should be noted here that the obtaining module 502, the first querying module 504, the second querying module 506, the determining module 508 and the processing module 510 correspond to steps S102 to S110 in the implementation of the data processing method, and a plurality of modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the foregoing embodiment 1.
Example 3
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a processor; a memory for storing processor executable instructions, wherein the processor is configured to execute the instructions to implement the data processing method of any of the above.
Example 4
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, wherein instructions of the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform any one of the data processing methods described above.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A data processing method, comprising:
acquiring an index page in a database;
inquiring an initial byte number corresponding to the index page in the database;
inquiring a target byte number corresponding to the index page in the database at a preset period;
determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number;
and if the ratio is larger than a preset threshold value, processing based on the index page to release the additionally occupied storage space of the index page.
2. The method of claim 1, wherein the obtaining an index page in a database comprises:
determining a database table according to the database;
determining an index built in the database table;
and acquiring an index page corresponding to the database table according to the database table and the index.
3. The method of claim 1, wherein the determining the proportion of the storage space additionally occupied by the index page according to the initial byte number and the target byte number comprises:
determining a difference between the target byte number and the initial byte number;
and determining the proportion of the difference value to the target byte number, and taking the proportion as the proportion of the extra occupied storage space of the index page.
4. The method of claim 1, wherein the processing based on the index page to release additional occupied storage space of the index page comprises:
sending alarm information to a terminal, wherein the alarm information carries the proportion of the index page additionally occupying the storage space and is used for reminding processing based on the index page;
receiving a processing request sent by the terminal;
and responding to the processing request, and processing based on the index page to release the additionally occupied storage space of the index page.
5. The method of claim 1, wherein the processing based on the index page to release additional occupied storage space of the index page comprises:
and processing based on the index page at a preset time to reduce the additionally occupied storage space of the index page.
6. The method according to claim 2, wherein the processing based on the index page to release the additionally occupied storage space of the index page comprises:
deleting the index corresponding to the index page;
and establishing a target index to release the additionally occupied storage space of the index page.
7. The method of claim 6, wherein after the creating the target index to free the extra occupied storage space of the index page, further comprising:
and acquiring a target index page according to the target index, wherein the content displayed by the target index page is the same as the content displayed by the index page, and the proportion of the extra storage space occupied by the target index page is different from the proportion of the extra storage space occupied by the index page.
8. A data processing apparatus, comprising:
the acquisition module is used for acquiring an index page in a database;
the first query module is used for querying the initial byte number corresponding to the index page in the database;
the second query module is used for querying the target byte number corresponding to the index page in the database at a preset period;
the determining module is used for determining the proportion of the extra occupied storage space of the index page according to the initial byte number and the target byte number;
and the processing module is used for processing based on the index page to release the additionally occupied storage space of the index page under the condition that the ratio is greater than a preset threshold value.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the data processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111621112.5A CN114328526A (en) | 2021-12-27 | 2021-12-27 | Data processing method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111621112.5A CN114328526A (en) | 2021-12-27 | 2021-12-27 | Data processing method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114328526A true CN114328526A (en) | 2022-04-12 |
Family
ID=81015171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111621112.5A Pending CN114328526A (en) | 2021-12-27 | 2021-12-27 | Data processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328526A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056359A (en) * | 2023-10-09 | 2023-11-14 | 宁波银行股份有限公司 | Table reconstruction method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656885A (en) * | 2018-12-18 | 2019-04-19 | Oppo广东移动通信有限公司 | Memory space monitoring method and device, electric terminal, storage medium |
| CN110879810A (en) * | 2019-12-02 | 2020-03-13 | 北京明略软件系统有限公司 | Data processing method and device |
| US20200242097A1 (en) * | 2019-01-29 | 2020-07-30 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus, device and storage medium for managing index |
-
2021
- 2021-12-27 CN CN202111621112.5A patent/CN114328526A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656885A (en) * | 2018-12-18 | 2019-04-19 | Oppo广东移动通信有限公司 | Memory space monitoring method and device, electric terminal, storage medium |
| US20200242097A1 (en) * | 2019-01-29 | 2020-07-30 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus, device and storage medium for managing index |
| CN110879810A (en) * | 2019-12-02 | 2020-03-13 | 北京明略软件系统有限公司 | Data processing method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056359A (en) * | 2023-10-09 | 2023-11-14 | 宁波银行股份有限公司 | Table reconstruction method and device, electronic equipment and storage medium |
| CN117056359B (en) * | 2023-10-09 | 2024-01-09 | 宁波银行股份有限公司 | Table reconstruction method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111176976A (en) | Test data generation method and device, storage medium and electronic device | |
| CN103312544B (en) | A kind of control terminal reports the method, apparatus and system of journal file | |
| CN102906751A (en) | Method and device for data storage and data query | |
| CN109299157B (en) | Data export method and device for distributed big single table | |
| CN110555012A (en) | data migration method and device | |
| CN102780726A (en) | Log analysis method and log analysis system based on WEB platform | |
| CN110245149B (en) | Metadata version management method and device | |
| CN113051460A (en) | Elasticissearch-based data retrieval method and system, electronic device and storage medium | |
| CN114328526A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN108762979B (en) | Terminal information backup method and backup device based on matching tree | |
| CN113849499A (en) | Data query method and device, storage medium and electronic device | |
| CN106844389B (en) | Method and device for processing URL (Uniform resource locator) | |
| CN110928681A (en) | Data processing method and device, storage medium and electronic device | |
| CN105162622B (en) | A kind of storage method and system | |
| CN112527824B (en) | Paging query method, paging query device, electronic equipment and computer-readable storage medium | |
| CN105338107A (en) | Stronghold operation synchronous management system and stronghold operation synchronous management method | |
| CN117874033A (en) | Database sub-table method and device, electronic equipment and storage medium | |
| CN107277095B (en) | Session segmentation method and device | |
| CN110968615A (en) | Data query method and device | |
| CN113434492B (en) | Data detection method and device, storage medium and electronic device | |
| CN116010340A (en) | Data table management method and device | |
| CN107070987B (en) | Data acquisition method and system for distributed object storage system | |
| CN103870478B (en) | A kind of method and device for restoring synchrodata | |
| CN114817396A (en) | Data synchronization method, device, equipment and storage medium | |
| CN101159775B (en) | Hardware module testing system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220412 |
|
| RJ01 | Rejection of invention patent application after publication |