CN106709005B - Method, device and system for processing redundant index in database system - Google Patents
Method, device and system for processing redundant index in database system Download PDFInfo
- Publication number
- CN106709005B CN106709005B CN201611208693.9A CN201611208693A CN106709005B CN 106709005 B CN106709005 B CN 106709005B CN 201611208693 A CN201611208693 A CN 201611208693A CN 106709005 B CN106709005 B CN 106709005B
- Authority
- CN
- China
- Prior art keywords
- index
- redundant
- indexes
- database
- database system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
Abstract
The invention discloses a method, a device and a system for processing redundant indexes in a database system. The method of the invention comprises the following steps: acquiring indexes of databases deployed on database servers in a database system, and storing the indexes in an index collection library; analyzing indexes in the index collection library, determining redundant indexes in the indexes, and storing the determined redundant index information into a redundant index library; and processing the redundant index in the database system according to the content in the redundant index library. The technical scheme can realize automatic analysis and determination of the redundant index, ensure the processing efficiency of the redundant index, release the disk space occupied by the redundant index and ensure the speed of data query.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method, a device and a system for processing redundant indexes in a database system.
Background
A database is a repository that organizes, stores, and manages data according to a data structure. With the demand of users for data storage and management, various databases, such as Mysql database, Oracle database, MongoDB database, etc., are emerging. The MongoDB database is a database based on distributed file storage, and has the biggest characteristic that the MongoDB database can realize most functions of single-table query of similar relational databases, and also supports the establishment of indexes on data, so that the speed of data access can be increased by using the indexes in the process of querying data from a large amount of data. Therefore, the MongoDB database is very sensitive to indexes, and data query depends on the indexes, but if the indexes are too many, particularly if the redundant indexes are too many, a large amount of disk space is occupied, and the speed of data query is influenced.
Disclosure of Invention
In view of the above, the present invention has been made to provide a method of handling redundant indexes in a database system and a corresponding apparatus, system that overcome or at least partially solve the above problems.
In accordance with one aspect of the present invention, there is provided a method of processing redundant indexes in a database system, comprising:
acquiring indexes of databases deployed on database servers in a database system, and storing the indexes in an index collection library;
analyzing the indexes in the index collection library, determining redundant indexes in the indexes, and storing the determined redundant index information into a redundant index library;
and processing the redundant index in the database system according to the content in the redundant index database.
Optionally, the obtaining an index of a database deployed on each database server in the database system, and storing the index into the index collection library includes:
receiving indexes of all data sets in a database deployed on each database server, which are sent by index scanning clients deployed on each database server in a database system;
and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
Optionally, the analyzing the indexes in the index collection library, and determining the redundant indexes therein includes:
acquiring data access information of the database system, and determining an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
Optionally, the analyzing the indexes in the index collection library, and determining the redundant indexes therein includes:
two indexes used by the same data access and belonging to the same data set, if one of the indexes is a subset of the other index, the index as the subset is determined to be a redundant index.
Optionally, the processing the redundant index in the database system according to the content in the redundant index library includes:
and deleting the redundant index in the database system according to the content in the redundant index database.
Optionally, the processing the redundant index in the database system according to the content in the redundant index library includes:
transmitting the content in the redundant index database to a corresponding service party using the database system;
and deleting or reserving the redundant index in the database system according to the processing opinion fed back by the corresponding service party.
Optionally, the processing the redundant index in the database system according to the content in the redundant index library further includes:
generating one or more execution statements for processing the redundant indexes for a data set containing the redundant indexes according to the content in the redundant index database;
displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface;
and receiving an instruction for triggering the corresponding execution statement through the redundant index display interface, and executing the corresponding execution statement.
According to another aspect of the present invention, there is provided an apparatus for processing a redundant index in a database system, comprising:
the index acquisition unit is suitable for acquiring indexes of databases deployed on database servers in the database system and storing the indexes into an index collection library;
the storage unit is suitable for storing an index collection library and a redundant index library;
the redundant index determining unit is suitable for analyzing the indexes in the index collection library, determining the redundant indexes in the index collection library and storing the determined redundant index information into a redundant index library;
and the redundant index processing unit is suitable for processing the redundant index in the database system according to the content in the redundant index database.
Optionally, the index obtaining unit is adapted to receive indexes of all data sets in the database deployed on each database server, which are sent by an index scanning client deployed on each database server in the database system; and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
Optionally, the redundant index determining unit is adapted to obtain data access information of the database system, and determine an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
Optionally, the redundant index determining unit is adapted to determine, when two indexes belonging to the same data set and used by the same data access are a subset of one index, that index as the subset is a redundant index.
Optionally, the redundant index processing unit is adapted to delete a redundant index in the database system according to the content in the redundant index library.
Optionally, the redundant index processing unit is adapted to send the content in the redundant index database to a corresponding service party using the database system; and deleting or reserving the redundant index in the database system according to the processing opinion fed back by the corresponding service party.
Optionally, the redundant index processing unit is adapted to generate one or more execution statements for processing the redundant index for the data set including the redundant index according to the content in the redundant index library; displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface; and receiving an instruction for triggering the corresponding execution statement through the redundant index display interface, and executing the corresponding execution statement.
According to still another aspect of the present invention, there is provided a system for processing a redundant index in a database system, wherein the system includes: a plurality of index scanning clients respectively deployed on each database server, and means for processing redundant indexes in a database system as described above;
each index scanning client is suitable for scanning indexes of all data sets in a database deployed on the database server and sending the indexes to the device for processing the redundant indexes in the database system.
According to the technical scheme of the invention, the indexes of the databases deployed on the database servers in the database system are obtained, and all the obtained indexes are stored in the index collection library; automatically analyzing and determining redundant indexes in the indexes stored in the index collection base, and storing the determined redundant index information in a redundant index base; and then processing the corresponding redundant index in the database system according to the content in the redundant index library. In conclusion, the invention can realize the automatic analysis and determination of the redundant index, ensure the processing efficiency of the redundant index, release the disk space occupied by the redundant index and ensure the speed of data query.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 illustrates a flow diagram of a method of processing redundant indexes in a database system in accordance with one embodiment of the present invention;
FIG. 2 is a block diagram illustrating an apparatus for processing redundant indexes in a database system according to an embodiment of the present invention;
FIG. 3 illustrates a block diagram of a system for handling redundant indexes in a database system, according to one embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
FIG. 1 illustrates a flow diagram of a method of processing redundant indexes in a database system in accordance with one embodiment of the present invention. As shown in fig. 1, the method includes:
step S110, an index of a database deployed on each database server in the database system is obtained and stored in the index collection library.
The MongoDB database can contain a plurality of data sets, when the indexes of the databases deployed on the database servers in the database system are obtained, all the indexes are obtained by traversing the indexes of all the data sets in the MongoDB databases deployed on the database servers in the database system, and all the indexes are stored in an index collection library for totaling, so that redundant indexes can be determined from all the indexes.
Step S120, the indexes in the index collection library are analyzed, the redundant indexes in the index collection library are determined, and the determined redundant index information is stored in the redundant index library.
All indexes stored in the index collection library are analyzed to determine which indexes are redundant indexes. On one hand, the redundant index can be determined according to the access information of a user during data query, and the index which is not used and the index which is used for a few times is regarded as the redundant index; on the other hand, if there are two indexes in a piece of data of a data set, and one of the indexes is a subset of the other index, then this index as a subset is also considered a redundant index. Because a user can query a certain piece of data in the corresponding data set through the two indexes, one index can satisfy all query conditions in the indexes as the subsets, and the indexes as the subsets exist in a redundant manner and need to be processed as redundant indexes. For example, two indexes of a data set are { id } and { id, name }, when a user queries a piece of data in the data set through id, the piece of data can be queried through both the indexes, but when the user queries through id and name, the piece of data can only be queried through the index { id, name }, that is, the indexes { id, name } can satisfy both the query according to a single condition of id and the query according to two conditions of id and name, so that the index { id } is redundant. Here, the index { id } is a subset of the index { id, name }, and the index { id } is a redundant index.
After the redundant indexes are determined, the redundant indexes are stored in a redundant index database for facilitating the processing of the redundant indexes.
The step can complete the automatic analysis and determination of the redundant index, and ensure the processing efficiency of the redundant index.
Step S130, according to the content in the redundant index database, processing the redundant index in the database system, and further releasing the disk space occupied by the redundant index, and ensuring the query speed.
When the redundant indexes in the redundant index database are processed, or the redundant indexes in the corresponding database data set are directly deleted; or the redundant indexes are displayed to a service party, and the processing decision of the service party on the redundant indexes is received, for example, the redundant indexes need to be reserved, the redundant indexes can be deleted, the redundant indexes need to be deleted after a period of time, and the like.
In conclusion, the invention can realize the automatic analysis and determination of the redundant index, has higher processing efficiency of the redundant index, releases the wasted disk space and ensures the speed of data query.
In an embodiment of the present invention, the obtaining the index of the database deployed on each database server in the database system in step S110, and storing the index into the index collection library includes: receiving indexes of all data sets in a database deployed on each database server, which are sent by index scanning clients deployed on each database server in a database system; and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
When the index of the database deployed on each database server in the database system is obtained, the database cannot be scanned and obtained by itself, so that a corresponding index scanning client needs to be deployed in each database, for example, a script is added to the database as a scanning tool. After the index scanning client finishes scanning, the index obtained by scanning is sent to an analysis center machine for processing redundant indexes in the database system, and the analysis center machine can be used as a corresponding device or a server. When the redundant indexes are determined and processed, the data sets corresponding to the redundant indexes are received, the data set information corresponding to the indexes is received while the indexes sent by the scanning client side are received, and when the data set information is stored, the received indexes are classified and stored in the index collection base by taking the data sets to which the indexes belong as units. For example, the form of the save in the index collection library is: data set 1: { index 1}, { index 2}, and { index 3 }; data set 2: { index a }, { index b }, { index c }, and { index d }; ....
In an embodiment of the present invention, analyzing the indexes in the index collection library in step S120, and determining the redundant indexes therein includes: acquiring data access information of a database system, and determining an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
In this embodiment, an unused index is determined as a redundant index according to data access information, when a user accesses data, a record or log of corresponding access information is stored, the access information includes a used corresponding index, the number of times of use of each index is determined from the access information, and an index with the number of times of use of 0 is determined as the redundant index; or, a preset value is set, and when the index with the use times smaller than the preset value is also determined as the redundant index. For example, if the preset value is set to 3 times, the index with the use number less than 3 times is used as the redundant index.
It should be noted that, when the preset value is set, a corresponding time period may also be set, for example, the total number of times of use after the index is established, or the number of times of use of the index in a period of time.
In an embodiment of the present invention, analyzing the indexes in the index collection library in step S120, and determining the redundant indexes therein includes: two indexes used by the same data access and belonging to the same data set, if one of the indexes is a subset of the other index, the index as the subset is determined to be a redundant index.
Because the user can query the data in the corresponding data set through the two indexes, one index can satisfy all query conditions in the index as the subset, and the index as the subset is redundant and needs to be processed as a redundant index. For example, two indexes of a data set are { id, name } and { id, name, birthday }, and when a user queries a piece of data in the data set through id and name, the piece of data can be queried through both indexes, but when the user queries the data through id, name and birthday comprehensively, the piece of data can only be queried through the indexes { id, name, birthday }. That is, the index { id, name, birthday } can satisfy both the query based on two conditions of id and name and the query based on three conditions of id, name and birthday, so that the index { id, name } is redundant. Here, the index { id, name } is a subset of the index { id, name, birthday } and the index { id, name } is a redundant index.
Therefore, in this embodiment, when determining whether a certain index is a redundant index, two indexes that are used by the same data access and belong to the same data set may be queried according to the access information, and whether there is a case where one of the indexes is a subset of the other index, and if so, the index that is the subset is the redundant index.
In an embodiment of the present invention, the processing the redundant index in the database system according to the content in the redundant index library in step S130 includes: and deleting the redundant indexes in the database system according to the contents in the redundant index library so as to release the disk space.
After the redundant index library is obtained, the redundant index deletion operation should not be performed immediately, and since the data query is for the corresponding service requirement and the index is also established by the service party, the permission of the service party needs to be obtained before the deletion operation is performed on the index. Therefore, in an embodiment of the present invention, the processing of the redundant index in the database system according to the content in the redundant index library in step S130 includes: transmitting the content in the redundant index database to a corresponding service party using the database system; and deleting or reserving the redundant indexes in the database system according to the processing opinions fed back by the corresponding service party. Therefore, the service party can delete some indexes according to service requirements and can also delete all indexes, and the use experience of the service party is not influenced.
However, those skilled in the art should appreciate that the above-mentioned manner in which the redundant index can be deleted after the business party confirms is not a limitation of the present invention. Because the redundant index is used frequently and has limited influence on the service, after the redundant index is determined and stored in the redundant index database, the operation of deleting can be directly carried out without the approval of a service end.
In order to facilitate the operation of the operator or the service party, in an embodiment of the present invention, the processing the redundant index in the database system according to the content in the redundant index database in step S130 further includes: generating one or more execution statements for processing the redundant indexes for the data set containing the redundant indexes according to the content in the redundant index database; displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface, for example, adding a 'delete' button and a 'keep' button for each displayed redundant index; when an operator or a service party presses a corresponding button, an instruction for triggering a corresponding execution statement is received through the redundant index display interface, and the corresponding execution statement is executed, so that the automation of redundant index processing is realized to the maximum extent.
For example, when receiving the redundant index library, the service side needs to inform the operator which redundant indexes are deleted and which redundant indexes are reserved, and needs to edit corresponding files to inform the operator. Then, through this embodiment, the service party can select the corresponding button, and the processing work of the redundant index can be completed.
Fig. 2 is a schematic structural diagram illustrating an apparatus for processing redundant indexes in a database system according to an embodiment of the present invention. As shown in fig. 2, the apparatus 200 for processing redundant indexes in a database system includes:
the index obtaining unit 210 is adapted to obtain an index of a database deployed on each database server in the database system, and store the index into the index collection library.
The MongoDB database can contain a plurality of data sets, when the indexes of the databases deployed on the database servers in the database system are obtained, all the indexes of traversal are obtained through the indexes of all the data sets in the MongoDB databases deployed on the database servers in the database system, and all the indexes are stored in an index collection library for totaling, so that redundant indexes can be determined from all the indexes.
The storage unit 220 is adapted to store an index collection library and a redundant index library.
The redundant index determining unit 230 is adapted to analyze the indexes in the index collection library, determine the redundant indexes therein, and store the determined redundant index information in the redundant index library.
All indexes stored in the index collection library are analyzed to determine which indexes are redundant indexes. On one hand, the redundant index can be determined according to the access information of a user during data query, and the index which is not used and the index which is used for a few times are both regarded as the redundant index; on the other hand, if there are two indexes in a piece of data of a data set, and one of the indexes is a subset of the other index, then this index as a subset is also considered a redundant index. Because a user can query a certain piece of data in the corresponding data set through the two indexes, one index can satisfy all query conditions in the indexes as the subsets, and the indexes as the subsets exist in a redundant manner and need to be processed as redundant indexes. For example, two indexes of a data set are { id } and { id, name }, when a user queries a piece of data in the data set through id, the piece of data can be queried through both the indexes, but when the user queries through id and name, the piece of data can only be queried through the index { id, name }, that is, the indexes { id, name } can satisfy both the query according to a single condition of id and the query according to two conditions of id and name, so that the index { id } is redundant. Here, the index { id } is an index that is a subset of the index { id, name }, and the index { id } is a redundant index.
After the redundant indexes are determined, the redundant indexes are stored in a redundant index database for facilitating the processing of the redundant indexes.
The unit can complete the automatic analysis and determination of the redundant index, and the processing efficiency of the redundant index is ensured.
The redundant index processing unit 240 is adapted to process the redundant indexes in the database system according to the contents in the redundant index library, so as to release the disk space occupied by the redundant indexes and ensure the query speed
When the redundant indexes in the redundant index database are processed, or the redundant indexes in the corresponding database data set are directly deleted; or the redundant indexes are displayed to a service party, and the processing decision of the service party on the redundant indexes is received, for example, the redundant indexes need to be reserved, the redundant indexes can be deleted, the redundant indexes need to be deleted after a period of time, and the like.
In conclusion, the invention can realize the automatic analysis and determination of the redundant index, has higher processing efficiency of the redundant index, releases the wasted disk space and ensures the speed of data query.
In an embodiment of the present invention, the index obtaining unit 210 is adapted to receive indexes of all data sets in the database deployed on each database server, which are sent by the index scanning client deployed on each database server in the database system; and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
When the index of the database deployed on each database server in the database system is obtained, the database cannot be scanned and obtained by itself, so that a corresponding index scanning client needs to be deployed in each database, for example, a script is added to the database as a scanning tool. After the index scanning client finishes scanning, the index obtained by scanning is sent to an analysis center machine for processing redundant indexes in the database system, and the analysis center machine can be used as a corresponding device or a server. When the redundant indexes are determined and processed, the data sets corresponding to the redundant indexes are received, the data set information corresponding to the indexes is received while the indexes sent by the scanning client side are received, and when the data set information is stored, the received indexes are classified and stored in the index collection base by taking the data sets to which the indexes belong as units. For example, the form of the save in the index collection library is: data set 1: { index 1}, { index 2}, and { index 3 }; data set 2: { index a }, { index b }, { index c }, and { index d }; ....
In an embodiment of the present invention, the sub-redundant index determining unit 230 is adapted to obtain data access information of the database system, and determine an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
In this embodiment, an unused index is determined as a redundant index according to data access information, when a user accesses data, a record or log of corresponding access information is stored, the access information includes a used corresponding index, the number of times of use of each index is determined from the access information, and an index with the number of times of use of 0 is determined as the redundant index; or, a preset value is set, and when the index with the use times smaller than the preset value is also determined as the redundant index. For example, if the preset value is set to 3 times, the index with the use number less than 3 times is used as the redundant index.
It should be noted that, when the preset value is set, a corresponding time period may also be set, for example, the total number of times of use after the index is established, or the number of times of use of the index in a period of time.
In an embodiment of the present invention, the remaining index determining unit 230 is adapted to, when two indexes belonging to the same data set and used by the same data access are included, determine that one of the indexes is a subset of the other index as a redundant index.
Because the user can query the data in the corresponding data set through the two indexes, one index can satisfy all query conditions in the index as the subset, and the index as the subset is redundant and needs to be processed as a redundant index. For example, two indexes of a data set are { id, name } and { id, name, birthday }, and when a user queries a piece of data in the data set through id and name, the piece of data can be queried through both indexes, but when the user queries the data through id, name and birthday comprehensively, the piece of data can only be queried through the indexes { id, name, birthday }. That is, the index { id, name, birthday } can satisfy both the query based on two conditions of id and name and the query based on three conditions of id, name and birthday, so that the index { id, name } is redundant. Here, the index { id, name } is an index that is a subset of the index { id, name, birthday } and the index { id, name } is a redundant index.
Therefore, in this embodiment, when determining whether a certain index is a redundant index, two indexes that are used by the same data access and belong to the same data set may be queried according to the access information, and whether there is a case where one of the indexes is a subset of the other index, and if so, the index that is the subset is the redundant index.
In an embodiment of the present invention, the redundant index processing unit is adapted to perform deletion processing on the redundant index in the database system according to the content in the redundant index library, so as to release the disk space.
After the redundant index library is obtained, the redundant index deletion operation should not be performed immediately, and since the data query is for the corresponding service requirement and the index is also established by the service party, the permission of the service party needs to be obtained before the deletion operation is performed on the index. Therefore, in one embodiment of the present invention, the redundant index processing unit 240 is adapted to transmit the contents in the redundant index database to the corresponding service party using the database system; and deleting or reserving the redundant indexes in the database system according to the processing opinions fed back by the corresponding service party. Therefore, the service party can delete some indexes according to service requirements and can also delete all indexes, and the use experience of the service party is not influenced.
However, those skilled in the art should appreciate that the above-mentioned manner in which the redundant index can be deleted after the business party confirms is not a limitation of the present invention. Because the redundant index is used frequently and has limited influence on the service, after the redundant index is determined and stored in the redundant index database, the operation of deleting can be directly carried out without the approval of a service end.
In order to facilitate the operation of the operator or the business side, in an embodiment of the present invention, the redundant index processing unit 240 is adapted to generate one or more execution statements for processing the redundant index for the data set containing the redundant index according to the content in the redundant index library; displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface, for example, adding a 'delete' button and a 'keep' button for each displayed redundant index; when an operator or a service party presses a corresponding button, an instruction for triggering a corresponding execution statement is received through the redundant index display interface, and the corresponding execution statement is executed, so that the automation of redundant index processing is realized to the maximum extent.
For example, when receiving the redundant index library, the service side needs to inform the operator which redundant indexes are deleted and which redundant indexes are reserved, and needs to edit corresponding files to inform the operator. Then, through this embodiment, the service party can select the corresponding button, and the processing work of the redundant index can be completed. .
FIG. 3 illustrates a block diagram of a system for handling redundant indexes in a database system, according to one embodiment of the present invention. As shown in FIG. 3, the system 300 for processing redundant indexes in a database system includes: an index scanning client 310, an index scanning client 320, an index scanning client 330, and a device 340 for processing redundant indexes in a database system as shown in fig. 2, which are respectively disposed on each database server.
The index scanning client 310, the index scanning client 320 and the index scanning client 330 are all adapted to scan indexes of all data sets in the database deployed on the database server and send the indexes to a device for processing redundant indexes in the database system.
It should be noted that the system shown in fig. 3 corresponds to the same method shown in fig. 1, and the above has been described in detail, and is not repeated herein.
In summary, according to the technical solution of the present invention, indexes of databases deployed on database servers in a database system are obtained, and all the obtained indexes are stored in an index collection library; automatically analyzing and determining redundant indexes in the indexes stored in the index collection base, and storing the determined redundant index information in a redundant index base; and then processing the corresponding redundant index in the database system according to the content in the redundant index library. Therefore, the invention can realize the automatic analysis and determination of the redundant index, ensure the processing efficiency of the redundant index, release the disk space occupied by the redundant index and ensure the speed of data query.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the apparatus and system for processing redundant indexes in a database system according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (13)
1. A method of processing redundant indexes in a database system, wherein the method comprises:
acquiring indexes of databases deployed on database servers in a database system, and storing the indexes in an index collection library;
analyzing the indexes in the index collection library, determining redundant indexes in the indexes, and storing the determined redundant index information into a redundant index library;
processing the redundant index in the database system according to the content in the redundant index library;
the analyzing the indexes in the index collection library and determining the redundant indexes comprises:
two indexes used by the same data access and belonging to the same data set, if one of the indexes is a subset of the other index, the index as the subset is determined to be a redundant index.
2. The method of claim 1, wherein the obtaining and saving the index of the database deployed on each database server in the database system to the index collection repository comprises:
receiving indexes of all data sets in a database deployed on each database server, which are sent by index scanning clients deployed on each database server in a database system;
and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
3. The method of claim 1, wherein analyzing the indexes in the index collection library to determine redundant indexes therein comprises:
acquiring data access information of the database system, and determining an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
4. The method of claim 1, wherein said processing redundant indexes in said database system based on contents in said redundant index repository comprises:
and deleting the redundant index in the database system according to the content in the redundant index database.
5. The method of claim 1, wherein said processing redundant indexes in said database system based on contents in said redundant index repository comprises:
transmitting the content in the redundant index database to a corresponding service party using the database system;
and deleting or reserving the redundant index in the database system according to the processing opinion fed back by the corresponding service party.
6. The method of claim 1, wherein said processing redundant indexes in said database system based on contents in said redundant index repository further comprises:
generating one or more execution statements for processing the redundant indexes for a data set containing the redundant indexes according to the content in the redundant index database;
displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface;
and receiving an instruction for triggering the corresponding execution statement through the redundant index display interface, and executing the corresponding execution statement.
7. An apparatus for processing redundant indexes in a database system, wherein the apparatus comprises:
the index acquisition unit is suitable for acquiring indexes of databases deployed on database servers in the database system and storing the indexes into an index collection library;
the storage unit is suitable for storing an index collection library and a redundant index library;
the redundant index determining unit is suitable for analyzing the indexes in the index collection library, determining the redundant indexes in the index collection library and storing the determined redundant index information into a redundant index library;
the redundant index processing unit is suitable for processing the redundant index in the database system according to the content in the redundant index database;
the redundant index determining unit is suitable for determining that the index as the subset is a redundant index if one index is a subset of the other index in two indexes which are used by the same data access and belong to the same data set.
8. The apparatus of claim 7, wherein,
the index acquisition unit is suitable for receiving indexes of all data sets in the databases deployed on the database servers, which are sent by the index scanning client deployed on the database servers in the database system; and classifying and storing the received indexes into an index collection library by taking the affiliated data set as a unit.
9. The apparatus of claim 7, wherein,
the redundant index determining unit is suitable for acquiring data access information of the database system and determining an unused index as a redundant index according to the data access information; or, determining an index with the number of times of use lower than a preset value as a redundant index according to the data access information.
10. The apparatus of claim 7, wherein,
and the redundant index processing unit is suitable for deleting the redundant index in the database system according to the content in the redundant index database.
11. The apparatus of claim 7, wherein,
the redundant index processing unit is suitable for sending the content in the redundant index database to a corresponding service party using the database system; and deleting or reserving the redundant index in the database system according to the processing opinion fed back by the corresponding service party.
12. The apparatus of claim 7, wherein,
the redundant index processing unit is suitable for generating one or more execution statements for processing the redundant indexes for the data set containing the redundant indexes according to the content in the redundant index database; displaying redundant index information of a data set containing redundant indexes and corresponding execution statements on a redundant index display interface; and receiving an instruction for triggering the corresponding execution statement through the redundant index display interface, and executing the corresponding execution statement.
13. A system for processing redundant indexes in a database system, wherein the system comprises: a plurality of index scanning clients respectively deployed on each database server, and means for processing redundant indexes in a database system according to any one of claims 7 to 12;
each index scanning client is suitable for scanning indexes of all data sets in a database deployed on the database server and sending the indexes to the device for processing the redundant indexes in the database system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611208693.9A CN106709005B (en) | 2016-12-23 | 2016-12-23 | Method, device and system for processing redundant index in database system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611208693.9A CN106709005B (en) | 2016-12-23 | 2016-12-23 | Method, device and system for processing redundant index in database system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106709005A CN106709005A (en) | 2017-05-24 |
| CN106709005B true CN106709005B (en) | 2020-11-24 |
Family
ID=58895650
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611208693.9A Active CN106709005B (en) | 2016-12-23 | 2016-12-23 | Method, device and system for processing redundant index in database system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106709005B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101059810A (en) * | 2007-03-16 | 2007-10-24 | 华为技术有限公司 | System and method for implementing automatic optimization of data base system |
| CN101079034A (en) * | 2006-07-10 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for eliminating redundancy file of file storage system |
| CN101101574A (en) * | 2006-07-06 | 2008-01-09 | 国际商业机器公司 | Method and system to manage hierarchical tables in array |
| CN101286160A (en) * | 2008-05-30 | 2008-10-15 | 同济大学 | Data base indexing process |
| CN101299213A (en) * | 2008-06-17 | 2008-11-05 | 中国地质大学（武汉） | N-dimension clustering order recording tree space index method |
| CN101382949A (en) * | 2008-10-28 | 2009-03-11 | 阿里巴巴集团控股有限公司 | Management method for database table and apparatus |
| CN101901237A (en) * | 2009-12-04 | 2010-12-01 | 四川川大智胜软件股份有限公司 | Type-Index-Value distributed database built based on SQIite |
| CN101950297A (en) * | 2010-09-10 | 2011-01-19 | 北京大学 | Method and device for storing and inquiring mass semantic data |
| CN102915380A (en) * | 2012-11-19 | 2013-02-06 | 北京奇虎科技有限公司 | Method and system for carrying out searching on data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8131709B2 (en) * | 2005-01-14 | 2012-03-06 | International Business Machines Corporation | Mechanism for providing efficient access to redundant number representations |
-
2016
- 2016-12-23 CN CN201611208693.9A patent/CN106709005B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101101574A (en) * | 2006-07-06 | 2008-01-09 | 国际商业机器公司 | Method and system to manage hierarchical tables in array |
| CN101079034A (en) * | 2006-07-10 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for eliminating redundancy file of file storage system |
| CN101059810A (en) * | 2007-03-16 | 2007-10-24 | 华为技术有限公司 | System and method for implementing automatic optimization of data base system |
| CN101286160A (en) * | 2008-05-30 | 2008-10-15 | 同济大学 | Data base indexing process |
| CN101299213A (en) * | 2008-06-17 | 2008-11-05 | 中国地质大学（武汉） | N-dimension clustering order recording tree space index method |
| CN101382949A (en) * | 2008-10-28 | 2009-03-11 | 阿里巴巴集团控股有限公司 | Management method for database table and apparatus |
| CN101901237A (en) * | 2009-12-04 | 2010-12-01 | 四川川大智胜软件股份有限公司 | Type-Index-Value distributed database built based on SQIite |
| CN101950297A (en) * | 2010-09-10 | 2011-01-19 | 北京大学 | Method and device for storing and inquiring mass semantic data |
| CN102915380A (en) * | 2012-11-19 | 2013-02-06 | 北京奇虎科技有限公司 | Method and system for carrying out searching on data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106709005A (en) | 2017-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| US9805022B2 (en) | Generation of topic-based language models for an app search engine | |
| CN104239395B (en) | The method and system of search | |
| KR101475335B1 (en) | Enhancing an inquiry for a search of a database | |
| CN107861981B (en) | Data processing method and device | |
| CN110737689B (en) | Data standard compliance detection method, device, system and storage medium | |
| US20130144867A1 (en) | Database processing method, database processing system and database server | |
| CN112232757A (en) | Work order processing method and device based on mail, electronic equipment and medium | |
| WO2015143910A1 (en) | Method and device for defining search engine result pages by user | |
| CN105528218A (en) | Data diagram cascaded method and data diagram cascaded system | |
| CN108734566A (en) | Collage-credit data querying method, terminal device and medium | |
| US9569465B2 (en) | Image processing | |
| CA2850883A1 (en) | Image processing | |
| CN106709005B (en) | Method, device and system for processing redundant index in database system | |
| US11481377B2 (en) | Compute-efficient effective tag determination for data assets | |
| CN110909072B (en) | Data table establishment method, device and equipment | |
| US20120323960A1 (en) | Automatic contact list aliasing in a collaboration system | |
| CN110717109B (en) | Method, device, electronic equipment and storage medium for recommending data | |
| CN113779403A (en) | Book information acquisition method, book information display method, server and user terminal | |
| CN108897873B (en) | Method and device for generating job file, storage medium and processor | |
| US20160232166A1 (en) | Method and Apparatus for Accessing File | |
| CN110019448B (en) | Data interaction method and device | |
| CN109344145B (en) | Data standard specification-based data cleaning method, device and system | |
| KR102214185B1 (en) | Resource Retrieval Method | |
| CN111159435A (en) | Multimedia resource processing method, system, terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |