CN113312431A - Identification method of database associated elements and establishment method of data structure diagram - Google Patents

Identification method of database associated elements and establishment method of data structure diagram Download PDF

Info

Publication number
CN113312431A
CN113312431A CN202010462483.2A CN202010462483A CN113312431A CN 113312431 A CN113312431 A CN 113312431A CN 202010462483 A CN202010462483 A CN 202010462483A CN 113312431 A CN113312431 A CN 113312431A
Authority
CN
China
Prior art keywords
sql
target
vertex
elements
database instance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010462483.2A
Other languages
Chinese (zh)
Inventor
于晨昕
张平
梁高中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN202010462483.2A priority Critical patent/CN113312431A/en
Publication of CN113312431A publication Critical patent/CN113312431A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/288Entity relationship models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method for identifying database associated elements and a method for establishing a data structure diagram. Wherein, the method comprises the following steps: determining a target element to be detected; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the SQL element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements; and outputting the associated element corresponding to the target element. The invention solves the technical problem that the scheme for effectively identifying the related SQL request does not exist in the related technology.

Description

Identification method of database associated elements and establishment method of data structure diagram
Technical Field
The invention relates to the field of databases, in particular to a method for identifying database associated elements and a method for establishing a data structure diagram.
Background
Database Autonomous Service (DAS) provides self-optimizing services for various Database platforms, and helps users to optimize Database performance. The SQL diagnosis optimization service is one of core functions of the DAS for providing database performance optimization. The SQL diagnostic service of DAS generates optimization recommendations for SQL. The optimization suggestions comprise index addition and deletion suggestions, sentence rewriting optimization suggestions and the like. Slow SQL optimization is an application scenario of SQL optimization services, and for optimization of multiple slow SQL requests (SQL requests are also called SQL statements or SQL queries), a large number of simultaneous indexes may be generated, and execution plans of other SQL requests may be affected. Therefore, the index proposal of an SQL request needs to consider related SQL requests and perform de-duplication integration on the index set of the related SQL requests. In addition, the automatic optimization service is an autonomous service provided by the DAS, the automatic optimization can automatically create indexes for database examples according to index suggestions, perform performance tracking in time after creation is finished, and perform rollback in time after performance regression is found. In the tracking process, besides tracking the optimized SQL, the performance of the related SQL is also required to be tracked. Thus, it is necessary to identify the relevant SQL requests.
However, in the related art, there is no technical means for identifying related SQL requests, which affects the optimization efficiency of SQL.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a method for identifying database associated elements and a method for establishing a data structure diagram, which at least solve the technical problem that an effective scheme for identifying related SQL requests does not exist in the related technology.
According to an aspect of the embodiments of the present invention, there is provided a method for identifying an association element in a database instance, including: determining a target element to be detected, wherein the target element is an SQL element in an SQL database; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements; and outputting the associated element corresponding to the target element.
According to another aspect of the embodiments of the present invention, there is also provided a method for establishing a data structure diagram, including: acquiring at least one SQL template in the database instance; analyzing at least one SQL template to obtain a first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template; generating a data structure diagram based on at least the constituent elements and the associated elements associated with the constituent elements, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements.
According to another aspect of the embodiments of the present invention, there is also provided a method for identifying an associated element in a database instance, including: receiving an SQL request, and acquiring a target element in a database instance from the SQL request; analyzing the target element based on the data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertices in the vertex set comprise: vertices for representing different SQL requests and vertices for representing SQL elements in a database instance, the set of edges including: an edge for representing an incidence relation between different SQL requests and SQL elements in the database instance, and an edge for representing an incidence relation between the SQL elements; and outputting the associated element corresponding to the target element.
According to another aspect of the present invention, there is also provided a method for optimizing elements in a database instance, where the method includes: receiving an optimization request from a target object, wherein the optimization request carries a target element to be optimized, and the target element is an SQL element in an SQL database; acquiring an association element having an association relation with a target element; and optimizing the indexes of the target elements and the indexes of the associated elements.
According to another aspect of the embodiments of the present invention, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the method for identifying the associated element in any database instance.
According to another aspect of the embodiments of the present invention, there is also provided a computing device, including: a processor; and a memory coupled to the processor for providing instructions to the processor to process the following process steps: determining a target element to be detected in a database instance, wherein the target element is an SQL element in an SQL database; analyzing the target element based on a data structure diagram of the database embodiment to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the steps that a vertex set and an edge set are included, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements; and outputting the association element corresponding to the target element.
In the embodiment of the present invention, a method of identifying related SQL is adopted, first, a target element to be detected in a database instance is determined, and then, the target element is analyzed based on a data structure diagram of the database instance to obtain an associated element associated with the target element, where the data structure diagram includes: the method comprises a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, the edge in the edge set is used for representing the incidence relation between the SQL elements, and finally, the incidence element corresponding to a target element is output, so that the purpose of identifying related SQL is achieved, powerful support is provided for an SQL optimization process, and the technical problem that an effective scheme for identifying related SQL requests does not exist in the related technology is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a block diagram of a hardware architecture of a computer terminal for a method of identifying associated elements in a database instance according to an embodiment of the present invention;
FIG. 2a is a flow chart illustrating a method for identifying associated elements in a database instance according to an embodiment of the present invention;
FIG. 2b is a schematic diagram of an alternative main flow framework for obtaining relational SQL for database embodiments, according to an embodiment of the invention;
FIG. 2c is a schematic diagram of an alternative build graph data model according to embodiments of the invention;
FIG. 3 is a flow chart illustrating a method for establishing a data structure according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for optimizing elements in a database instance, according to an embodiment of the invention;
FIG. 5 is a schematic structural diagram of an apparatus for identifying associated elements in a database example according to an embodiment of the present invention;
FIG. 6 is a block diagram of a computer device according to an embodiment of the present invention;
FIG. 7 is a flow chart illustrating a method for identifying associated elements in a database instance, implemented in accordance with the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
First, partial terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:
the DAS is a cloud Service which realizes self-perception, self-repair, self-optimization, self-operation and maintenance and self-safety of a Database based on machine learning and expert experience, helps a user to eliminate complexity of Database operation and maintenance and Service faults caused by manual operation, and effectively ensures stability, safety and high efficiency of Database Service.
SQL (Structured Query Language) is a special purpose programming Language used to manage a relational database management system (RDBMS), or stream processing in a relational stream data management system (RDSMS). SQL is based on relational algebra and tuple relational calculus, including a data definition language and a data manipulation language. SQL ranges include data insertion, query, update, and deletion, database schema creation and modification, and data access control.
T1.C1 is table abbreviation representing tables in the database, C is column abbreviation representing columns in the database, T1.C1 represents C1 column in the T1 table, and the meaning of other combinations in the application can be deduced therefrom.
Related SQL (related SQL) in the database domain, related SQL is a database instance, where multiple SQL queries are directed to the same target table or where predicates of multiple SQL points to the same column. In the actual execution process of the related SQL, the related SQL can affect each other and even cause lock waiting; related SQL may use the same or similar indexes. Adding index optimization to one of the SQL may affect the execution plans of the other SQL.
And (3) SQL automatic optimization, in the field of databases, in order to improve the overall performance of the databases, database manufacturers automatically recommend indexes for the databases, perform online operation at low peaks of business, and monitor the performance of the databases to perform quick rollback or gain calculation.
Graph calculation: the entities in the information and the relations among the entities are abstractly expressed into 'diagram' structure data of vertexes and edges among the vertexes respectively, and the relation analysis is carried out based on the data structure.
Example 1
There is also provided, in accordance with an embodiment of the present invention, a method embodiment for identification of associated elements in a database instance, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that illustrated herein.
The method provided by the embodiment one of the present application may be executed in a mobile terminal, a computer terminal or a similar operation device. Fig. 1 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing the identification method of the associated elements in the database instance. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit is controlled as a kind of processor (e.g. selection of a variable resistance termination path connected to the interface).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the identification method of the associated elements in the database example in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the above-mentioned vulnerability detection method of the application program. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission module 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).
Under the above operating environment, the present application provides a method for identifying associated elements in a database instance as shown in fig. 2 a. Fig. 2a is a flowchart of a method for identifying an association element in a database instance according to a first embodiment of the present invention.
As shown in fig. 2a, a method for identifying an association element in a database instance according to a first embodiment of the present invention may include the following steps:
step S202, determining a target element to be detected, wherein the target element is an SQL element in an SQL database;
step S204, analyzing the target element based on the data structure diagram of the database example to obtain the associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements;
in step S206, the associated element corresponding to the target element is output.
In the above embodiment of the present application, first, a target element to be detected is determined, and then, the target element is analyzed based on a data structure diagram of a database instance to obtain an associated element associated with the target element, where the data structure diagram includes: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database embodiment, the set of edges comprising at least: edges for representing associations between SQL elements; and finally, outputting the associated elements corresponding to the target elements, so that the purpose of identifying the related SQL is achieved, powerful support is provided for the SQL optimization process, and the technical problem that an effective scheme for identifying the related SQL request does not exist in the related technology is solved.
In some embodiments of the present application, the data structure diagram is determined by: acquiring at least one SQL template in a database instance; analyzing at least one SQL template to obtain a first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template; the data structure diagram is built based at least on the associated elements associated with the constituent elements. For example, the SQL template 1 is analyzed to obtain that the element of the SQL1 component includes a field T2.c1, and if there is a field T2.c2 in the element of the SQL template 2, then T2 is an associated element of the two templates, so the data structure diagram can be established according to the element T2, and it should be noted that the associated element at least includes but is not limited to: predicates, connection conditions, ordering fields, aggregation fields, projection fields, etc., for associations between SQL requests.
When the SQL template is obtained, there may be a plurality of implementation manners, for example: acquiring full SQL flow information in a database corresponding to a database instance; analyzing the full amount of SQL flow information to obtain a second analysis content, wherein the second analysis content can be an SQL statement in a time interval; formatting the second analysis content to obtain an initial SQL template, and then performing deduplication processing on the initial SQL template to obtain the SQL template. Specifically, the full amount of SQL running water information may be transmitted through the database kernel, the SQL running water information may be analyzed through real-time calculation and offline calculation, and the SQL template may be obtained by formatting the SQL, removing parameters, and then templating, where the SQL template may be the SQL template in the specified time interval of the database instance.
Wherein the vertices used to represent SQL elements in the database instance include the following vertices: a field vertex corresponding to the field in the database instance, and a table vertex corresponding to the table in the database instance; at this time, the edges used to represent the incidence relationship between the SQL elements include edges indicating the following relationship: the relationship between the fields; table vertices and field vertices, where SQL elements may be fields, tables, etc. in a database embodiment. Each vertex in the vertex set may include a type and an attribute of the vertex, for example, an SQL vertex includes information such as a vertex type and an SQL template unique identifier; in the edge set, the relationship between the table vertex and the field vertex may be an affiliation. Establishing a relation between the SQL request and the field, and establishing different types for the relation according to the type of the SQL request associated field, such as: predicate relationships, join field relationships, sort field relationships, aggregate field relationships, project field relationships, and the like.
In some optional embodiments, the table vertex and the field vertex may be further merged into a type of vertex to obtain a target vertex; and establishing a data structure diagram based on the incidence relation between the target vertexes.
In other optional embodiments, the vertex set further includes: vertexes corresponding to different SQL requests; and/or the following vertexes are also included in the vertex set: index vertex corresponding to the index of the SQL request; the edge set also comprises: the edge for indicating the relation between the table and the index and the edge for indicating the relation between the SQL request and the index are used for recording the relation between the table and the index by introducing the index vertex and constructing a data structure diagram of four types of vertexes.
In some optional embodiments of the present application, the data structure diagram may also be determined by: for example, the table vertex and the field vertex are combined into one type of vertex, and the data structure diagram of the two types of vertices is constructed.
In some embodiments of the present application, after determining the data structure diagram, in order to obtain the related SQL request, the related SQL request may be obtained in a graph calculation manner, for example, a unicom algorithm may be implemented, where the unicom algorithm has multiple implementation manners; existing graph computation methods provided by third parties may also be utilized, for example: connected Component vector Program supplied by Apache Tinker Pop.
In some embodiments of the present application, the manner of obtaining the related SQL request includes, but is not limited to, identifying the related SQL request by means of predicate string matching or by means of establishing a machine learning model by AI technology using the SQL request with correlation identified on the user side as training data.
In some embodiments of the present application, after determining the data structure diagram, based on the diagram calculation, related SQL requests of different types may be obtained. For example, given a table, obtain all SQL sets related to the table, give an SQL request, obtain an SQL set related to the SQL request, obtain related SQL according to the SQL request, may customize the set of related SQL, such as calculating a correlation coefficient according to the number of associated edges, according to the correlation coefficient, find the most relevant SQL request, given column, find related column, given table, find related table, and so on.
In the related art, although SQL Advisor of a commercial database manufacturer such as Microsoft SQL Server/Oracle/IBM DB2 has a function of optimizing a single SQL request, the influence of related SQL is not considered when diagnosing and optimizing a single SQL request, so that when diagnosing multiple SQL requests, duplicate or identical indexes are generated, and the generated indexes affect execution paths of other SQL requests, thereby resulting in low index accuracy and certain influence on execution efficiency of other SQL requests. The index refers to a data structure for helping MYSQL to efficiently acquire data, and the index can be divided into two types: the index management method comprises the steps of aggregating indexes and non-aggregating indexes, wherein the order of the aggregating indexes is the physical storage order of data, and the indexes and the data are stored in the same file; non-aggregated indexing: the sequence of the aggregation index is different from the physical storage sequence of the data, and the index and the data are stored in the same file.
In order to solve the above problem, the embodiment of the present application may optimize the related SQL request, and at this time, the step of determining the target element to be detected may be implemented in the following manner: receiving an optimization request from a target object, wherein the optimization request carries target elements; analyzing the target element based on a data structure diagram of the database instance, and acquiring a first index corresponding to the target element and an index set of associated elements associated with the target element after acquiring the associated elements associated with the target element; integrating and de-duplicating the first index and a second index in the index set, wherein the integrating of the indexes comprises: segment-based merging (e.g., merging by the number of segment documents) and subdirectory-based merging (merging the directory to be merged to the specified directory or merging the directory to be merged to the new directory). The deduplication refers to removing duplicate indexes, for example, when the target element is a field C1, the index information of the field includes ab and abc, the SQL element associated with the target element is a field C2, and the index information of the field C2 includes abc, abd and abe, then integration and deduplication can be performed, which removes the abc common to the two indexes, thereby making the retrieval more accurate. It should be noted that there may be a plurality of indexes of the associated elements associated with the target element.
It should be noted that the target elements include: a table corresponding to the database instance; the association elements corresponding to the target elements include: a set of SQL requests or tables associated with a table; and/or the target elements include: a column corresponding to the database instance; the association element corresponding to the target element includes: a set of columns or a set of SQL requests associated with the columns; and/or the target elements include: a target SQL request; the association element corresponding to the target element includes: a set of SQL requests associated with the target SQL request.
In some embodiments of the present application, the SQL request set associated with the target SQL request may also be determined by: determining the number of edges in the data structure diagram associated with the target SQL request to determine a correlation coefficient, wherein the edges associated with the target SQL request comprise: an edge passes through from a vertex corresponding to the target SQL request to a target associated vertex, wherein the target associated vertex is a vertex corresponding to the SQL request associated with the target SQL request; and determining the SQL request associated with the target SQL request according to the correlation coefficient, and storing the SQL request associated with the target SQL request to an SQL request set.
Fig. 2b is a main flowchart of an optional method for obtaining a related SQL request of a database instance according to an embodiment of the present invention, and as shown in fig. 2b, the flowchart mainly includes the following steps:
step 1: and acquiring full SQL, specifically acquiring all SQL templates in a specified time interval of the database instance. When the full SQL of the database instance is obtained, the full SQL flow information can be transmitted through the database kernel, the SQL flow information is analyzed through real-time calculation and off-line calculation, the SQL is formatted, the parameters are removed and then templated, and the full SQL template information on the database instance is obtained based on the deduplication of the full SQL template of the database instance.
Step 2: and SQL analysis, namely analyzing all SQL templates, and acquiring predicates, connection conditions, sequencing fields, aggregation fields, projection fields and the like associated with each SQL request.
And step 3: the data structure diagram is constructed in a plurality of modeling manners, only one construction manner is taken as an example in the embodiment, and fig. 2c is a diagram data model which can be optionally constructed in the embodiment of the present application.
As shown in fig. 2c, according to the SQL parsing result, the SQL is associated with the columns to construct a data structure diagram, which may be represented in a diagram data model, wherein vertices are represented by circles, for example, t1.c 1; the edges are indicated by straight lines shaped as arrowheads. There are three types of vertices in the model: SQL vertices, field vertices, table vertices, for example, SQL1 is SQL vertex, T1.C1 denotes C1 column in T1 table, field vertex, T2 is T2 table vertex. It should be noted that each vertex may include the type and attribute of the vertex; for example: the SQL vertex comprises information such as vertex type, SQL template unique identification and the like. SQL is associated with fields, such as SQL1 and T1C1, and different types are established for the relationships according to the type of the SQL associated fields, such as: predicate relations, join field relations, sort field relations, aggregate field relations, projection field relations, etc., where the relations between table vertices and field vertices are subordinates, e.g., field vertex T2.c2 belongs to T2 table vertex. It should be noted that Select (field 1, field 2, … …) from (table name) where is a query data statement.
And 4, step 4: and (4) graph calculation, specifically, after the data structure graph model is constructed, acquiring the related SQL in a graph calculation mode. There are many ways to compute the graph, either by implementing the unicom graph algorithm or by using existing graph computation methods provided by third parties. For example: apache Tinker Pop provides a Connected Component vector Program.
And 5: obtaining related SQL, specifically, building a data structure diagram, and based on the basis of graph computation, may obtain different types of related SQL requests or related schemas, for example:
(1) given a table, all SQL sets associated with the table are obtained.
(2) Given an SQL request, an SQL set related to the SQL is obtained. The related SQL is obtained according to the SQL, and the set of the related SQL can be customized. And calculating a correlation coefficient according to the number of the associated edges, and searching the SQL request with the most correlation according to the correlation coefficient.
(3) Given a column, find the relevant SQL request.
(4) Given a column, the relevant column is looked up.
(5) Given a table, look up the correlation table.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts or the sequence of acts described, as some steps may be performed in other orders or concurrently according to the present invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for building a data structure diagram, it being noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.
The method provided by the second embodiment of the present application can still be executed in a mobile terminal, a computer terminal, or a similar computing device. It should be noted here that the method embodiment provided in the second embodiment can still be run on the computer terminal shown in fig. 1.
Under the operating environment, the application provides a method for establishing a data structure diagram as shown in fig. 3. Fig. 3 is a flowchart illustrating a method for establishing a data structure diagram according to a second embodiment of the present invention.
S302, acquiring at least one SQL template in a database instance;
s304, analyzing at least one SQL template to obtain a first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template;
s306, generating a data structure diagram at least based on the composition elements and the related elements related to the composition elements, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements.
In the above embodiment, the method includes obtaining at least one SQL template in a database instance; and analyzing at least one SQL template to obtain first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template; finally, generating a data structure diagram at least based on the composition elements and the associated elements associated with the composition elements, wherein the data structure diagram comprises: the data graph construction method comprises a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing the incidence relation among the SQL elements, so that the technical effect of quickly constructing the data graph is achieved.
Specifically, the full amount of SQL running water information may be transmitted through a database kernel, the SQL running water information is analyzed through real-time calculation and offline calculation, and the SQL is formatted, and is templated after parameters are removed to obtain an SQL template, where the SQL template may be an SQL template in a specified time interval of a database instance.
Optionally, the vertices used to represent SQL elements in the database instance include the following vertices: a field vertex corresponding to the field in the database instance, and a table vertex corresponding to the table in the database instance; edges used to represent associations between SQL elements include edges used to indicate the following relationships: the relationship between the fields; the relationship between table vertices and field vertices. Each vertex in the vertex set may include a type and an attribute of the vertex, for example, an SQL vertex includes information such as a vertex type and an SQL template unique identifier; in the edge set, the relationship between the table vertex and the field vertex may be an affiliation. And establishing a relationship between SQL and the field, and establishing different types for the relationship according to the type of the SQL related field, such as: predicate relationships, join field relationships, sort field relationships, aggregate field relationships, project field relationships, and the like.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts or the sequence of acts described, as some steps may be performed in other orders or concurrently according to the present invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 3
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for optimizing elements in a database instance, the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions, and although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that illustrated.
The method provided by the third embodiment of the present application can still be executed in a mobile terminal, a computer terminal, or a similar computing device. It should be noted here that the method embodiment provided by the third embodiment can still be run on the computer terminal shown in fig. 1.
Under the above operating environment, the present application provides an element optimization method in the database instance as shown in fig. 4. Fig. 4 is a flowchart illustrating an element optimization method in a database example according to a second embodiment of the present invention.
S402, receiving an optimization request from a target object, wherein the optimization request carries target elements to be optimized;
s404, acquiring an association element having an association relation with the target element;
s406, optimizing the indexes of the target elements and the indexes of the associated elements.
In the above embodiment, an optimization request from a target object is received, where the optimization request carries a target element to be optimized; secondly, acquiring an associated element having an association relation with the target element; and finally, optimizing the indexes of the target elements and the indexes of the associated elements to achieve the aim of optimizing the indexes and the aim of identifying the related SQL, thereby providing powerful support for the SQL optimization process and further solving the technical problem that an effective scheme for identifying the related SQL request does not exist in the related technology.
It should be noted that optimizing the indexes of the target element and the associated element includes, but is not limited to, performing the integration and deduplication processes.
In some embodiments of the present application, the target elements include: a table corresponding to the database instance; the association elements corresponding to the target elements include: a set of SQL statements or tables associated with a table; and/or the target elements include: a column corresponding to a database instance; the association element corresponding to the target element includes: a set of columns or a set of SQL statements associated with the columns.
In some embodiments of the present application, the target elements include: a table corresponding to the database instance; the association elements corresponding to the target elements include: a set of SQL requests or tables associated with a table; and/or the target elements include: a column corresponding to a database instance; the association element corresponding to the target element includes: a set of columns or a set of SQL requests associated with the columns; and/or the target elements include: a target SQL request; the association element corresponding to the target element includes: a set of SQL requests associated with the target SQL request.
In some embodiments of the present application, the SQL request set associated with the target SQL request may be determined by: determining the number of edges in the data structure diagram associated with the target SQL request to determine a correlation coefficient, wherein the edges associated with the target SQL request comprise: an edge passes through from a vertex corresponding to the target SQL request to a target associated vertex, wherein the target associated vertex is a vertex corresponding to the SQL request associated with the target SQL request; and determining the SQL request associated with the target SQL request according to the correlation coefficient, and storing the SQL request associated with the target SQL request to an SQL request set.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts or the sequence of acts described, as some steps may be performed in other orders or concurrently according to the present invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 4
According to an embodiment of the present invention, there is also provided an identification apparatus for implementing the method for identifying an association element in the database example, as shown in fig. 5, the apparatus includes:
a determination module 50 for determining the target element to be detected;
an analysis module 52, configured to analyze the target element based on a data structure diagram of the database instance to obtain an associated element associated with the target element, where the data structure diagram includes: the method comprises a vertex set and an edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements;
and an output module 54, configured to output the association element corresponding to the target element.
In the identification device of the identification method of the associated elements in the database example, the determination module is used for determining the target elements to be detected; the analysis module is used for analyzing the target element based on a data structure diagram of the database instance to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements; the output module is used for outputting the related elements corresponding to the target elements, so that the purpose of identifying related SQL is achieved, powerful support is provided for the SQL optimization process, and the technical problem that an effective scheme for identifying related SQL requests does not exist in related technologies is solved.
In some embodiments of the present application, the analysis module 52 is configured to obtain at least one SQL template in a database instance; analyzing at least one SQL template to obtain a first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template; the data structure diagram is established based at least on the associated elements associated with the constituent elements. It should be noted that the above-mentioned associated elements at least include, but are not limited to: predicates, connection conditions, ordering fields, aggregation fields, projection fields, etc., for associations between SQL requests. For example, the SQL template 1 is analyzed, and the field T2.c1 is included in the element of the SQL1, and if there is T2.c2 in the element of the SQL template 2, then T2 is the related element of the two templates, so the data structure diagram can be established according to the element T2.
In some embodiments of the present application, the analysis module is further configured to obtain at least one SQL template in a database instance, including: acquiring full SQL flow information in a database corresponding to a database instance; analyzing the full amount of SQL flow information to obtain a second analysis content, wherein the second analysis content can be an SQL statement in a time interval; formatting the second analysis content to obtain an initial SQL template; and then, carrying out duplicate removal processing on the initial SQL template to obtain the SQL template. Specifically, the full amount of SQL running water information may be transmitted through the database kernel, the SQL running water information may be analyzed through real-time calculation and offline calculation, and the SQL template may be obtained by formatting the SQL, removing parameters, and then templating, where the SQL template may be the SQL template in the specified time interval of the database instance.
In some optional embodiments of the present application, the vertices used to represent SQL elements in the database instance include the following vertices: a field vertex corresponding to the field in the database instance, and a table vertex corresponding to the table in the database instance; edges used to represent associations between SQL elements include edges used to indicate the following relationships: the relationship between the fields; table vertices and field vertices, where SQL elements may be fields, tables, etc. in a database embodiment. Each vertex in the vertex set may include a type and an attribute of the vertex, for example, an SQL vertex includes information such as a vertex type and an SQL template unique identifier; in the edge set, the relationship between the table vertex and the field vertex may be an affiliation. Establishing a relation between the SQL request and the field, and establishing different types for the relation according to the type of the SQL request associated field, such as: predicate relationships, join field relationships, sort field relationships, aggregate field relationships, project field relationships, and the like.
In some embodiments of the present application, the table vertex and the field vertex may also be merged into a class of vertex to obtain a target vertex; and establishing a data structure diagram based on the incidence relation between the target vertexes.
In some embodiments of the present application, the vertex set further includes: vertexes corresponding to different SQL requests; and/or the following vertexes are also included in the vertex set: index vertex corresponding to the index of the SQL request; the edge set also comprises: the edge for indicating the relation between the table and the index and the edge for indicating the relation between the SQL request and the index construct a data structure diagram of four types of vertexes, for example, by introducing an index vertex, recording the relation between the table and the index and the relation between the SQL request and the index.
In an alternative embodiment of the present application, the data structure diagram may also be determined in the following manner: for example, the table vertex and the field vertex are combined into one type of vertex, and the data structure diagram of the two types of vertices is constructed.
In some embodiments of the present application, after determining the data structure diagram, in order to obtain the related SQL request, the related SQL request may be obtained in a diagram calculation manner, for example, through a unicom algorithm, or in an existing diagram calculation manner provided by a third party, for example: connected Component vector Program supplied by Apache Tinker Pop.
In some embodiments of the present application, the manner of obtaining the related SQL request includes, but is not limited to, identifying the related SQL request by means of predicate string matching or by means of a machine learning model established by AI technology by identifying the correlation as training data from the user side.
In some embodiments of the present application, after determining the data structure diagram, based on the diagram calculation, related SQL requests of different types may be obtained. For example, given a table, obtain all SQL sets related to the table, give an SQL request, obtain an SQL set related to the SQL request, obtain related SQL according to the SQL request, may customize the set of related SQL, such as calculating a correlation coefficient according to the number of associated edges, according to the correlation coefficient, find the most relevant SQL request, given column, find related column, given table, find related table, and so on.
In an optional embodiment of the present application, the determining module is configured to receive an optimization request from a target object, where the optimization request carries a target element; after analyzing the target element based on the data structure diagram of the database instance and obtaining the associated element associated with the target element, the method further comprises: acquiring a first index corresponding to a target element and an index set of associated elements associated with the target element; and integrating and de-duplicating the first index and a second index in the index set.
It should be noted that the target element may be: a table corresponding to the database instance; the association elements corresponding to the target elements include: a set of SQL requests or tables associated with a table; and/or the target elements include: a column corresponding to the database instance; the association element corresponding to the target element includes: a set of columns or a set of SQL requests associated with the columns; and/or the target elements include: a target SQL request; the association element corresponding to the target element includes: a set of SQL requests associated with the target SQL request.
In some embodiments of the present application, the analysis module 52 is further configured to determine the number of edges associated with the target SQL request in the data structure diagram to determine the correlation coefficient, where the edges associated with the target SQL request include: an edge passes through from a vertex corresponding to the target SQL request to a target associated vertex, wherein the target associated vertex is a vertex corresponding to the SQL request associated with the target SQL request; and determining the SQL request associated with the target SQL request according to the correlation coefficient, and storing the SQL request associated with the target SQL request to an SQL request set.
It should be noted here that the determining module 50, the analyzing module 52 and the output module 54 correspond to steps S202 to S206 in embodiment 1, and 3 modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the modules described above as a part of the apparatus may be operated in the computer terminal 10 provided in the first embodiment.
Example 5
Embodiments of the invention may provide a computing device that may be any one of a computing device group. Optionally, in this embodiment, the computing device may also be replaced with a terminal device such as a mobile terminal.
Optionally, in this embodiment, the computing device may be located in at least one network device of a plurality of network devices of a computer network.
In this embodiment, the computing device may execute program codes of the following steps in the vulnerability detection method of the application program: determining a target element to be detected in a database instance, wherein the target element is an SQL element in an SQL database; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the steps that a vertex set and an edge set are included, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements; and outputting the associated element corresponding to the target element.
In the above embodiment of the present application, first, a target element to be detected in a database instance is determined, and then, the target element is analyzed based on a data structure diagram of the database instance to obtain an associated element associated with the target element, where the data structure diagram includes: the method comprises the steps that a vertex set and an edge set are included, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements; and finally, outputting the associated elements corresponding to the target elements, so that the purpose of identifying the related SQL is achieved, powerful support is provided for the SQL optimization process, and the technical problem that an effective scheme for identifying the related SQL request does not exist in the related technology is solved.
Alternatively, fig. 6 is a block diagram of a computing device according to an embodiment of the invention. As shown in fig. 6, the computing device 60 may include: one or more (only one shown) processors 602, memory 604.
The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for identifying associated elements in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the method for identifying associated elements in the database example. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory remotely located from the processor, and these remote memories may be connected to terminal a through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor can call the information and the application program stored in the memory through the transmission module to execute the following steps: determining a target element to be detected in a database instance; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the steps of a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in a database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements; and outputting the associated element corresponding to the target element.
Optionally, the processor may further execute the program code of the following steps: determining a target element to be detected; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: the vertex used for expressing SQL element in database example, the set of edges includes at least: edges for representing associations between SQL elements; and outputting the associated element corresponding to the target element.
Optionally, the processor may further execute the program code of the following steps: acquiring at least one SQL template in a database instance; analyzing at least one SQL template to obtain a first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of at least one SQL template; the data structure diagram is built based at least on the associated elements associated with the constituent elements.
Optionally, the processor may further execute the program code of the following steps: acquiring full SQL flow information in a database corresponding to a database example; analyzing the full SQL flow information to obtain second analysis content; formatting the second analysis content to obtain an initial SQL template; and carrying out duplication removal processing on the initial SQL template to obtain at least one SQL template. Specifically, the full amount of SQL running water information may be transmitted through the database kernel, the SQL running water information is analyzed through real-time calculation and offline calculation, and the SQL is formatted, and templated after parameters are removed to obtain the SQL template, where the SQL template may be an SQL template in a time interval specified by a database instance.
Optionally, the processor may further execute the program code of the following steps: acquiring full SQL flow information in a database corresponding to a database example; analyzing the full SQL flow information to obtain second analysis content; formatting the second analysis content to obtain an initial SQL template; and carrying out duplication removal processing on the initial SQL template to obtain at least one SQL template. .
Optionally, the processor may further execute the program code of the following steps: merging the table vertex and the field vertex into a class of vertex to obtain a target vertex; and establishing a data structure diagram based on the incidence relation between the target vertexes.
Optionally, the processor may further execute the program code of the following steps: determining a target element to be detected, comprising: receiving an optimization request from a target object, wherein the optimization request carries target elements; after analyzing the target element based on the data structure diagram of the database instance and obtaining the associated element associated with the target element, the method further comprises: acquiring a first index corresponding to a target element and an index set of associated elements associated with the target element; and integrating and de-duplicating the first index and a second index in the index set.
Optionally, the processor may further execute the program code of the following steps: determining the number of edges in the data structure diagram associated with the target SQL request to determine a correlation coefficient, wherein the edges associated with the target SQL request comprise: an edge passes through from a vertex corresponding to the target SQL request to a target associated vertex, wherein the target associated vertex is a vertex corresponding to the SQL request associated with the target SQL request; and determining the SQL request associated with the target SQL request according to the correlation coefficient, and storing the SQL request associated with the target SQL request to an SQL request set.
By adopting the embodiment of the invention and adopting a mode of identifying related SQL, firstly, the target element to be detected is determined, and secondly, the target element is analyzed based on the data structure diagram of the database example to obtain the related element related to the target element, wherein the data structure diagram comprises: a vertex set and an edge set, wherein the vertex set at least comprises: vertices for representing SQL elements in a database instance, the set of edges comprising at least: the method is used for representing the edges of the incidence relation between the SQL elements, and finally, the incidence element corresponding to the target element is output, so that the purpose of identifying the related SQL is achieved, and the technical problem that an effective scheme for identifying the related SQL request does not exist in the related technology is solved.
It will be understood by those skilled in the art that the structure shown in fig. 6 is only an illustration, and the computing device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 6 is a diagram illustrating a structure of the electronic device. For example, computing device 60 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 6, or have a different configuration than shown in FIG. 6.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the nonvolatile storage medium may be configured to store a program code executed by the method for identifying an associated element in the database instance provided in the first embodiment.
Optionally, in this embodiment, the storage medium may be located in any one of computing devices in a computing device group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a target element to be detected; analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in a database instance, the set of edges comprising at least: edges for representing associations between SQL elements; and outputting the associated element corresponding to the target element.
Example 6
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for identifying associated elements in a database instance, the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions, and although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that presented herein.
The method provided by the sixth embodiment of the present application can still be executed in a mobile terminal, a computer terminal, or a similar computing device. It should be noted here that the method embodiment provided in the sixth embodiment can still be run on the computer terminal shown in fig. 1.
Under the operating environment, the application provides a method for identifying the associated elements in the database example as shown in fig. 7. Fig. 7 is a flowchart illustrating a method for identifying an association element in a database example according to a sixth embodiment of the present invention.
S702, receiving an SQL request, and acquiring a target element in a database instance from the SQL request;
s704, analyzing the target element based on the data structure diagram of the database instance to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertices in the vertex set comprise: vertices for representing different SQL requests and vertices for representing SQL elements in a database instance, the set of edges including: edges for representing associations between different SQL requests and SQL elements in the database instance, edges for representing associations between SQL elements;
s706, outputting the related element corresponding to the target element.
In the above embodiment, first, an SQL request may be received, and a target element in a database instance may be obtained from the SQL request; secondly, analyzing the target element based on a data structure diagram of the database example to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertices in the vertex set comprise: vertices for representing different SQL requests and vertices for representing SQL elements in a database instance, the set of edges including: the SQL database comprises edges used for representing the association relation between different SQL requests and SQL elements in a database instance and edges used for representing the association relation between the SQL elements; and finally, outputting the related elements corresponding to the target elements, so that the purpose of identifying related SQL is achieved, powerful support is provided for the SQL optimization process, and the technical problem that an effective scheme for identifying related SQL requests does not exist in the related technology is solved.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts or the sequence of acts described, as some steps may be performed in other orders or concurrently according to the present invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be construed as the protection scope of the present invention.

Claims (14)

1. A method for identifying associated elements in a database instance comprises the following steps:
determining a target element to be detected, wherein the target element is an SQL element in an SQL database;
analyzing the target element based on a data structure diagram of a database instance to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the following steps of vertex set and edge set, wherein the vertex set at least comprises the following steps: vertices for representing SQL elements in the database instance, the set of edges including at least: edges for representing associations between the SQL elements;
and outputting the association element corresponding to the target element.
2. The method of claim 1, wherein the data structure map is determined by:
acquiring at least one SQL template in the database instance;
analyzing the at least one SQL template to obtain first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of the at least one SQL template;
the data structure diagram is established based at least on the associated elements associated with the constituent elements.
3. The method of claim 2, wherein obtaining at least one SQL template in the database instance comprises:
acquiring full SQL flow information in a database corresponding to the database instance;
analyzing the full SQL flow information to obtain second analysis content;
formatting the second analysis content to obtain an initial SQL template;
and carrying out duplication elimination processing on the initial SQL template to obtain the at least one SQL template.
4. The method of claim 1, wherein the vertices used to represent SQL elements in the database instance comprise the following vertices: a field vertex corresponding to the field in the database instance, and a table vertex corresponding to the table in the database instance; edges for representing associations between the SQL elements include edges for indicating the following relationships: the relationship between the fields; the relationship between table vertices and field vertices.
5. The method of claim 4, wherein the method further comprises:
merging the table vertex and the field vertex into a class of vertex to obtain a target vertex; and establishing the data structure diagram based on the incidence relation between the target vertexes.
6. The method of claim 1, wherein,
the vertex set further comprises: vertexes corresponding to different SQL requests; and/or
The vertex set further includes the following vertices: index vertex corresponding to the index of the SQL request; the edge set further comprises: an edge indicating the relationship of the table to the index, and an edge indicating the relationship of the SQL request to the index.
7. The method of claim 1, wherein,
determining a target element to be detected, comprising: receiving an optimization request from a target object, wherein the optimization request carries the target element;
after analyzing the target element based on the data structure diagram of the database instance and obtaining an associated element associated with the target element, the method further comprises: acquiring a first index corresponding to the target element and an index set of associated elements associated with the target element; and integrating and de-duplicating the first index and a second index in the index set.
8. The method of claim 1, wherein,
the target elements include: a table corresponding to the database instance; the association element corresponding to the target element comprises: a set of SQL requests or tables associated with the table; and/or
The target elements include: a column corresponding to the database instance; the association element corresponding to the target element comprises: a set of columns or a set of SQL requests associated with the columns; and/or
The target elements include: a target SQL request; the association element corresponding to the target element comprises: a set of SQL requests associated with the target SQL request.
9. The method of claim 8, wherein the SQL request set associated with the target SQL request is determined by:
determining the number of edges in the data structure diagram associated with the target SQL request to determine a correlation coefficient, wherein the edges associated with the target SQL request comprise: an edge passes through from a vertex corresponding to the target SQL request to a target associated vertex, wherein the target associated vertex is a vertex corresponding to the SQL request associated with the target SQL request;
and determining the SQL request associated with the target SQL request according to the correlation coefficient, and storing the SQL request associated with the target SQL request to the SQL request set.
10. A method for establishing a data structure diagram comprises the following steps:
acquiring at least one SQL template in a database instance;
analyzing the at least one SQL template to obtain first analysis content, wherein the first analysis content at least comprises: an association element associated with a constituent element of the at least one SQL template;
generating a data structure diagram at least based on the component elements and the associated elements associated with the component elements, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertex in the vertex set is used for representing SQL elements in the database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements.
11. A method for identifying associated elements in a database instance comprises the following steps:
receiving an SQL request, and acquiring a target element in a database instance from the SQL request;
analyzing the target element based on the data structure diagram of the database instance to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises a vertex set and an edge set, wherein the vertices in the vertex set comprise: vertices for representing different SQL requests and vertices for representing SQL elements in the database instance, the set of edges including: an edge for representing an incidence relation between different SQL requests and SQL elements in the database instance, and an edge for representing an incidence relation between the SQL elements;
and outputting the association element corresponding to the target element.
12. A method for optimizing elements in a database instance, comprising:
receiving an optimization request from a target object, wherein the optimization request carries a target element to be optimized, and the target element is an SQL element in an SQL database;
acquiring an association element having an association relation with the target element;
and optimizing the index of the target element and the index of the associated element.
13. A non-volatile storage medium comprising a stored program, wherein, when the program runs, a device in which the non-volatile storage medium is located is controlled to execute the method for identifying the associated element in the database instance according to any one of claims 1 to 9.
14. A computing device, comprising:
a processor; and
a memory coupled to the processor for providing instructions to the processor for processing the following processing steps:
determining a target element to be detected in a database instance, wherein the target element is an SQL element in an SQL database;
analyzing the target element based on the data structure diagram of the database instance to obtain an associated element associated with the target element, wherein the data structure diagram comprises: the method comprises the steps that a vertex set and an edge set are used, wherein the vertex in the vertex set is used for representing SQL elements in the database instance, and the edge in the edge set is used for representing an incidence relation between the SQL elements;
and outputting the association element corresponding to the target element.
CN202010462483.2A 2020-05-27 2020-05-27 Identification method of database associated elements and establishment method of data structure diagram Pending CN113312431A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010462483.2A CN113312431A (en) 2020-05-27 2020-05-27 Identification method of database associated elements and establishment method of data structure diagram

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010462483.2A CN113312431A (en) 2020-05-27 2020-05-27 Identification method of database associated elements and establishment method of data structure diagram

Publications (1)

Publication Number Publication Date
CN113312431A true CN113312431A (en) 2021-08-27

Family

ID=77370163

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010462483.2A Pending CN113312431A (en) 2020-05-27 2020-05-27 Identification method of database associated elements and establishment method of data structure diagram

Country Status (1)

Country Link
CN (1) CN113312431A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116756184A (en) * 2023-08-17 2023-09-15 腾讯科技(深圳)有限公司 Database instance processing method, device, equipment, storage medium and program product

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116756184A (en) * 2023-08-17 2023-09-15 腾讯科技(深圳)有限公司 Database instance processing method, device, equipment, storage medium and program product
CN116756184B (en) * 2023-08-17 2024-01-12 腾讯科技(深圳)有限公司 Database instance processing method, device, equipment, storage medium and program product

Similar Documents

Publication Publication Date Title
US10120930B2 (en) Identifying entity mappings across data assets
CN109241068B (en) Method and device for comparing foreground and background data and terminal equipment
CN111858649B (en) Heterogeneous data fusion method based on ontology mapping
CN112445875B (en) Data association and verification method and device, electronic equipment and storage medium
GB2507094A (en) Selecting and loading a subset of data from a database backup
CN109376873B (en) Operation and maintenance method, operation and maintenance device, electronic equipment and computer readable storage medium
US20160154840A1 (en) Avoid double counting of mapped database data
US20130054539A1 (en) Database Record Repair
CN109491989A (en) Data processing method and device, electronic equipment, storage medium
CN114598539B (en) Root cause positioning method and device, storage medium and electronic equipment
CN113312191A (en) Data analysis method, device, equipment and storage medium
CN111652299A (en) Method and equipment for automatically matching service data
CN111414410A (en) Data processing method, device, equipment and storage medium
CN113312431A (en) Identification method of database associated elements and establishment method of data structure diagram
CN111984745B (en) Database field dynamic expansion method, device, equipment and storage medium
EP2742443B1 (en) Database record repair
CN115599871A (en) Lake and bin integrated data processing system and method
CN115374105A (en) Data processing method and device
CN115481105A (en) Data management method, device, electronic equipment and storage medium
CN111427902B (en) Metadata management method, device, equipment and medium based on lightweight database
CN112634004A (en) Blood margin map analysis method and system for credit investigation data
CN116541418B (en) Data query method, electronic device and computer readable storage medium
CN116680445B (en) Knowledge-graph-based multi-source heterogeneous data fusion method and system for electric power optical communication system
CN116755683B (en) Data processing method and related device
EP4328759A1 (en) Storing and querying knowledge graphs in column stores

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40059822

Country of ref document: HK