CN116069804A - Search engine data synchronization method and device, electronic equipment and storage medium - Google Patents
Search engine data synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116069804A CN116069804A CN202211528290.8A CN202211528290A CN116069804A CN 116069804 A CN116069804 A CN 116069804A CN 202211528290 A CN202211528290 A CN 202211528290A CN 116069804 A CN116069804 A CN 116069804A
- Authority
- CN
- China
- Prior art keywords
- data table
- data
- name
- names
- association
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a search engine data synchronization method, apparatus, electronic device, and storage medium, the method comprising: acquiring a plurality of data table names corresponding to a database and association relations among the plurality of data table names, constructing a wide table structure of a search engine according to the plurality of data table names and the association relations among the plurality of data table names, wherein the wide table structure comprises a plurality of association chains composed of the plurality of data table names, acquiring an updating instruction aiming at the database, determining a target data table name corresponding to the updating instruction and updating contents, determining a target association chain comprising the target data table name in the wide table structure, and updating a target data table corresponding to the target data table name included in the target association chain according to the updating contents. Therefore, the synchronous updating of the data of the search engine is realized while the database is updated conveniently and rapidly.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a search engine data synchronization method, apparatus, electronic device, and storage medium.
Background
Search engines have faster data query speeds relative to databases, such as ES (elasticsearch) search engines, and in order to ensure accuracy of data query results, it is desirable to achieve data synchronization between the search engine and the database.
In the related art, data synchronization between a search engine and a database cannot be conveniently and quickly realized.
Disclosure of Invention
The purpose of the present disclosure is to provide a method, an apparatus, an electronic device and a storage medium for synchronizing search engine data, so as to conveniently and rapidly realize data synchronization between a search engine and a database.
According to a first aspect of an embodiment of the present disclosure, there is provided a search engine data synchronization method, including:
acquiring a plurality of data table names corresponding to a database and an association relation among the plurality of data table names;
constructing a wide table structure of the search engine according to the data table names and the association relations among the data table names, wherein the wide table structure comprises a plurality of association chains formed by the data table names;
acquiring an update instruction aiming at the database, and determining a target data table name and update content corresponding to the update instruction;
Determining a target association chain comprising the target data table name in the wide table structure;
and updating the target data table corresponding to the target data table name included in the target association chain according to the updating content.
Optionally, the step of constructing the broad table structure of the search engine according to the plurality of data table names and the association relationship between the plurality of data table names specifically includes:
aiming at any one of the data table names, constructing an association chain taking the data table name as a chain head according to the association relation among the data table names;
and obtaining the wide table structure of the search engine according to the constructed association chains.
Optionally, the step of constructing, for any one of the plurality of data table names, an association chain using the data table name as a chain head according to an association relationship between the plurality of data table names specifically includes:
constructing an association secondary tree according to the data table names and the association relation among the data table names, wherein a first level of the association secondary tree comprises the data table names, a second level of the association secondary tree comprises a subset of each data table name, and each subset of the data table names is all data table names associated with the data table name;
And constructing an association chain taking any data table name in the first stage as a chain head according to the association secondary tree.
Optionally, the step of constructing an association chain with any data table name in the first level as a chain head according to the association secondary tree specifically includes:
and determining the data table name as an associated chain corresponding to the data table name when the subset of the data table names is empty aiming at any data table name in the first stage.
Optionally, the constructing an association chain with any data table name in the first level as a chain head according to the association secondary tree includes:
and aiming at any data table name in the first stage, under the condition that the subset of the data table names comprises at least one data table name, taking the data table name as a chain head, and determining a chain obtained by sequentially connecting all the data table names included in the subset of the data table names at the tail of the data table name as an associated chain corresponding to the data table name.
Optionally, the step of constructing an association chain with any data table name in the first level as a chain head according to the association secondary tree specifically includes:
Determining that any data table name in the first level is a first data table name;
when the second data table names included in the subset of the first data table names exist in the first stage and the subset of the second data table names does not include the first data table names, taking the first data table names as chain heads, and sequentially connecting all the data table names included in the subset of the first data table names to the tail parts of the first data table names to obtain a first chain;
sequentially connecting all data table names included in the subset of the second data table names to obtain a second chain;
and inserting the second chain into the tail part of the second data table name in the first chain to obtain an associated chain corresponding to the first data table name.
Optionally, the step of obtaining a plurality of data table names corresponding to the database and association relations between the plurality of data table names specifically includes:
acquiring initial names of all data tables in a database and association relations among all the data tables;
setting a mapping name corresponding to the initial name of each data table, taking the mapping name as the data table name of each data table, and determining the association relationship between the data table names according to the association relationship between the data tables, wherein the length of the mapping name of each data table is smaller than that of the initial name of the data table.
Optionally, the step of acquiring an update instruction for the database and determining a target data table name and update content corresponding to the update instruction specifically includes:
establishing a monitor aiming at the structured query statement according to the database connection pool;
determining, by the monitor, whether a structured query statement for the database is a data control statement and a data definition statement;
in response to any one of the structured query statement being the data control statement or a definition statement for the data, determining the structured query statement as an update instruction for the database;
and analyzing the updating instruction to obtain the name and the updating content of the target data table.
Optionally, the method further comprises:
intercepting a query instruction for the database;
acquiring package format data of the database aiming at a query result;
and sending the query instruction and the package format data to the search engine, wherein the query instruction is used for the search engine to acquire a query result according to the query instruction, and the package format data is used for the search engine to package the query result according to the package format data.
According to a second aspect of embodiments of the present disclosure, there is provided a search engine data synchronization apparatus, including:
the first acquisition module is configured to acquire a plurality of data table names corresponding to the database and association relations among the plurality of data table names;
a building module configured to build a broad table structure of the search engine according to the plurality of data table names and the association relation among the plurality of data table names, wherein the broad table structure comprises a plurality of association chains composed of the plurality of data table names;
the acquisition module is configured to acquire an update instruction aiming at the database and determine a target data table name and update content corresponding to the update instruction;
a determining module configured to determine a target association chain in the wide table structure that includes the target data table name;
and the updating module is configured to update the target data table corresponding to the target data table name included in the target association chain according to the updating content.
Optionally, the building module includes:
the construction submodule is configured to construct an association chain taking the data table name as a chain head according to the association relation among the data table names aiming at any one of the data table names;
And the obtaining sub-module is configured to obtain the wide table structure of the search engine according to the constructed plurality of associated chains.
Optionally, the constructing sub-module includes:
a first construction unit configured to construct an association secondary tree according to the plurality of data table names and association relations between the plurality of data table names, a first level of the association secondary tree including the plurality of data table names, a second level of the association secondary tree including a subset of each of the data table names, each subset of the data table names being all data table names associated with the data table name;
and the second construction unit is configured to construct an association chain with any data table name in the first stage as a chain head according to the association secondary tree.
Optionally, the second building unit includes:
a first determining subunit configured to determine, for any one of the data table names in the first stage, the data table name as an associated chain corresponding to the data table name if a subset of the data table names are empty.
Optionally, the second building unit includes:
and the second determining subunit is configured to determine, for any data table name in the first stage, a chain obtained by sequentially connecting all data table names included in the subset of the data table names at the tail of the data table name with the data table name as a chain head when the subset of the data table names includes at least one data table name, as an associated chain corresponding to the data table name.
Optionally, the second building unit includes:
a third determination subunit configured to determine any one of the data table names in the first stage as a first data table name;
a first obtaining subunit, configured to, when a second data table name included in the subset of the first data table names exists in the first stage and the subset of the second data table names does not include the first data table name, sequentially connect all the data table names included in the subset of the first data table names to the tail of the first data table name with the first data table name as a chain head, so as to obtain a first chain;
a second obtaining subunit, configured to sequentially connect all the data table names included in the subset of the second data table names, so as to obtain a second chain;
and the third obtaining subunit is configured to insert the second chain into the tail part of the second data table name in the first chain to obtain an associated chain corresponding to the first data table name.
Optionally, the first acquisition module includes:
the first acquisition sub-module is configured to acquire initial names of all data tables in the database and association relations among all the data tables;
The first obtaining sub-module is configured to set a mapping name corresponding to the initial name of each data table, and determine an association relationship between the data table names according to the association relationship between the data tables by taking the mapping name as the data table name of each data table, so as to obtain the structural data, wherein the length of the mapping name of each data table is smaller than that of the initial name of the data table.
Optionally, the obtaining module includes:
the building sub-module is configured to build a monitor aiming at the structured query statement according to the database connection pool;
a first determination submodule configured to determine, by the monitor, whether a structured query statement for the database is a data control statement and a data definition statement;
a second determination submodule configured to determine the structured query statement as an update instruction for the database in response to either the structured query statement being the data control statement or the data definition statement;
and the second determining submodule is configured to analyze the updating instruction to obtain the name and the updating content of the target data table.
Optionally, the apparatus further comprises:
an interception module configured to intercept a query instruction for the database;
the second acquisition module is configured to acquire package format data of the database aiming at the query result;
the sending module is configured to send the query instruction and the package format data to the search engine, the query instruction is used for the search engine to obtain a query result according to the query instruction, and the package format data is used for the search engine to package the query result according to the package format data.
According to a third aspect of embodiments of the present disclosure, there is provided an electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the search engine data synchronization method of the first aspect.
According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the search engine data synchronization method of the first aspect described above.
According to the technical scheme, the wide table structure of the corresponding search engine is constructed according to the plurality of data table names of the database and the association relation among the plurality of data table names, the updating instruction of the database is obtained, the updated target data table names and the updating content are determined, all target association chains comprising the target data table names are searched from the wide table structure of the search engine according to the target data table names, and the target data tables corresponding to the target data table names in all the target chains are updated according to the updating content, so that the synchronous updating of the data of the search engine is realized while the database is updated conveniently and rapidly.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a flow chart illustrating a method of search engine data synchronization according to an exemplary embodiment.
Fig. 2 is a flow chart illustrating a method of acquiring structural data according to an exemplary embodiment.
FIG. 3 is a flowchart illustrating a method of constructing a wide table structure, according to an example embodiment.
Fig. 4 is a flowchart illustrating a method of constructing an associated chain, according to an exemplary embodiment.
FIG. 5 is a schematic diagram of an associated secondary tree, shown according to an exemplary embodiment.
FIG. 6 is a flowchart illustrating a method of search engine data synchronization, according to an example embodiment.
FIG. 7 is a flowchart illustrating another method of search engine data synchronization, according to an example embodiment.
FIG. 8 is a block diagram illustrating a search engine data synchronization apparatus according to an example embodiment.
Fig. 9 is a block diagram of an electronic device, according to an example embodiment.
Detailed Description
Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.
In order to realize data synchronization between the search engine and the database, when a user performs update operation on the database, for example, when adding, deleting and changing a data table in the database, the data in the search engine needs to be updated in the same way so as to ensure the accuracy of a query result obtained by the search engine. However, in the related art, data synchronization between the search engine and the database cannot be conveniently and quickly achieved.
In view of the above technical problems, an embodiment of the present disclosure provides a method, an apparatus, an electronic device, and a storage medium for synchronizing search engine data, where a broad table structure of a corresponding search engine is constructed according to a plurality of data table names of a database and association relations between the plurality of data table names, an update instruction for the database is obtained, updated target data table names and update contents are determined, then, according to the target data table names, all target association chains including the target data table names are found from the broad table structure of the search engine, and according to the update contents, the target data tables corresponding to the target data table names in all target chains are updated, so that synchronous update of data for the search engine is realized while updating the database is conveniently and rapidly realized.
FIG. 1 is a flowchart illustrating a method of search engine data synchronization, as shown in FIG. 1, according to an exemplary embodiment, the method may include the steps of:
in step S101, a plurality of table names corresponding to the database and an association relationship between the plurality of table names are acquired.
In this embodiment, the database may be a relational database, and the database may include a plurality of data tables, for example: the user table, the mechanism table, the role table and the like can determine the structural data of the database according to the generated document of the database, and the database does not need to be traversed to acquire the structural data corresponding to the database, so that the method is convenient and quick. The structure data may include a plurality of data table names and an association relationship between the plurality of data table names.
In step S102, a broad table structure of the search engine is constructed according to the plurality of data table names and the association relationship between the plurality of data table names, the broad table structure including a plurality of association chains composed of the plurality of data table names.
In this embodiment, a broad table structure of the corresponding search engine may be constructed according to a plurality of table names of the database and association relationships between the plurality of table names, and the broad table structure may include a plurality of association chains, each of which may be composed of at least one of the plurality of table names. Through a plurality of associated chains, the inquiry of the data table can be rapidly realized. The broad table is composed of at least one data table, one associated chain corresponds to one broad table, the broad table structure is a structure composed of a plurality of associated chains corresponding to the plurality of broad tables, and the search engine can store the broad table corresponding to each associated chain, namely, the search engine stores the plurality of broad tables, so that the corresponding data table can be quickly determined and the content contained in the data table can be obtained according to the associated chains, and therefore the data table can be added, deleted, changed and searched.
In step S103, an update instruction for the database is acquired, and a target data table name and update content corresponding to the update instruction are determined.
In this embodiment, the user may generate a corresponding update instruction for updating the database, and may obtain the update instruction for the database, and parse the update instruction to obtain specific content included in the update instruction, for example, a target data table name corresponding to the data table for which the update instruction is directed and the update content included in the update instruction. The update instruction may include an identification of the target data table, such that the target data table name may be determined based on the identification of the target data table. The method does not affect the update of the database by the update instruction, and the update instruction can normally reach the database so as to update the content of the database. And updating the content in the search engine according to the acquired target data table name and the updated content. The target data table name and the update content are acquired by acquiring the update instruction, and the target data table name and the update content do not need to be acquired from the database, so that decoupling of an interface of the database can be realized, and the universality is improved.
In step S104, a target association chain including the target data table name in the wide table structure is determined.
The wide table structure comprises a plurality of associated chains, each associated chain at least comprises one data table name, and the same data table name can exist in the plurality of associated chains. According to the target data table name, the wide table structure is inquired, and all target associated chains containing the target data table name in the wide table structure can be determined, namely, a target data table name can be used for obtaining a multi-item target associated chain related to the target data table name.
In step S105, the target data table corresponding to the target data table name included in the target associated chain is updated according to the update content.
In this embodiment, a target data table corresponding to the target data table name included in each target association chain may be acquired, and the target data table may be updated according to the update content, for example, the target data table may be added, deleted, and changed. Because the updating instruction is only acquired but not intercepted, the database can be updated according to the updating instruction, so that the synchronous updating of the data of the search engine can be realized while the database is updated.
In this embodiment, a corresponding broad table structure of the search engine is constructed according to the structure of the database, an update instruction for the database is obtained, an updated target data table name and update content are determined, all target associated chains including the target data table name are searched from the broad table structure of the search engine according to the target data table name, and the target data table corresponding to the target data table name in all target chains is updated according to the update content, so that synchronous update of data of the search engine is realized while updating the database is conveniently and rapidly realized.
FIG. 2 is a flowchart illustrating a method of obtaining structural data according to an exemplary embodiment, as shown in FIG. 2, in a possible implementation, obtaining a plurality of data table names corresponding to a database and an association relationship between the plurality of data table names may include the following steps:
in step S201, initial names of the data tables in the database and association relations between the data tables are acquired.
In step S202, a mapping name corresponding to the initial name is set for the initial name of each data table, and the mapping name is used as the data table name of each data table, and the association relationship between the data table names is determined according to the association relationship between the data tables, wherein the length of the mapping name of each data table is smaller than the length of the initial name of the data table.
In this embodiment, a mapping relationship between an original name and a mapping name of each data table may be preset, and a length of the mapping name of each data table is smaller than a length of the original name of the data table. For example, for a user table, its initial name is user, and its corresponding mapping name may be u. And obtaining the mapping name corresponding to the initial name of each data table according to the preset relation, and taking the mapping name as the data table name of the corresponding data table. By acquiring the association relationship between the data tables, the association relationship between the data table names can be determined, so that a plurality of data table names and the association relationship between the data table names are obtained.
In this embodiment, since the initial name of each data table may be long, the expression of the data table name can be facilitated and the query of the data table name can be facilitated by replacing the initial name of each data table with a mapping name having a shorter length.
FIG. 3 is a flow chart illustrating a method of building a broad table structure according to an exemplary embodiment, as shown in FIG. 3, in one possible implementation, building a broad table structure of a search engine from a plurality of data table names and an association between the plurality of data table names may include the steps of:
In step S301, for any one of the plurality of table names, an association chain having the table name as a chain head is constructed from an association relationship between the plurality of table names.
In this embodiment, a plurality of table names may be traversed, and according to any table name included in the structure data, the table name is used as a chain head, and table names associated with the table name are searched for, thereby forming an association chain. Wherein the data table names associated with the data table names may include directly associated data table names and indirectly associated data table names.
In step S302, a broad table structure of the search engine is obtained according to the constructed plurality of association chains.
By the method, each data table name can obtain one associated chain, and all the constructed associated chains can be determined to be a wide table structure of the search engine. The search engine may store a set of a plurality of data tables corresponding to each associated chain, where the set of a plurality of data tables corresponding to each associated chain is a broad table. The search engine can conveniently and rapidly inquire the data table by carrying out redundant storage on the data table in a mode of associating chains.
FIG. 4 is a flowchart illustrating a method of constructing an association chain according to an exemplary embodiment, as shown in FIG. 4, in one possible implementation, for any one of a plurality of data table names, constructing an association chain with the data table name as a chain head according to an association relationship between the plurality of data table names, may include the steps of:
in step S401, an association secondary tree is constructed according to the plurality of data table names and the association relationship between the plurality of data table names, the first level of the association secondary tree including the plurality of data table names, the second level of the association secondary tree including a subset of each data table name, the subset of each data table name being all data table names associated with the data table name.
In this embodiment, the association secondary tree may be constructed from a plurality of table names and association relationships between the plurality of table names according to the manner of the brain map. The first level of the association secondary tree comprises a plurality of data table names obtained by traversing the structural data, the second level of the association secondary tree comprises a subset of each data table name, the subset of each data table name is all data table names associated with the data table name, and each data table name can be respectively connected with a plurality of data table names associated with the data table name.
In step S402, an association chain with any one of the data table names in the first level as the head of the chain is constructed from the association secondary tree.
In this embodiment, the association secondary tree may be parsed, and according to the connection structure of the association secondary tree, any data table name in the first level is used as a link head, and other data table names associated with the data table name are searched, so as to jointly form an association chain corresponding to the data table name, and the data tables corresponding to all the data table names included in the association chain form a wide table corresponding to the data table name. Each data table name in the first level may result in a corresponding one of the broad tables. For example, for a user table, the intermediate results of its parsing may be: select from user as u left join department as d ON u.dept_id=d.id LEFT JOIN role as r ON u.role_id=r.id, the associated chain corresponding to the obtained user table may be: u- > d- > r.
In one possible implementation, according to the association secondary tree, the method for constructing the association chain with any data table name in the first level as the chain head can be as follows: for any one of the data table names in the first level, in the case that the subset of the data table names is empty, determining the data table name as an associated chain corresponding to the data table name.
In this embodiment, if the subset of the table names is empty, the table corresponding to the table name has no other table associated therewith, and the association chain corresponding to the table name is composed of the table names. Fig. 5 is a schematic diagram of an association secondary tree according to an exemplary embodiment, where, as shown in fig. 5, the association chain corresponding to E is E, the association chain corresponding to G is G, and the association chain corresponding to H is H.
In one possible implementation, another method of constructing an association chain with any data table name in the first level as the head of the chain from the association secondary tree may be: and aiming at any data table name in the first stage, under the condition that the subset of the data table names comprises at least one data table name, taking the data table name as a chain head, and determining a chain obtained by sequentially connecting all the data table names included in the subset of the data table names at the tail of the data table name as an associated chain corresponding to the data table name.
In this embodiment, if the subset of the table names includes at least one table name, the table name is used as a chain head, and all the table names included in the subset of the table names are sequentially connected to the tail of the table name, so that the associated chain corresponding to the table name can be obtained. For example, for user table name u, a subset of which includes mechanism table name d and role table name r, its corresponding association chain may be: u- > d- > r. As shown in fig. 5, the association chain corresponding to B is B- > D- > E, and the association chain corresponding to C is C- > F- > G- > H.
In one possible implementation, according to the association secondary tree, another method for constructing an association chain with any data table name in the first level as the chain head may be: determining any data table name in the first level as a first data table name; under the condition that second data table names included in the subset of the first data table names exist in a first stage and the subset of the second data table names does not include the first data table names, using the first data table names as chain heads, sequentially connecting all the data table names included in the subset of the first data table names at the tail parts of the first data table names to obtain a first chain; sequentially connecting all the data table names included in the subset of the second data table names to obtain a second chain; and inserting a second chain into the tail part of the second data table name in the first chain to obtain an associated chain corresponding to the first data table name.
In this embodiment, when the second data table name included in the subset of the first data table names exists in the first stage and the subset of the second data table names does not include the first data table name, the first data table name also exists in indirect association with the first data table name, and the searching for the related data table name may be continued until a subset includes the first data table name. And connecting all the data tables by using the first data table name as a chain head to form an associated chain corresponding to the first data table name.
As shown in fig. 5, for an associated chain of a, a subset includes B and C, a subset of B includes D and E, the subset of C comprises F, G and H, and the associated chain corresponding to A is A- > B- > D- > E- > C- > F- > G- > H.
Optionally, the data table may further include an association table, for example, the association table may store a many-to-many relationship between a user and a character, that is, an association relationship between one user and a plurality of characters and an association relationship between one character and a plurality of users. At this time, when the association table exists at the second level in the association secondary tree, the association table may be ignored when the association chain of the corresponding data table name is constructed, or the association table in the second level in the association secondary tree may be removed first when the association chain is constructed.
FIG. 6 is a flowchart of a method for synchronizing search engine data according to an exemplary embodiment, as shown in FIG. 6, in one possible implementation, the steps of obtaining an update instruction for a database, and determining a target data table name and update content corresponding to the update instruction may include:
in step S601, a monitor for structured query statements is built from a database connection pool.
In this embodiment, a monitor for the structured query statement may be newly created using a monitoring technique of the database connection pool, and the update instruction may be at least one of a data control statement and a data definition statement, and the monitor may be used to monitor the data control statement and the data definition statement in the structured query statement so as to obtain the update instruction.
In step S602, it is determined by the monitor whether the structured query statement for the database is a data control statement or a data definition statement.
The monitor may be used to monitor data control statements and data definition statements in the structured query statement to determine whether the structured query statement is an update instruction.
In step S603, in response to either of the structured query statement being a data control statement or a data definition statement, the structured query statement is determined to be an update instruction for the database.
In this embodiment, when the monitor monitors that any one of the structured query statement is a data control statement or a data definition statement, the monitor determines the structured query statement as an update instruction for the database, and may obtain the update instruction, for example, copy the update instruction.
In step S604, the update command is parsed to obtain the target data table name and the update content.
In this embodiment, the database connection pool has a better parsing capability for the structured query statement, and by parsing the data control statement and the data definition statement in the structured query statement, changes of the table structure and the table data can be monitored and obtained, that is, a changed table and changed data parameters in the corresponding table are determined. Therefore, the table structure and the change of the table data of the data table do not need to be monitored from the database, and the decoupling function is realized on the interface of the data table.
FIG. 7 is a flowchart illustrating another method of search engine data synchronization, as shown in FIG. 7, according to an exemplary embodiment, the method further comprising, in one possible implementation, the steps of:
in step S701, a query instruction for the database is intercepted.
In this embodiment, the structured query statement may include query instructions, and an interceptor for the query instructions in the structured query statement may be established from the database connection pool.
In step S702, package format data of the database for the query result is acquired.
In this embodiment, the package format data is a manner in which the database packages the query result.
In step S703, a query instruction and package format data are sent to the search engine, where the query instruction is used for the search engine to obtain a query result according to the query instruction, and the package format data is used for the search engine to package the query result according to the package format data.
In this embodiment, the data table in the search engine is a data table chain corresponding to a plurality of associated chains, so that the data query speed can be increased in a redundant storage manner, and when a user queries data, a query instruction for a database is intercepted, so that the search engine is used for querying, the query speed can be increased, and the user experience can be improved. The query result is packaged by adopting the package format of the original database, so that analysis and display of the query result can not be affected.
FIG. 8 is a block diagram illustrating a search engine data synchronization apparatus according to an example embodiment. Referring to fig. 8, the apparatus 800 includes a first acquisition module 801, a construction module 802, an acquisition module 803, a determination module 804, and an update module 805.
The first obtaining module 801 is configured to obtain a plurality of data table names corresponding to the database and association relations between the plurality of data table names;
the construction module 802 is configured to construct a broad table structure of the search engine according to the plurality of data table names and the association relation among the plurality of data table names, where the broad table structure includes a plurality of association chains composed of the plurality of data table names;
the obtaining module 803 is configured to obtain an update instruction for the database, and determine a target data table name and update content corresponding to the update instruction;
the determining module 804 is configured to determine a target association chain including the target data table name in the wide table structure;
the updating module 805 is configured to update, according to the update content, a target data table corresponding to the target data table name included in the target association chain.
Optionally, the building module 802 includes:
the construction submodule is configured to construct an association chain taking the data table name as a chain head according to the association relation among the data table names aiming at any one of the data table names;
and the obtaining sub-module is configured to obtain the wide table structure of the search engine according to the constructed plurality of associated chains.
Optionally, the constructing sub-module includes:
a first construction unit configured to construct an association secondary tree according to the plurality of data table names and association relations between the plurality of data table names, a first level of the association secondary tree including the plurality of data table names, a second level of the association secondary tree including a subset of each of the data table names, each subset of the data table names being all data table names associated with the data table name;
and the second construction unit is configured to construct an association chain with any data table name in the first stage as a chain head according to the association secondary tree.
Optionally, the second building unit includes:
a first determining subunit configured to determine, for any one of the data table names in the first stage, the data table name as an associated chain corresponding to the data table name if a subset of the data table names are empty.
Optionally, the second building unit includes:
and the second determining subunit is configured to determine, for any data table name in the first stage, a chain obtained by sequentially connecting all data table names included in the subset of the data table names at the tail of the data table name with the data table name as a chain head when the subset of the data table names includes at least one data table name, as an associated chain corresponding to the data table name.
Optionally, the second building unit includes:
a third determination subunit configured to determine any one of the data table names in the first stage as a first data table name;
a first obtaining subunit, configured to, when a second data table name included in the subset of the first data table names exists in the first stage and the subset of the second data table names does not include the first data table name, sequentially connect all the data table names included in the subset of the first data table names to the tail of the first data table name with the first data table name as a chain head, so as to obtain a first chain;
a second obtaining subunit, configured to sequentially connect all the data table names included in the subset of the second data table names, so as to obtain a second chain;
And the third obtaining subunit is configured to insert the second chain into the tail part of the second data table name in the first chain to obtain an associated chain corresponding to the first data table name.
Optionally, the first obtaining module 801 includes:
the first acquisition sub-module is configured to acquire initial names of all data tables in the database and association relations among all the data tables;
the first obtaining sub-module is configured to set a mapping name corresponding to the initial name of each data table, and determine an association relationship between the data table names according to the association relationship between the data tables by taking the mapping name as the data table name of each data table, so as to obtain the structural data, wherein the length of the mapping name of each data table is smaller than that of the initial name of the data table.
Optionally, the obtaining module 803 includes:
the building sub-module is configured to build a monitor aiming at the structured query statement according to the database connection pool;
a first determination submodule configured to determine, by the monitor, whether a structured query statement for the database is a data control statement and a data definition statement;
A second determination submodule configured to determine the structured query statement as an update instruction for the database in response to either the structured query statement being the data control statement or the data definition statement;
and the second determining submodule is configured to analyze the updating instruction to obtain the name and the updating content of the target data table.
Optionally, the apparatus 800 further comprises:
an interception module configured to intercept a query instruction for the database;
the second acquisition module is configured to acquire package format data of the database aiming at the query result;
the sending module is configured to send the query instruction and the package format data to the search engine, the query instruction is used for the search engine to obtain a query result according to the query instruction, and the package format data is used for the search engine to package the query result according to the package format data.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Fig. 9 is a block diagram of an electronic device, according to an example embodiment. For example, the electronic device 900 may be provided as a server. Referring to fig. 9, the electronic device 900 includes a processor 922, which may be one or more in number, and a memory 932 for storing computer programs executable by the processor 922. The computer program stored in memory 932 may include one or more modules each corresponding to a set of instructions. Further, the processor 922 may be configured to execute the computer program to perform the search engine data synchronization method described above.
In addition, the electronic device 900 may further include a power supply component 926 and a communication component 950, the power supply component 926 may be configured to perform power management of the electronic device 900, and the communication component 950 may be configured to enable communication of the electronic device 900, e.g., wired or wireless communication. In addition, the electronic device 900 may also include an input/output interface 958. The electronic device 900 may operate based on an operating system stored in the memory 932.
In another exemplary embodiment, a computer readable storage medium is also provided that includes program instructions that, when executed by a processor, implement the steps of the search engine data synchronization method described above. For example, the non-transitory computer readable storage medium may be the memory 932 described above that includes program instructions executable by the processor 922 of the electronic device 900 to perform the search engine data synchronization method described above.
In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable apparatus, the computer program having code portions for performing the search engine data synchronization method described above when executed by the programmable apparatus.
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.
In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations are not described further in this disclosure in order to avoid unnecessary repetition.
Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.
Claims (12)
1. A method for synchronizing search engine data, comprising:
Acquiring a plurality of data table names corresponding to a database and an association relation among the plurality of data table names;
constructing a wide table structure of the search engine according to the data table names and the association relations among the data table names, wherein the wide table structure comprises a plurality of association chains formed by the data table names;
acquiring an update instruction aiming at the database, and determining a target data table name and update content corresponding to the update instruction;
determining a target association chain comprising the target data table name in the wide table structure;
and updating the target data table corresponding to the target data table name included in the target association chain according to the updating content.
2. The method for synchronizing data of a search engine according to claim 1,
the step of constructing the broad table structure of the search engine according to the plurality of data table names and the association relation among the plurality of data table names specifically comprises the following steps:
aiming at any one of the data table names, constructing an association chain taking the data table name as a chain head according to the association relation among the data table names;
And obtaining the wide table structure of the search engine according to the constructed association chains.
3. The method for synchronizing data of a search engine according to claim 2, wherein,
the step of constructing an association chain with the data table name as a chain head according to the association relation among the data table names aiming at any one of the data table names specifically comprises the following steps:
constructing an association secondary tree according to the data table names and the association relation among the data table names, wherein a first level of the association secondary tree comprises the data table names, a second level of the association secondary tree comprises a subset of each data table name, and each subset of the data table names is all data table names associated with the data table name;
and constructing an association chain taking any data table name in the first stage as a chain head according to the association secondary tree.
4. The method for synchronizing data of a search engine according to claim 3,
the step of constructing an associated chain with any data table name in the first stage as a chain head according to the associated secondary tree specifically comprises the following steps:
And determining the data table name as an associated chain corresponding to the data table name when the subset of the data table names is empty aiming at any data table name in the first stage.
5. The method for synchronizing data of a search engine according to claim 3,
the step of constructing an associated chain with any data table name in the first stage as a chain head according to the associated secondary tree specifically comprises the following steps:
and aiming at any data table name in the first stage, under the condition that the subset of the data table names comprises at least one data table name, taking the data table name as a chain head, and determining a chain obtained by sequentially connecting all the data table names included in the subset of the data table names at the tail of the data table name as an associated chain corresponding to the data table name.
6. The method for synchronizing data of a search engine according to claim 3,
the step of constructing an associated chain with any data table name in the first stage as a chain head according to the associated secondary tree specifically comprises the following steps:
determining that any data table name in the first level is a first data table name;
when the second data table names included in the subset of the first data table names exist in the first stage and the subset of the second data table names does not include the first data table names, taking the first data table names as chain heads, and sequentially connecting all the data table names included in the subset of the first data table names to the tail parts of the first data table names to obtain a first chain;
Sequentially connecting all data table names included in the subset of the second data table names to obtain a second chain;
and inserting the second chain into the tail part of the second data table name in the first chain to obtain an associated chain corresponding to the first data table name.
7. The method for synchronizing search engine data according to claim 1, wherein the step of obtaining a plurality of table names corresponding to the database and an association relationship between the plurality of table names specifically comprises:
acquiring initial names of all data tables in a database and association relations among all the data tables;
setting a mapping name corresponding to the initial name of each data table, taking the mapping name as the data table name of each data table, and determining the association relationship between the data table names according to the association relationship between the data tables, wherein the length of the mapping name of each data table is smaller than that of the initial name of the data table.
8. The method for synchronizing data of a search engine according to claim 1,
the step of acquiring the update instruction for the database and determining the name and the update content of the target data table corresponding to the update instruction specifically comprises the following steps:
Establishing a monitor aiming at the structured query statement according to the database connection pool;
determining, by the monitor, whether a structured query statement for the database is a data control statement and a data definition statement;
in response to any one of the structured query statement being the data control statement or a definition statement for the data, determining the structured query statement as an update instruction for the database;
and analyzing the updating instruction to obtain the name and the updating content of the target data table.
9. The method of claim 1, further comprising:
intercepting a query instruction for the database;
acquiring package format data of the database aiming at a query result;
and sending the query instruction and the package format data to the search engine, wherein the query instruction is used for the search engine to acquire a query result according to the query instruction, and the package format data is used for the search engine to package the query result according to the package format data.
10. A search engine data synchronization apparatus, comprising:
The first acquisition module is configured to acquire structural data corresponding to a database, wherein the structural data comprises a plurality of data table names and an association relation between each data table name;
a building module configured to build a broad-table structure of the search engine according to the structure data, the broad-table structure including a plurality of association chains composed of the plurality of data table names;
the obtaining module is configured to monitor an updating instruction aiming at the database to obtain a target data table name corresponding to a data table aimed at by the updating instruction and updating content included by the updating instruction;
a determining module configured to determine a target association chain in the wide table structure that includes the target data table name;
and the updating module is configured to update the target data table corresponding to the target data table name included in the target association chain according to the updating content.
11. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor implements the steps of the search engine data synchronization method of any of claims 1-9.
12. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the search engine data synchronization method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211528290.8A CN116069804A (en) | 2022-11-30 | 2022-11-30 | Search engine data synchronization method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211528290.8A CN116069804A (en) | 2022-11-30 | 2022-11-30 | Search engine data synchronization method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116069804A true CN116069804A (en) | 2023-05-05 |
Family
ID=86169065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211528290.8A Pending CN116069804A (en) | 2022-11-30 | 2022-11-30 | Search engine data synchronization method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116069804A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116955417A (en) * | 2023-09-19 | 2023-10-27 | 武汉大数据产业发展有限公司 | Optimization method and device for multi-table combined retrieval of data and electronic equipment |
-
2022
- 2022-11-30 CN CN202211528290.8A patent/CN116069804A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116955417A (en) * | 2023-09-19 | 2023-10-27 | 武汉大数据产业发展有限公司 | Optimization method and device for multi-table combined retrieval of data and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109299110B (en) | Data query method and device, storage medium and electronic equipment | |
| US20180060389A1 (en) | Query optimization over distributed heterogeneous execution engines | |
| US8682859B2 (en) | Transferring records between tables using a change transaction log | |
| CN107451208B (en) | Data searching method and device | |
| CN108228229B (en) | Method and system for managing Maven dependence | |
| CN111786814B (en) | Log processing method and device | |
| CN111209280A (en) | Database and table dividing method and device | |
| CN111026782B (en) | Remote database object optimization method, device, equipment and storage medium | |
| CN116069804A (en) | Search engine data synchronization method and device, electronic equipment and storage medium | |
| US20230289351A1 (en) | Mirror List Query method of Multi-Architecture Cloud Platform Mirror Warehouse | |
| CN113553339B (en) | Data query method, middleware, electronic device and storage medium | |
| CN107491463B (en) | Optimization method and system for data query | |
| EP3373158A1 (en) | Data storage method and coordinator node | |
| CN111930770A (en) | Data query method and device and electronic equipment | |
| US20090106216A1 (en) | Push-model based index updating | |
| CN114756569A (en) | Multi-layer parsing method of structured query statement, computer device and storage medium | |
| CN112506964A (en) | Data query method, system and computer readable storage medium | |
| CN112905620B (en) | Data query method and device, electronic equipment and storage medium | |
| US20090106324A1 (en) | Push-model based index deletion | |
| CN104536785B (en) | Real-time system update method and device | |
| CN109684351B (en) | Execution plan viewing method, device, server and storage medium | |
| CN117312643A (en) | Data query method and device based on online analysis engine and electronic equipment | |
| CN111767282A (en) | MongoDB-based storage system, data insertion method and storage medium | |
| CN104536784B (en) | Real-time system update method and device | |
| CN111555981A (en) | Data transmission method, switch and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |