CN117216109A - Data query method, device and storage medium for multi-type mixed data - Google Patents
Data query method, device and storage medium for multi-type mixed data Download PDFInfo
- Publication number
- CN117216109A CN117216109A CN202311182733.7A CN202311182733A CN117216109A CN 117216109 A CN117216109 A CN 117216109A CN 202311182733 A CN202311182733 A CN 202311182733A CN 117216109 A CN117216109 A CN 117216109A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- user
- target result
- content information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 21
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 6
- 238000007726 management method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data query method, a device and a storage medium for multi-type mixed data, which respond to receiving a first query instruction initiated by a current user according to a preset query language system; obtaining a corresponding routing rule according to the user query content information; obtaining at least two second query instructions based on the routing rule; inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions; and returning the target result data to the current user. The application can shield different data source details of the bottom layer, and enables a user to realize one-key inquiry of cross-system multi-type mixed data through a unified statement format, thereby realizing the fusion of various mixed data, simplifying the data inquiry operation of the user and improving the data inquiry efficiency.
Description
Technical Field
The present application relates to the field of computer data technologies, and in particular, to a method and apparatus for querying data of multiple types of hybrid data, and a storage medium.
Background
With the rapid development of computers and information technologies, the quantity and types of information data are rapidly increasing, and the massive characteristics and heterogeneous characteristics of the information data bring great challenges to traditional database technologies, particularly centralized databases.
In order to provide distributed support for open source centralized databases such as MySQL (a relational database management system), postgreSQL (an object-relational database management system with very complete characteristics of free software) and the like which are widely used at present, a series of database middleware is generated, and the middleware provides a transparent scheme for constructing a database cluster for users, so that the existing single-machine centralized database and applications can be smoothly migrated to a cloud end, and the solution becomes an important distributed data management solution. Meanwhile, the distributed database middleware can integrate different types of bottom databases and applications, and if the relational database and the NoSQL (Not Only SQL) database are integrated uniformly at the bottom, the distributed database middleware is expected to carry out self-adaptive storage and query management on mixed data with different sources and different structures, so that effective management of heterogeneous big data is realized.
However, the current SQL (Structured Query Language ) statement has limited query functions, and cannot be applied to all types of databases, so that the conventional query operations such as connection, grouping and the like of mixed type data cannot be supported, and the data query of multi-type mixed data is inconvenient.
Disclosure of Invention
The application aims to solve the technical problems of the prior art, and provides a data query method, a device and a storage medium for multi-type mixed data, which are used for solving the problem of inconvenient mixed data query in the prior art.
In a first aspect, the present application provides a data query method for multi-type hybrid data, the method comprising:
s1, responding to a first query instruction initiated by a current user according to a preset query language system, and carrying out statement analysis on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
s2, obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
S3, obtaining at least two second query instructions based on the routing rule, wherein the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located;
s4, inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions;
s5, returning the target result data to the current user.
In some embodiments, S3 comprises:
s31, acquiring a preset data source conversion rule, wherein the preset data source conversion rule represents a query field and conversion relations between query conditions and executable query sentences of different types of data sources;
s32, generating at least two second query instructions according to the preset data source conversion rules according to query fields and query conditions of the target result data in at least two different types of data sources contained in the routing rules.
In some embodiments, S5 comprises:
s51, if the result data corresponding to the user query content information contains original data in a data source, converting the target result data into standard formatted data, and returning the converted standard formatted data to the current user;
And/or the number of the groups of groups,
s52, if the result data corresponding to the user query content information contains processing data obtained based on original data in a data source, converting the target result data into standard formatted data, performing data processing based on the standard formatted data to obtain processed data, and returning the processed data obtained by processing to the current user.
In some embodiments, after S1, before S2, further comprising:
judging whether target result data corresponding to the user query content information exists in the cache, and if so, returning the target result data to the current user;
if not, s2 is performed.
In some embodiments, further comprising:
s01, registering data of an original multi-type data source to obtain the multi-type database, wherein the original multi-type data source comprises at least two of a structured data source, an unstructured data source and an object data source;
s02, establishing a basic query view of the multi-type database;
s03, constructing the preset query language system based on the basic query view.
In some embodiments, further comprising:
s04, performing role authority registration based on different user identities to obtain role authority data of different users;
S3, including:
and acquiring the role authority data corresponding to the current user, and acquiring the at least two second query instructions based on the routing rule and the role authority data corresponding to the current user.
In some embodiments, further comprising:
s6, caching the user query content information and the corresponding target result data, and updating the cached content based on a least recently used algorithm.
In a second aspect, the present application provides a data query device for multi-type mixed data, the device comprising:
the query statement analysis module is used for responding to the received first query instruction initiated by the current user according to a preset query language system, and carrying out statement analysis on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
the routing rule determining module is used for obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
The query instruction generation module is used for obtaining at least two second query instructions based on the routing rule, and the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located;
the result data acquisition module is used for inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions;
and a result data return module configured to return the target result data to the current user.
In a third aspect, the present application provides a data query device for multi-type mixed data, including: the system comprises a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the data query method of the multi-type mixed data.
In a fourth aspect, the present application provides a computer readable storage medium, on which a computer program is stored, the computer program implementing the data query method of multi-type hybrid data according to the first aspect, when executed by a processor.
The application provides a data query method, a device and a readable storage medium of multi-type mixed data, and particularly, in response to receiving a first query instruction initiated by a current user according to a preset query language system, statement analysis is carried out on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources; obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources; obtaining at least two second query instructions based on the routing rule, wherein the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located; inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions; and returning the target result data to the current user. The application can shield the details of different data sources at the bottom layer, namely, the user does not need to input query sentences corresponding to different data sources, but realizes one-key query of cross-system multi-type mixed data through a unified sentence format, and realizes the fusion of various mixed data, thereby simplifying the data query operation of the user and improving the data query efficiency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic view of a scenario in the prior art of a user performing a data query on multi-type hybrid data across systems;
FIG. 2 is a schematic diagram of a data query method for multi-type hybrid data according to an embodiment of the present application;
FIG. 3 is a schematic view of a scenario in which a user performs data query on multi-type hybrid data across systems based on the data query method of the present application;
FIG. 4 is another schematic diagram of a data query method for multi-type hybrid data according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a data query device for multi-type hybrid data according to an embodiment of the present application;
fig. 6 is another schematic diagram of a data query device for multi-type mixed data according to an embodiment of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
In order to make the technical scheme of the present application better understood by those skilled in the art, the following detailed description of the embodiments of the present application will be given with reference to the accompanying drawings.
It is to be understood that the specific embodiments and figures described herein are merely illustrative of the application, and are not limiting of the application.
It is to be understood that the various embodiments of the application and the features of the embodiments may be combined with each other without conflict.
It is to be understood that only the portions relevant to the present application are shown in the drawings for convenience of description, and the portions irrelevant to the present application are not shown in the drawings.
It should be understood that each unit and module in the embodiments of the present application may correspond to only one physical structure, may be formed by a plurality of physical structures, or may be integrated into one physical structure.
It will be appreciated that the terms "first," "second," and the like in embodiments of the present application are used to distinguish between different objects or to distinguish between different processes on the same object, and are not used to describe a particular order of objects.
It will be appreciated that, without conflict, the functions and steps noted in the flowcharts and block diagrams of the present application may occur out of the order noted in the figures.
It is to be understood that the flowcharts and block diagrams of the present application illustrate the architecture, functionality, and operation of possible implementations of systems, apparatuses, devices, methods according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a unit, module, segment, code, or the like, which comprises executable instructions for implementing the specified functions. Moreover, each block or combination of blocks in the block diagrams and flowchart illustrations can be implemented by hardware-based systems that perform the specified functions, or by combinations of hardware and computer instructions.
It should be understood that the units and modules related in the embodiments of the present application may be implemented by software, or may be implemented by hardware, for example, the units and modules may be located in a processor.
In a real scenario, there are often situations where a user needs to perform a data query across data sources, for example, need to query information of an employee, the employee needs to write papers on "big data, artificial intelligence, technology management", and the like, and need to query wage and business age information of the employee. The wage and age information of the staff needs to be queried in the traditional database, because the information is often stored in the MySQL database, but the data for counting the publication of the staff articles needs to be queried in the document database system. In this case, a query person is required to perform a query operation back and forth across systems.
Fig. 1 is a schematic view of a scenario in the prior art of a user performing data query on cross-system multi-type hybrid data, as shown in fig. 1, for the current cross-system multi-type hybrid data, for each type of data source, the corresponding query statement format is different.
For example, assume that table a is from a traditional SQL database, such as MySQL, and the corresponding query statement format is: select a. Field to be queried b From a while a. Condition field = value;
for another example, assume that table b is from an Hbase database (a distributed, column-oriented, open source database), and the corresponding query statement format is: "Get ' Xxxx ', ' column group i: qualifier k' ".
Referring to fig. 1, in the prior art, if a user needs to query data in a data source 1, a query sentence 1 in a first format needs to be input; if the user needs to query the data in the data source 2, a query sentence 2 in the second format needs to be input, and if the user needs to query the data in the data source 3, a query sentence 3 in the third format needs to be input.
Based on the above situation, because the query function of the current SQL statement is limited, the method cannot be suitable for all types of databases, and cannot support the connection, grouping and other common query operations of mixed type data, when a user queries a certain type of data source through a computer, the user needs to input a query statement in a corresponding format, so that the user queries the data source with complicated operation, low query efficiency and inconvenience in data query of multiple types of mixed data.
The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a schematic diagram of a data query method of multi-type hybrid data according to an embodiment of the present application, and fig. 3 is a schematic diagram of a scenario in which a user performs data query on multi-type hybrid data across systems based on the data query method of the present application, where, as shown in fig. 2 and fig. 3, the present application provides a data query method of multi-type hybrid data, where the method includes:
s1, responding to a first query instruction initiated by a current user according to a preset query language system, and carrying out statement analysis on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
specifically, the current user can perform data query on multi-type mixed data of a cross-system through electronic equipment (for convenience of understanding, the application uses a computer as an example for explanation), and when performing data query, the current user only needs to input a first query instruction according to a preset query language system, that is, the user only needs to input a query statement in a preset format (namely, a preset query language system).
And for the computer, after receiving a first query instruction initiated by the current user according to a preset query language system, obtaining the user query content information by carrying out statement analysis on the first query instruction.
The preset query language system may be, for example, the following format:
Select-Select returned content
From-selection data table
Where-defined return condition
After the computer acquires the first query instruction, according to the content of the from, the data table to be accessed can be determined, and the content of the select and the where can be combined, the user query content information can be determined. The return condition may include, among other things, a cross-table constraint, a table internal constraint, and the like.
S2, obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
after the user query content information is obtained through statement analysis, the computer further forms a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources, and the routing rule is used for representing data screening conditions corresponding to different data sources, namely, corresponding data content is searched in different data sources according to the routing rule.
Specifically, the routing rule may be, for example, the following:
table a: querying field 1; query condition 1
Table b: querying field 2; query condition 2
Table c: query field 3; query condition 3
S3, obtaining at least two second query instructions based on the routing rule, wherein the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located;
after forming the routing rule, the computer generates a corresponding second query instruction according to the routing rule, wherein each second query instruction has the same format as the query statement of the corresponding data source.
For example, in conjunction with the routing rule provided in s2 above, the computer generates the following three second query instructions:
s4, inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions;
after generating at least two second query instructions, the computer queries data of at least two different types of data sources according to the generated second query instructions, so that the target result data is obtained.
For example, in combination with the second query instruction provided in s3, the computer obtains data corresponding to the query field 1 and the query condition 1 from the table a as data1, obtains data corresponding to the query field 2 and the query condition 2 from the table b as data2, obtains data corresponding to the query field 3 and the query condition 3 from the table c as data3, and obtains the target result data as data1+data2+data3.
S5, returning the target result data to the current user.
After the target result data is obtained by inquiry, the computer returns the target result data to the current user, so that the current user can realize data inquiry of cross-system multi-type mixed data only by inputting inquiry sentences in one format.
Alternatively, when the target result data is returned to the current user, the computer may display the target result data in a data view in Json format. The Json format is a universal format for front-end display, and can facilitate the subsequent identification conversion for data display.
Based on the processing procedure, the application provides a data query method of multi-type mixed data, which can shield the details of different data sources at the bottom layer, namely, a user does not need to input query sentences corresponding to different data sources, and the user can realize one-key query of cross-system multi-type mixed data through a unified sentence format, so that the fusion of various mixed data is realized, thereby simplifying the data query operation of the user and improving the data query efficiency.
In some embodiments, S3 obtains at least two second query instructions based on the routing rule, including:
S31, acquiring a preset data source conversion rule, wherein the preset data source conversion rule represents a query field and conversion relations between query conditions and executable query sentences of different types of data sources;
s32, generating at least two second query instructions according to the preset data source conversion rules according to query fields and query conditions of the target result data in at least two different types of data sources contained in the routing rules.
Specifically, a user may set a preset data source conversion rule in the computer, where the preset data source conversion rule characterizes a query field and a conversion relationship between a query condition and executable query statements of different types of data sources. After the computer acquires the routing rule subsequently, the computer can generate a corresponding second query instruction according to the preset data source conversion rule.
For example, assuming that the table a is from a conventional SQL database, such as MySQL, according to a preset data source conversion rule, the second query instruction format correspondingly generated is:
select a. Field to be queried 1, a. Field to be queried 2From a while a. Condition field = value
For another example, assuming that table b is from the Hbase database, the routing rule is that the query field is "column family 1: qualifier 1", query condition is rowkey=" Xxxx "; according to a preset data source conversion rule, the format of the second query instruction correspondingly generated is as follows:
"Get ' Xxxx ', ' column group 1: qualifier 1'
For another example, assuming that the table c is from Select Name, age From C Where Age >25, according to a preset data source conversion rule, the second query instruction format correspondingly generated is:
Db.C.Find({Age:{$Gt:25}}，{Name:1，Age:1})；
in this embodiment, the computer generates the at least two second query instructions according to the preset data source conversion rule according to the query fields and the query conditions of the target result data in the at least two different types of data sources included in the routing rule, so that the computer can complete corresponding query actions according to the obtained second query instructions, thereby completing access to different data sources to obtain corresponding target result data, enabling a user to realize one-key query of cross-system multi-type hybrid data, and realizing fusion of various hybrid data.
In some embodiments, S5 returns the target result data to the current user, including:
s51, if the result data corresponding to the user query content information contains original data in a data source, converting the target result data into standard formatted data, and returning the converted standard formatted data to the current user;
And/or the number of the groups of groups,
s52, if the result data corresponding to the user query content information contains processing data obtained based on original data in a data source, converting the target result data into standard formatted data, performing data processing based on the standard formatted data to obtain processed data, and returning the processed data obtained by processing to the current user.
In this embodiment, for the determination of whether or not the data processing is required, the computer may determine based on the statement analysis result in step S1, for example, if an operator portion, such as an ave (average value) operator, is also involved in the statement analysis result, it is determined that further data processing is required, that is, it is determined that the result data required by the user includes processing data obtained based on the original data in the data source.
Specifically, in an actual scenario, the following three cases may occur:
(1) The user only needs to query the original data in the data source;
(2) The user only needs to obtain the processing data according to the original data;
(3) The user needs to query both the raw data in the data source and the processed data derived from the raw data.
For the case (1) in the above scenario, after obtaining the target result data, the computer may directly display the data view of the target result data in Json format to the user.
For the cases (2) and (3) in the above scenario, after obtaining the target result data, the computer needs to further perform data processing, and for such an access query scenario of mixed complex data, this embodiment is implemented in a two-stage query mode of "first simplifying query and then complex query", where the simplifying query refers to obtaining the original data in the data source by only making table association, and the complex query refers to obtaining the corresponding processing data by processing the original data according to the operator part involved in the statement analysis result.
In addition, since the return data is standard formatted data, a standard structured data view can be formed such that the user can use standard SQL to do further operations on the view.
In this embodiment, for the access query scene of the mixed complex data, the more complex language supports, the greater the pressure on the corresponding processing module and the more influencing the efficiency, so that the problem is easily caused.
Fig. 4 is another schematic diagram of a data query method for multi-type hybrid data according to an embodiment of the present application, as shown in fig. 4, in some embodiments, after S1 obtains user query content information, S2 further includes:
s11, judging whether target result data corresponding to the user query content information exists in a cache, and if so, returning the target result data to the current user; if not, s2 is performed.
In addition, after S5 returns the target result data to the current user, the method further includes: s6, caching the user query content information and the corresponding target result data, and updating the cached content based on a least recently used algorithm.
In this embodiment, after each time the target result data is returned, the computer caches the user query content information and the corresponding target result data, that is, caches the content in the device memory, and updates the cached content based on an LRU (Least Recently Used ) algorithm.
Specifically, the computer may allocate a portion of the device memory for caching the user query content information and the corresponding target result data, where the portion of the device memory has a certain data storage capacity, so when new data is cached, the cached content needs to be updated, at this time, the computer may use an LRU algorithm to update the cached content, and the algorithm may select the data content that is not used the longest recently to eliminate, so as to store the popular data that is accessed frequently recently.
After acquiring the user query content information, in order to improve the data acquisition speed, the computer can firstly judge whether target result data corresponding to the user query content information exists in the cache, and if so, the computer can directly return the target result data to the current user according to the cache content; if not, the processing flow of step S2 is executed again.
In this embodiment, the computer may cache the content queried by the user, and when the user performs the next query, if there is corresponding result data in the cache, the computer may directly perform data return according to the cache content.
In some embodiments, prior to step S1, further comprising:
s01, registering data of an original multi-type data source to obtain the multi-type database, wherein the original multi-type data source comprises at least two of a structured data source, an unstructured data source and an object data source;
specifically, for traditional structured data sources, a schema (database object set) of the registry database is required, including information such as tables, columns, data types, views, stored procedures, relationships, primary keys, foreign keys, and the like.
In addition, for unstructured data sources, i.e. noSQL (Not Only Sql) type of non-relational databases, schema data, such as Hbase, return table, row_key, column family, column qualifier, etc. may also be returned.
In addition, for an object class data source, such as document class data, metadata (data to be registered) thereof is a field of document name, document type, author, time, keyword, etc.
Specifically, for document class data, topic classification is required, and the topic classification is that search contents are not limited to words which appear in the document, but more importantly, the document can be extracted according to the topic. Such as a document, which includes content that is a vocabulary of user's retention rate, revenue retention rate, etc., it is essentially a "stock administration" subject document. The purpose of this step is to extract topic names that may not appear in the article.
Optionally, when classifying the subject of the document class data, the following method may be adopted:
(1) Based on the document with the clear theme, an LDA (Latent Dirichlet Allocation, hidden Dirichlet distribution) model is adopted for training, the existing document is applied according to the training result, the theme of the document is extracted, and the result is updated into the document class data.
(2) Keyword extraction is realized based on a TF-IDF (Term Frequency-inverse text Frequency index) model, and words with higher importance of the documents are extracted and used as main query fields of articles.
S02, establishing a basic query view of the multi-type database;
specifically, based on the registration result obtained in step S01, a corresponding basic query view may be established for the external query user.
Alternatively, the base query view may be, for example, in the following format:
table a, column 1, column 2, column 3
Table b, column 1, column 2, column 3
Table 3, rowkey, column 1: qualifier 1, column 1: qualifier 2, column 2
Document name, topic, author name
Wherein the column names may be determined based on header content of the respective data sources.
S03, constructing the preset query language system based on the basic query view.
After the basic query view is established, a corresponding preset query language system is constructed based on the view.
Optionally, the preset query language system may be in the following format:
select Table a, column 1, table 3, column 2, qualifier 1
From Table a, table 3
Where table a. Column 2 = table 3.Rowkey name = Id And table 3. Rowkey= "Xxx" -employee Id
Based on the preset query language system, the user can select the returned content by using the language system, the From selects the data table, and the white limits the returned condition, and the preset query language system is a simplified version imitating the standard SQL language.
Alternatively, if a new data source needs to be added or an existing data source needs to be modified, only the data registration work of the data source needs to be repeated.
In this embodiment, before the query step of the present application is executed, data registration is first performed according to an original data source, and then a corresponding basic query view and a preset query language system are constructed, so that a user can perform one-key query of multiple types of mixed data across systems by adopting query sentences of the preset query language system based on the basic query view, and various types of mixed data are melted through, thereby simplifying data query operation of the user and improving data query efficiency.
In some embodiments, prior to step S1, further comprising:
s04, performing role authority registration based on different user identities to obtain role authority data of different users;
s3, obtaining at least two second query instructions based on the routing rule, wherein the second query instructions comprise:
and acquiring the role authority data corresponding to the current user, and acquiring the at least two second query instructions based on the routing rule and the role authority data corresponding to the current user.
Specifically, in an actual scenario, considering that access rights of different roles to different data sources are different, the embodiment further includes a processing step of performing role rights registration on different user identities, thereby ensuring privacy and security of the data sources.
For example, the multi-type database includes two tables, one is a basic information table USERS (user), which includes information such as USERID (user identification), USERNAMEs (user name), phone (telephone), MAIL (mailbox), AGE (MAIL), DEPTID (DEPARTMENT identification), and the other is a DEPARTMENT table Departs (DEPARTMENT), which includes information such as DEPTID (DEPARTMENT identification), DEPTNAME (DEPARTMENT name), direction (DEPARTMENT management), and the like.
There are two access roles for the database, role A having the right to read the USERS and DEPARTMENT tables, and role B having only the right to read the DEPARTMENT tables. By performing role authority registration, access authorities of different data tables can be limited for the roles A and B, and the condition of unauthorized access, such as accessing data of a USERS table by the role B, is avoided.
When the role authority registration of different users is carried out, the role authority data of the different users can be obtained, and the content comprises a user name and corresponding tables with readable authorities.
When the computer obtains at least two second query instructions based on the routing rule, the embodiment further comprises a processing flow for verifying role authority except the routing rule, namely judging whether the current user has readable access authority to a data source where target result data are located according to the role authority data corresponding to the current user, and if the current user has the readable access authority, the corresponding second query instruction content is the data source corresponding to query access; if the current user does not have the readable access right, the corresponding second query instruction content is that the corresponding data source is not accessed, and at the moment, the returned result data is null.
In the embodiment, the role authority registration is performed based on different user identities, so that the computer can perform data query according to the role authority data of different users, strictly according to the role authority data of the users, thereby ensuring the limitation of the application range of the bottom layer source data and being beneficial to improving the privacy and the safety of the data source.
In some embodiments, when a user performs query processing of cross-system multi-type mixed data, the computer automatically records contents such as query operation sentences of the user, related underlying data sources, target result data and the like, and can form corresponding data access logs so as to reserve data access evidence and facilitate tracing work of problems such as follow-up data leakage and the like.
It should be understood that, although the steps in the flowcharts in the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily occurring in sequence, but may be performed alternately or alternately with other steps or at least a portion of the other steps or stages.
Fig. 5 is a schematic diagram of a data query device for multi-type mixed data according to an embodiment of the present application, and as shown in fig. 5, the present application provides a data query device for multi-type mixed data, where the device includes:
the query statement analyzing module 11 is configured to respond to receiving a first query instruction initiated by a current user according to a preset query language system, and analyze the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
a routing rule determining module 12, configured to obtain a corresponding routing rule according to the user query content information, where the routing rule includes a query field and a query condition of target result data corresponding to the user query content information in at least two different types of data sources;
the query instruction generating module 13 is configured to obtain at least two second query instructions based on the routing rule, where the statement format of the second query instructions is matched with at least two different types of data sources where the target result data is located;
A result data obtaining module 14 configured to query and obtain the target result data from the at least two different types of data sources according to the at least two second query instructions;
a result data return module 15 arranged to return the target result data to the current user.
The application provides a data query device for multi-type mixed data, which can shield the details of different data sources at the bottom layer, namely, a user does not need to input query sentences corresponding to different data sources, and the user can realize one-key query of cross-system multi-type mixed data through a unified sentence format, so that the fusion of various mixed data is realized, the data query operation of the user can be simplified, and the data query efficiency is improved.
Regarding the limitation of the data query device for multi-type mixed data, reference may be made to the limitation of the data query method for multi-type mixed data in the above embodiments of the present application, and this embodiment is not repeated here.
Fig. 6 is another schematic diagram of a data query device for multi-type mixed data according to an embodiment of the present application, and as shown in fig. 6, the present application provides a data query device for multi-type mixed data, including:
The parser 21 is configured to parse the first query instruction in a sentence in response to receiving the first query instruction initiated by the current user according to the preset query language system, so as to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
a data router 22 configured to obtain a corresponding routing rule according to the user query content information, where the routing rule includes query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
a converter 23, configured to obtain at least two second query instructions based on the routing rule, where a statement format of the second query instructions matches at least two different types of data sources where the target result data is located;
a processor 24 configured to query the at least two different types of data sources for the target result data according to the at least two second query instructions; and returning the target result data to the current user.
The application provides a data query device for multi-type mixed data, which can shield the details of different data sources at the bottom layer, namely, a user does not need to input query sentences corresponding to different data sources, and the user can realize one-key query of cross-system multi-type mixed data through a unified sentence format, so that the fusion of various mixed data is realized, the data query operation of the user can be simplified, and the data query efficiency is improved.
Regarding the limitation of the data query device for multi-type mixed data, reference may be made to the limitation of the data query method for multi-type mixed data in the above embodiments of the present application, and this embodiment is not repeated here.
In some embodiments, the present application provides a data query device for multi-type mixed data, comprising: the system comprises a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the data query method of the multi-type mixed data.
In some embodiments, the present application provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the data query method of the multi-type hybrid data in the above embodiments of the present application.
Computer-readable storage media include volatile or nonvolatile, removable or non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, computer program modules or other data. Computer-readable storage media includes, but is not limited to, ram (Random Access Memory ), rom (Read-Only Memory), eeprom (Electrically Erasable Programmable Read Only Memory, electrically erasable programmable Read-Only Memory), flash Memory or other Memory technology, cd-Rom (Compact Disc Read-Only Memory), digital versatile disk (Dvd) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.
It is to be understood that the above embodiments are merely illustrative of the application of the principles of the present application, but not in limitation thereof. Various modifications and improvements may be made by those skilled in the art without departing from the spirit and substance of the application, and are also considered to be within the scope of the application.
Claims (10)
1. A method for querying data of multiple types of mixed data, the method comprising:
s1, responding to a first query instruction initiated by a current user according to a preset query language system, and carrying out statement analysis on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
s2, obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
s3, obtaining at least two second query instructions based on the routing rule, wherein the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located;
S4, inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions;
s5, returning the target result data to the current user.
2. The data query method of multi-type mixed data according to claim 1, wherein S3 comprises:
s31, acquiring a preset data source conversion rule, wherein the preset data source conversion rule represents a query field and conversion relations between query conditions and executable query sentences of different types of data sources;
s32, generating at least two second query instructions according to the preset data source conversion rules according to query fields and query conditions of the target result data in at least two different types of data sources contained in the routing rules.
3. The data query method of multi-type mixed data according to claim 1, wherein S5 comprises:
s51, if target result data corresponding to the user query content information contains original data in a data source, converting the target result data into standard formatted data, and returning the converted standard formatted data to the current user;
And/or the number of the groups of groups,
s52, if the target result data corresponding to the user query content information contains processing data obtained based on original data in a data source, converting the target result data into standard formatted data, performing data processing based on the standard formatted data to obtain processed data, and returning the processed data obtained by processing to the current user.
4. The method for querying data of multiple types of mixed data according to claim 1, wherein after S1 and before S2, further comprising:
judging whether target result data corresponding to the user query content information exists in the cache, and if so, returning the target result data to the current user;
if not, s2 is performed.
5. The data query method of multi-type hybrid data of any one of claims 1-4, further comprising:
s01, registering data of an original multi-type data source to obtain the multi-type database, wherein the original multi-type data source comprises at least two of a structured data source, an unstructured data source and an object data source;
s02, establishing a basic query view of the multi-type database;
S03, constructing the preset query language system based on the basic query view.
6. The data query method of multi-type hybrid data of claim 5, further comprising:
s04, performing role authority registration based on different user identities to obtain role authority data of different users;
s3, including:
and acquiring the role authority data corresponding to the current user, and acquiring the at least two second query instructions based on the routing rule and the role authority data corresponding to the current user.
7. The data query method of multi-type hybrid data of claim 1, further comprising:
s6, caching the user query content information and the corresponding target result data, and updating the cached content based on a least recently used algorithm.
8. A data query device for multiple types of mixed data, the device comprising:
the query statement analysis module is used for responding to the received first query instruction initiated by the current user according to a preset query language system, and carrying out statement analysis on the first query instruction to obtain user query content information; the preset query language system is constructed based on a multi-type database, and the multi-type database is obtained based on at least two different types of data sources;
The routing rule determining module is used for obtaining a corresponding routing rule according to the user query content information, wherein the routing rule comprises query fields and query conditions of target result data corresponding to the user query content information in at least two different types of data sources;
the query instruction generation module is used for obtaining at least two second query instructions based on the routing rule, and the statement format of the second query instructions is matched with at least two different types of data sources where the target result data are located;
the result data acquisition module is used for inquiring and acquiring the target result data from the at least two different types of data sources according to the at least two second inquiry instructions;
and a result data return module configured to return the target result data to the current user.
9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, which when executed by the processor performs a data querying method for multi-type hybrid data according to any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements a data query method of multi-type hybrid data as claimed in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311182733.7A CN117216109A (en) | 2023-09-13 | 2023-09-13 | Data query method, device and storage medium for multi-type mixed data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311182733.7A CN117216109A (en) | 2023-09-13 | 2023-09-13 | Data query method, device and storage medium for multi-type mixed data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117216109A true CN117216109A (en) | 2023-12-12 |
Family
ID=89040141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311182733.7A Pending CN117216109A (en) | 2023-09-13 | 2023-09-13 | Data query method, device and storage medium for multi-type mixed data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117216109A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117453731A (en) * | 2023-12-22 | 2024-01-26 | 北京宇信科技集团股份有限公司 | Multi-source data query system and multi-source data query method |
-
2023
- 2023-09-13 CN CN202311182733.7A patent/CN117216109A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117453731A (en) * | 2023-12-22 | 2024-01-26 | 北京宇信科技集团股份有限公司 | Multi-source data query system and multi-source data query method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bizer et al. | Linked data-the story so far | |
| CN107038207B (en) | Data query method, data processing method and device | |
| CN110941612B (en) | Autonomous data lake construction system and method based on associated data | |
| US10394803B2 (en) | Method and system for semantic-based queries using word vector representation | |
| US11941034B2 (en) | Conversational database analysis | |
| CN110866123B (en) | Method for constructing data map based on data model and system for constructing data map | |
| US8380750B2 (en) | Searching and displaying data objects residing in data management systems | |
| US9785725B2 (en) | Method and system for visualizing relational data as RDF graphs with interactive response time | |
| US11334549B2 (en) | Semantic, single-column identifiers for data entries | |
| CN108874971A (en) | A kind of tool and method applied to the storage of magnanimity labeling solid data | |
| US20220083618A1 (en) | Method And System For Scalable Search Using MicroService And Cloud Based Search With Records Indexes | |
| CN105468605A (en) | Entity information map generation method and device | |
| CN108509543B (en) | Streaming RDF data multi-keyword parallel search method based on Spark Streaming | |
| US20140122455A1 (en) | Systems and Methods for Intelligent Parallel Searching | |
| CN112000773B (en) | Search engine technology-based data association relation mining method and application | |
| US11449477B2 (en) | Systems and methods for context-independent database search paths | |
| CN106874425B (en) | Storm-based real-time keyword approximate search algorithm | |
| CN111967761A (en) | Monitoring and early warning method and device based on knowledge graph and electronic equipment | |
| CN104750776A (en) | Accessing information content in a database platform using metadata | |
| US10445370B2 (en) | Compound indexes for graph databases | |
| CN117216109A (en) | Data query method, device and storage medium for multi-type mixed data | |
| CN105095436A (en) | Automatic modeling method for data of data sources | |
| CN113190687A (en) | Knowledge graph determining method and device, computer equipment and storage medium | |
| CN115422155A (en) | Modeling method of data lake metadata model | |
| US11068459B2 (en) | Computer implemented and computer controlled method, computer program product and platform for arranging data for processing and storage at a data storage engine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |