CN116150211B - Multi-data source query method, platform and application system - Google Patents
Multi-data source query method, platform and application system Download PDFInfo
- Publication number
- CN116150211B CN116150211B CN202310410511.XA CN202310410511A CN116150211B CN 116150211 B CN116150211 B CN 116150211B CN 202310410511 A CN202310410511 A CN 202310410511A CN 116150211 B CN116150211 B CN 116150211B
- Authority
- CN
- China
- Prior art keywords
- data
- module
- query
- platform
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000013500 data storage Methods 0.000 claims description 36
- 238000006243 chemical reaction Methods 0.000 claims description 34
- 238000007726 management method Methods 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000013479 data entry Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a multi-data source query method, a platform and an application system. The multi-data source query platform can define the structure of the data in advance according to the required conditions, prepares the data according to the conditions in the synchronization process and writes the data into other data sources, improves the query efficiency by changing the time with space, and reduces the storage space.
Description
Technical Field
The present application relates to the field of data source query technologies, and in particular, to a multi-data source query method, platform, and application system.
Background
Most of the current use is that some sub-table sub-library components or companies solve the current problems in a form of business large table assembly. Because service tables typically use relational databases, relational databases can lead to performance degradation when a certain amount of data is reached. The problem of mass data of people cannot be solved. The sub-table sub-library component is not applicable in some specific scenes and has poor performance.
Disclosure of Invention
The application provides a multi-data source query method, a platform and an application system. The multi-data source query platform can define the structure of the data in advance according to the required conditions, prepares the data according to the conditions in the synchronization process and writes the data into other data sources, improves the query efficiency by changing the time with space, and reduces the storage space. The application adopts the following technical scheme:
the multi-data source query platform comprises an increment synchronization module, a full synchronization module, a data query module, a structure management module, a data conversion module and a data writing module; wherein,,
the incremental synchronization module is respectively connected with a plurality of data sources and the message queue MQ and is used for monitoring the data changes of the plurality of data sources, and after the data changes, the changed data are obtained and forwarded to the message queue MQ for processing;
the full synchronization module is respectively connected with a plurality of data sources and the message queue MQ, and is used for inquiring data in batches from the plurality of data sources and forwarding the inquired data to the message queue MQ for processing;
the data query module is respectively connected with the service module, the structure management module and the data storage device, and is used for realizing the JDBC protocol, analyzing according to SQL sentences transmitted from the service module, performing data query corresponding to languages which can be identified by the data storage device, and feeding back query results to the service module;
the structure management module is respectively connected with the data conversion module, the data query module and the data storage and is used for querying and managing the storage structure of the data in the data storage device;
the data conversion module is respectively connected with the message queue MQ, the structure management module and the data writing module and is used for correspondingly converting the data in the main table and the sub-table according to the structure of the data storage in the data storage device determined by the structure management module and forwarding the converted data to the data writing module;
the data writing module is respectively connected with the data conversion module and the data storage device, and writing of data is completed through adding, modifying or deleting according to the data received from the data conversion module.
Further, the structure management module obtains the storage structure of the data from the data conversion module and the data query module respectively, and determines that the table relationship between the main table and the sub-table is one-to-one or one-to-many by distinguishing the main table from the sub-table.
Further, the data storage structure in the data storage device is a large data table structure, and each user corresponds to one piece of data in the large data table structure.
Further, the big data table structure is composed of a user expansion table, a user table and an account table.
Further, the user expansion table and the user table are core tables, and the two tables are in one-to-one correspondence; the account table is a sub-table, and the user table and the account table are in one-to-many relation.
Further, the user expansion table includes: ID. User ID, data value.
Further, the user table includes: ID. Name, age, address Add, number Num.
Further, the account table includes: ID. Account number, balance, type, user ID.
The multi-data source query method is applied to the multi-data source query platform and comprises the following steps:
step 1, the business module writes business data into corresponding data sources in the business executing process;
step 2, the service module submits a query request to the multi-data source query platform, and informs the multi-data source query platform of a query model required by the query request through the query request;
step 3, the multi-data source query platform determines the format of the data to be converted according to the query model, performs data grid conversion and splicing operation on the original data acquired from the data source according to the format to be converted, and inserts the processed data into the data storage device;
and 4, the service module acquires the queried data which is converted into a data format according to the self requirement from the multi-data source query platform.
An application system containing multiple data sources comprises multiple service modules, multiple data sources, a multiple data source query platform, a query platform management system and a data storage device.
By the embodiment of the application, the following technical effects can be obtained:
(1) The multi-data source query platform can store mass data, provides unified SDK query to realize multi-table combination and multi-condition query across database types and data sources, and improves query efficiency;
(2) The multi-data source query platform is accessed by adopting a business non-invasive mode, and provides a friendly interface for function configuration.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the overall architecture of an application system of a multi-data source querying platform;
FIG. 2 is a schematic diagram of a big data table structure;
FIG. 3 is a schematic diagram of a multi-data source query platform.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Under the architecture of the micro-service architecture, multiple data sources must be trending. Meanwhile, the method is oriented to mass data of the Internet, and under the condition that the response time requirement of a user to a system is high, the data query efficiency is imperative to be improved. Under the condition that multiple types of database applications or single databases and multiple data sources exist, the query efficiency needs to be improved, and the multi-table combination and multi-condition query of the type of the cross-database and the cross-data sources is solved.
FIG. 1 is a schematic diagram of the overall architecture of an application system of a multi-data source query platform. The application system comprises a plurality of service modules, a data source, a multi-data source query platform, a query platform management system and a data storage.
The service modules represent service parties, and a multi-data source query platform is required to be called to query data, and each service module is provided with different query models respectively to realize different service functions. The service module is not necessarily a provider of data, that is to say the data provider is not necessarily in contact with the data consumer.
A data source generally refers to a database, but is not limited to the type of database (e.g., a relational database or a non-relational database). The multi-data-source query platform obtains raw data from data sources through a subscription schema. But not only the original data is obtained by a database, but also by various modes such as service call-up, asynchronous request sending by the MQ and the like.
The multi-data source query platform is used as a unified data entry in the application system, and after the original data is obtained, the original data is assembled into a complete large data table structure through multiple data conversion and splicing operations and is stored in a corresponding database in a data source. The manner of data conversion corresponds to the output data structure defined by the service module.
In the data storage, a document type structure is generally used for storing data, and the document type structure is used for storing the data after data conversion and splicing operation by a multi-data source query platform. There are many mature products on the market such as ES clusters, hbase clusters, etc.
And the query platform management system is respectively in data interaction with the data storage and multi-data source query platform and is used for monitoring the execution condition of data conversion and splicing operation in the multi-data source query platform. Meanwhile, the query platform management system also defines a large data table structure, the large data table comprises a core table and a sub-table, and the relation between the core table and the sub-table is defined by the query platform management system. In addition, the query platform management system also provides a visual man-machine interaction operation interface for carrying out function configuration operation.
In the above application system, the multi-data source query method includes the following steps:
step 1, a service module writes service data into a data source (such as MySQL, mongodb and the like) in the execution process of service;
step 2, the business module submits a query request to the multi-data source query platform, and informs the multi-data source query platform of a query model required by the query request through the query request;
step 3, the multi-data source query platform determines the format of the data to be converted according to the query model, performs data grid conversion and splicing operation on the original data acquired from the data source according to the format of the data to be converted, and inserts the processed data into data storage (such as an ES cluster, an Hbase cluster and the like);
and 4, the service module acquires the queried data which is converted into a data format according to the self requirement from the multi-data source query platform.
In the whole process, the query platform management system monitors the execution condition of data conversion and splicing operation in the multi-data source query platform. The data format conversion is carried out through the multi-data source query platform, so that the query requirements of different service modules are met, and the data can be obtained only by one query of the different service modules when the query is carried out, thereby greatly improving the query efficiency.
Fig. 2 is a schematic diagram of a big data table structure. The big data table structure is composed of three tables, namely a user extension table (user_ext), a user table (users) and an account table (Acct). The user table (users) and the user extension table (user_ext) are core tables, and are in one-to-one correspondence; the account table (Acct) is a sub-table, and the user table (users) and the account table (Acct) are in one-to-many relation. The specific definition of each table is as follows:
(1) The user extension table (user_ext) includes: ID. User ID (user_id), data value (evaluation), wherein the data value evaluation is of a common type;
(2) The user table (users) includes: ID. Name, age, address Add, number Num;
(3) The account table (Acct) includes: ID. Account number (acct_num), balance (balance), type (Type), user ID (user_id).
The three tables of the user extension table (user_ext), the user table (user) and the account table (Acct) are spliced into a large table, namely a large data table structure, and each user table user corresponds to one piece of data (database. Users) in the large data table structure and is specifically defined as follows:
ID
core table number: users.id, users.name, users.age, users.num
Sub-table data (one-to-one: common type): users_ext.evaluation
Sub-table data (one-to-many: array type): acct.id, acct.acct_num, acct.type
FIG. 3 is a schematic diagram of a multi-data source query platform. The multi-data source query platform comprises an increment synchronization module, a full synchronization module, a data query module, a message queue MQ structure management module, a data conversion module, a data writing module and a data storage module;
the incremental synchronization module is respectively connected with the data source and the MQ and is used for monitoring the data change of the data source, and after the data change occurs, the changed data is obtained and forwarded to the MQ for processing;
the full synchronization module is respectively connected with the data source and the MQ, and is used for inquiring data in batches from the data source and forwarding the inquired data to the MQ for processing;
the data query module is respectively connected with the service module, the structure management module and the data storage and is used for realizing the JDBC protocol, analyzing the language which can be identified by the corresponding data storage according to the SQL statement which is transmitted from the service module, and feeding back the query result to the service module;
the structure management module is respectively connected with the data conversion module, the data query module and the data storage and is used for querying and managing the storage structure of the data in the data storage, the structure management module respectively acquires the storage structure of the data from the data conversion module and the data query module, and the one-to-one or one-to-many table relations between the main table and the sub-table are defined by distinguishing the main table from the sub-table, so that the main table is ensured to always be one party of the table relations;
the data conversion module is respectively connected with the MQ, the structure management module and the data writing module, and is used for correspondingly converting the data in the main table and the sub-table according to the storage structure in the data storage determined by the structure management module and forwarding the converted data to the data writing module;
the data writing module is respectively connected with the data conversion module and the data storage and completes writing of data through adding, modifying or deleting according to the data received from the data conversion module;
and the data storage is used for storing data by using a document type structure and is used for storing the data converted by the multi-data source query platform.
In the structure management, the bottom layer of the data storage is a large table composed of N tables, but the data source is modified by the dimension of one table, so that a data conversion module is needed for data conversion. The data writing module decides which operation (new addition, modification and deletion) is used to process the data submitted by the data conversion module, and the modification can be converted into deletion, the new addition can be converted into modification, the modification can be converted into new addition, and the judgment of the operation modes is decided by the data writing module.
In summary, the multi-data source query platform solves the technical problems existing at present through some schemes of big data storage. And synchronizing the data into the ES in real time by monitoring Binlog or Oplong of the database, and simultaneously providing a query scheme of the ES data. In the process of data synchronization, the relationships between the data and other tables are determined by establishing a core table (a main table with uniqueness) by taking the core table as a starting point when the corresponding relationship is established by considering the relationships of the data assembly structures (such as one-to-many, many-to-one, one-to-one and many-to-many), so that different processing modes are adopted for each relationship.
Various implementations of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
Finally, it is noted that the above-mentioned preferred embodiments are only intended to illustrate rather than limit the application, and that, although the application has been described in detail by means of the above-mentioned preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the application as defined by the appended claims.
Claims (6)
1. The multi-data source query platform is characterized by comprising an increment synchronization module, a full synchronization module, a data query module, a structure management module, a data conversion module and a data writing module; wherein,,
the incremental synchronization module is respectively connected with a plurality of data sources and the message queue MQ and is used for monitoring the data changes of the plurality of data sources, and after the data changes, the changed data are obtained and forwarded to the message queue MQ for processing;
the full synchronization module is respectively connected with a plurality of data sources and the message queue MQ, and is used for inquiring data in batches from the plurality of data sources and forwarding the inquired data to the message queue MQ for processing;
the data query module is respectively connected with the service module, the structure management module and the data storage device, and is used for realizing the JDBC protocol, analyzing according to SQL sentences transmitted from the service module, performing data query corresponding to languages which can be identified by the data storage device, and feeding back query results to the service module;
the structure management module is respectively connected with the data conversion module, the data query module and the data storage and is used for querying and managing the storage structure of the data in the data storage device; the data conversion module is respectively connected with the message queue MQ, the structure management module and the data writing module and is used for correspondingly converting the data in the main table and the sub-table according to the structure of the data storage in the data storage device determined by the structure management module and forwarding the converted data to the data writing module;
the data writing module is respectively connected with the data conversion module and the data storage device, and writing of data is completed through adding, modifying or deleting according to the data received from the data conversion module;
the data storage structure in the data storage device is a large data table structure, and each user corresponds to one piece of data in the large data table structure;
the big data table structure consists of a user expansion table, a user table and an account table;
the user expansion table and the user table are core tables, and are in one-to-one correspondence; the account table is a sub-table, and the user table and the account table are in one-to-many relation;
the structure management module acquires the storage structure of the data from the data conversion module and the data query module respectively, and determines that the table relationship between the main table and the sub-table is one-to-one or one-to-many by distinguishing the main table from the sub-table;
the multi-data source query platform is used as a unified data entry in the application system, after the original data is obtained, the original data is assembled into a complete large data table structure through multiple data conversion and splicing operations, and the complete large data table structure is stored in a corresponding database in a data source; the data conversion mode corresponds to the output data structure defined by the service module;
the multi-data source query platform synchronizes data into the ES in real time by monitoring Binlog or Oplong of the database, and provides a query scheme of the ES data; in the process of data synchronization, the relation of the data assembly structure is considered, when the corresponding relation is established, the relation with other tables is determined by establishing a unique core table and taking the core table as a starting point, so that different processing modes are adopted aiming at each relation.
2. The platform of claim 1, wherein the user extension table comprises: ID. User ID, data value.
3. The platform of claim 1, wherein the user table comprises: ID. Name, age, address Add, number Num.
4. The platform of claim 1, wherein the account table comprises: ID. Account number, balance, type, user ID.
5. A multi-data source query method applied to the multi-data source query platform as claimed in any one of claims 1 to 4, comprising the steps of:
step 1, the business module writes business data into corresponding data sources in the business executing process;
step 2, the service module submits a query request to the multi-data source query platform, and informs the multi-data source query platform of a query model required by the query request through the query request;
step 3, the multi-data source query platform determines the format of the data to be converted according to the query model, performs data grid conversion and splicing operation on the original data acquired from the data source according to the format to be converted, and inserts the processed data into the data storage device;
step 4, the business module obtains the queried data which is converted into a data format according to the self requirement from the multi-data source query platform;
the query platform management system monitors the execution condition of data conversion and splicing operation in the multi-data source query platform; the data format conversion is carried out through the multi-data source query platform, so that the query requirements of different service modules are met.
6. An application system comprising multiple data sources, wherein the application system comprises a plurality of business modules, a plurality of data sources, a multiple data source query platform as claimed in any one of claims 1 to 4, a query platform management system, and a data storage device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310410511.XA CN116150211B (en) | 2023-04-18 | 2023-04-18 | Multi-data source query method, platform and application system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310410511.XA CN116150211B (en) | 2023-04-18 | 2023-04-18 | Multi-data source query method, platform and application system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116150211A CN116150211A (en) | 2023-05-23 |
| CN116150211B true CN116150211B (en) | 2023-08-18 |
Family
ID=86354565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310410511.XA Active CN116150211B (en) | 2023-04-18 | 2023-04-18 | Multi-data source query method, platform and application system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116150211B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760952A (en) * | 2021-08-09 | 2021-12-07 | 深圳前海爱客风信息技术有限公司 | Data query method and device, storage medium and electronic device |
| CN114003614A (en) * | 2021-11-02 | 2022-02-01 | 广州市创乐信息技术有限公司 | Data synchronization device, method, online transaction system, computer equipment and storage medium |
| WO2022108461A1 (en) * | 2020-11-23 | 2022-05-27 | Goldenline Spolka Z Ograniczona Odpowiedzialnoscia | A system and method for managing and processing data in a dispersed environment with incompatible data storage sources |
| CN114547076A (en) * | 2022-02-21 | 2022-05-27 | 京东方科技集团股份有限公司 | Data processing method and data processing system |
| CN114579584A (en) * | 2022-05-06 | 2022-06-03 | 腾讯科技（深圳）有限公司 | Data table processing method and device, computer equipment and storage medium |
| CN114942953A (en) * | 2022-03-30 | 2022-08-26 | 中国人寿保险股份有限公司 | Cross-system data updating and querying method and related equipment |
| CN115905313A (en) * | 2022-09-23 | 2023-04-04 | 上海客佳信息科技有限公司 | MySQL big table association query system and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11921715B2 (en) * | 2014-01-27 | 2024-03-05 | Microstrategy Incorporated | Search integration |
| US20200117737A1 (en) * | 2018-10-16 | 2020-04-16 | LeapAnalysis Inc. | Fast heterogeneous multi-data source search and analytics |
-
2023
- 2023-04-18 CN CN202310410511.XA patent/CN116150211B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022108461A1 (en) * | 2020-11-23 | 2022-05-27 | Goldenline Spolka Z Ograniczona Odpowiedzialnoscia | A system and method for managing and processing data in a dispersed environment with incompatible data storage sources |
| CN113760952A (en) * | 2021-08-09 | 2021-12-07 | 深圳前海爱客风信息技术有限公司 | Data query method and device, storage medium and electronic device |
| CN114003614A (en) * | 2021-11-02 | 2022-02-01 | 广州市创乐信息技术有限公司 | Data synchronization device, method, online transaction system, computer equipment and storage medium |
| CN114547076A (en) * | 2022-02-21 | 2022-05-27 | 京东方科技集团股份有限公司 | Data processing method and data processing system |
| CN114942953A (en) * | 2022-03-30 | 2022-08-26 | 中国人寿保险股份有限公司 | Cross-system data updating and querying method and related equipment |
| CN114579584A (en) * | 2022-05-06 | 2022-06-03 | 腾讯科技（深圳）有限公司 | Data table processing method and device, computer equipment and storage medium |
| CN115905313A (en) * | 2022-09-23 | 2023-04-04 | 上海客佳信息科技有限公司 | MySQL big table association query system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116150211A (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| CN110032604B (en) | Data storage device, translation device and database access method | |
| CN109471863B (en) | Information query method and device based on distributed database and electronic equipment | |
| CN109491989B (en) | Data processing method and device, electronic equipment and storage medium | |
| US9298775B2 (en) | Changing the compression level of query plans | |
| CN111966677B (en) | Data report processing method and device, electronic equipment and storage medium | |
| CN108363741B (en) | Big data unified interface method, device, equipment and storage medium | |
| CN105405070A (en) | Distributed memory power grid system construction method | |
| CN112084208A (en) | Data visualization method and system, storage medium and electronic device | |
| CN113641700A (en) | Data processing method and device based on Spring boot frame | |
| CN113190517B (en) | Data integration method and device, electronic equipment and computer readable medium | |
| CN113220710B (en) | Data query method, device, electronic equipment and storage medium | |
| CN115617849A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114265966A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN108198595B (en) | Multi-source heterogeneous unstructured medical record data fusion method | |
| WO2024012195A1 (en) | Unified verification method and apparatus, and device and storage medium | |
| CN116150211B (en) | Multi-data source query method, platform and application system | |
| WO2024108638A1 (en) | Adaptive query method based on sharding indexes, and apparatus | |
| WO2024001029A1 (en) | Method and apparatus for maintaining blockchain data, electronic device, and storage medium | |
| CN112817930A (en) | Data migration method and device | |
| CN111047427A (en) | Data reporting method, device, server and storage medium | |
| WO2023164294A1 (en) | Query splitter for an inverted index datastore | |
| CN112148705A (en) | Data migration method and device | |
| US20210141791A1 (en) | Method and system for generating a hybrid data model | |
| CN115168507A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |