CN107402989A - The method and distributed NewSQL Database Systems that a kind of full-text search is established - Google Patents

The method and distributed NewSQL Database Systems that a kind of full-text search is established Download PDF

Info

Publication number
CN107402989A
CN107402989A CN201710580720.3A CN201710580720A CN107402989A CN 107402989 A CN107402989 A CN 107402989A CN 201710580720 A CN201710580720 A CN 201710580720A CN 107402989 A CN107402989 A CN 107402989A
Authority
CN
China
Prior art keywords
units
user
full
data
text search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710580720.3A
Other languages
Chinese (zh)
Other versions
CN107402989B (en
Inventor
晋彤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunrun Da Data Service Co ltd
Original Assignee
Guangzhou Special Road Mdt Infotech Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Special Road Mdt Infotech Ltd filed Critical Guangzhou Special Road Mdt Infotech Ltd
Publication of CN107402989A publication Critical patent/CN107402989A/en
Application granted granted Critical
Publication of CN107402989B publication Critical patent/CN107402989B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9017—Indexing; Data structures therefor; Storage structures using directory or table look-up
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2379—Updates performed during online database operations; commit processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2219—Large Object storage; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2272—Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2433—Query languages
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24542—Plan optimisation
    • G06F16/24545—Selectivity estimation or determination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31—Indexing; Data structures therefor; Storage structures
    • G06F16/316—Indexing structures
    • G06F16/319—Inverted lists
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5022—Workload threshold

Abstract

The invention discloses a kind of method that full-text search is established, and suitable for distributed NewSQL Database Systems, distributed NewSQL Database Systems include solr units, and the method that full-text search is established includes：Asked with JDCB/ODBC interface mode accessing user, wherein, user asks the data field for including the full-text search that needs write；Parse user's request, executive plan corresponding to compiling and generation；According to executive plan, the data field of full-text search is write into tables of data；When the data field of full-text search writes tables of data, the index data of inverted index form is generated according to the description of the data field of full-text search, and index data is written to the concordance list of solr units；The result that the data field for the full-text search for needing to write is had been written into returns to user.The present invention also provides a kind of distributed NewSQL Database Systems.The present invention meets the needs of user creates full-text index by supporting distributed full-text search.

Description

The method and distributed NewSQL Database Systems that a kind of full-text search is established
Technical field
The present invention relates to big data technical field, more particularly to a kind of method of full-text search foundation and distributed NewSQL Database Systems.
Background technology
Hbase is one of foremost distributed NoSQL databases in Hadoop ecosystems at present.Its design concept is come Come from Google Bigtable.Hbase primary clusterings include HMaster and HRegionsever, provide the user form class The data model of type, multiple region being divided into by major key scope, HMaster is responsible for and distributed region, HRegionserver is responsible for the read-write of region data.The advantages of Hbase, becomes current most widely used distribution One of NoSQL databases, moved to increasing using trial on Hbase, but Hbase only supports looking into based on major key Ask, do not support full-text search, inconvenience is brought to many applications.
The content of the invention
The purpose of the embodiment of the present invention is to provide method and the distributed NewSQL data base sets that a kind of full-text search is established System, by supporting distributed full-text search, meets the needs of user creates full-text index.
To achieve the above object, the embodiments of the invention provide a kind of method that full-text search is established, suitable for distribution NewSQL Database Systems, the distributed NewSQL Database Systems include Solr units, the side that the full-text search is established Method includes：
Asked with JDCB/ODBC interface mode accessing user, wherein, user's request includes the full text that needs write The data field of retrieval；
Parse user's request, executive plan corresponding to compiling and generation；
According to the executive plan, the data field of the full-text search is write into the tables of data；
When the data field of the full-text search writes the tables of data, according to the data field of the full-text search Description generates the index data of inverted index form, and the index data is written to the concordance list of the solr units；
Result is returned into user, the result has been write for the data field of the full-text search for needing to write The result entered.
Further, in addition to：The user is asked to the SQL request of conversion SQL statement form.
Further, parsing user's request, executive plan corresponding to compiling and generation include：
Judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then exporting The corresponding executive plan corresponding to SQL statement that prestores, if it is not, then,
Syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
Semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
View and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
Optimizer, optimizer selection result corresponding to acquisition are selected according to the transformation result；
According to data connection approach and the order of connection corresponding to optimizer selection result selection；
According to connected mode and the path of order of connection selection search；
Executive plan is generated according to searching route, and exports executive plan.
Accordingly, the embodiment of the present invention also provides a kind of distributed NewSQL Database Systems, including：
JDCB/ODBC interface units, for interacting operation with user, including user's request is received, return to processing knot Fruit is to user；Wherein, user's request includes the data field for the full-text search that needs write, and the result is needs The result that the data field of the full-text search of write-in has been written into；
Master units, the user's request accessed for accessing JDCB/ODBC interface units, and coordinate multiple processors Between data communication and management overall flow, and by the user request be preferentially sent to SQLPlaner units；master Unit is additionally operable to the result and returns to JDCB/ODBC interface units；
SQLPlaner units, for parsing user's request, compiling and customization is asked to perform meter according to the user Draw；
Worker units, for being performed in parallel the plan, including：According to executive plan, by the full-text search Data field writes tables of data as general character string type；It is additionally operable to the result of the Hbase units being back to institute State master units；
Hbase units, for storing the tables of data；The Hbase units also include the coprocessor modules, The coprocessor modules are used for when the data field of the full-text search writes the tables of data, according to the full text The index data of the description generation inverted index form of the data field of retrieval, and the index data is written to concordance list；
Solr units, for storing the concordance list；
Distributed transaction management device, for when the worker units executive plan is related to affairs, coordinating multi-party complete Distributed transaction management.
Further, JDCB/ODBC interface units are additionally operable to ask the SQL of conversion SQL statement form please the user Ask.
Further, the SQLPlaner units are used for：
Judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then exporting The corresponding executive plan corresponding to SQL statement that prestores, if it is not, then,
Syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
Semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
View and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
Optimizer, optimizer selection result corresponding to acquisition are selected according to the transformation result；
According to data connection approach and the order of connection corresponding to optimizer selection result selection；
According to connected mode and the path of order of connection selection search；
Executive plan is generated according to searching route, and exports executive plan.
Further, in addition to：
Monitor, for being responsible for metadata management, the Region of Hbase units load is monitored, and pass through institute The coprocessor modules for stating Hbase units redistribute Region；The monitor is connected with the master units.
Further, the Region of monitoring Hbase units load, and pass through the Hbase units Coprocessor modules, which redistribute Region, to be included：
The Data distribution information of the Hbase units is received, receives the worker units in the master units Load information, wherein, the load information includes the load deviation value of the worker units；
By the load deviation value of the worker units compared with default load deviation threshold, if it is determined that the load Deviation exceedes threshold values, triggers the Hbase units by the Region on the higher server of hit rate and the relatively low service of hit rate Region on device is carried out from new distribution；
Every Region data volume is obtained, each Region data volume and preset data amount threshold value are sentenced It is disconnected, if it is determined that the data volume of the Region exceedes threshold values, the Hbase units are triggered by more than the institute of preset data amount threshold value State Region and be cut into two.
Further, the JDCB/ODBC interface units include：
JDBC application program modules, for receiving user's request, and JDBC object method are called to provide SQL statement, And return to user for extracting result；
JDBC driver manager modules, for being loaded for the JDBC application program modules and calling JDBC to drive journey Sequence module；
JDBC driver modules, for performing the calling of the JDBC object method, send corresponding to user's request Database of the SQL statement to bottom, and the result obtained from the database of the bottom is returned into JDBC application program modules.
Compared with prior art, a kind of full-text search method for building up disclosed by the invention and distributed NewSQL data base sets System, by being asked with JDCB/ODBC interface mode accessing user, parse the user and ask, corresponding to compiling and generation Executive plan；According to the executive plan, the data field of the full-text search is write into the tables of data；Examined in the full text When the data field of rope writes the tables of data, inverted index form is generated according to the description of the data field of the full-text search Index data, and the index data is written to concordance list；Result is returned into user, the result is described The technical scheme for the result that the data field for the full-text search for needing to write has been written into, solves Hbase in the prior art The inquiry based on major key is only supported, the problem of full-text search is not supported, full-text search is established by Solr, to support distribution Full-text search, meet the needs of user creates full-text index, improve Consumer's Experience.
Brief description of the drawings
Fig. 1 is a kind of schematic flow sheet for full-text search method for building up that the embodiment of the present invention 1 provides；
Fig. 2 is a kind of structural representation for distributed NewSQL Database Systems that the embodiment of the present invention 2 provides.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made Embodiment, belong to the scope of protection of the invention.
Referring to Fig. 1, Fig. 1 is a kind of schematic flow sheet for full-text search method for building up that the embodiment of the present invention 1 provides；This reality Apply example 1 and be applied to distributed NewSQL Database Systems, the distributed NewSQL Database Systems include solr units, this reality Applying example includes step：
S1, the interface mode accessing user request with JDCB/ODBC, wherein, user's request includes what needs write The data field of full-text search；
S2, parsing user's request, executive plan corresponding to compiling and generation；
S3, according to the executive plan, the data field of the full-text search is write into the tables of data；
S4, the full-text search data field write the tables of data when, according to the data word of the full-text search The index data of the description generation inverted index form of section, and the index data is written in the concordance list of Solr units；
S5, result is returned to user, the data field for the full-text search that the result writes for the needs The result having been written into.
The present embodiment supports distributed full-text search, and user can be that the table of oneself creates full-text index, and make in SQL In full retrieval grammer scans for.Which is the special extension of secondary index, for needing the field of full-text search no longer By index datastore into other concordance list.
Further, step S1 also includes：The user is asked to the SQL request of conversion SQL statement form.
Further, the user's request of parsing described in step S2, executive plan corresponding to compiling and generation include：
S21, judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then The corresponding executive plan corresponding to SQL statement that prestores is exported, if it is not, then,
S22, syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
S23, semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
S24, view and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
S25, according to the transformation result select optimizer, optimizer selection result corresponding to acquisition；
S26, data connection approach and the order of connection according to corresponding to optimizer selection result selection；
S27, the path for selecting to search for according to connected mode and the order of connection；
S28, executive plan generated according to searching route, and export executive plan.
When it is implemented, first, asked with JDCB/ODBC interface mode accessing user, wherein, user's request bag Include the data field for the full-text search for needing to write；Then, user's request is parsed, meter is performed corresponding to compiling and generation Draw；Then, according to the executive plan, the data field of the full-text search is write into the tables of data；And in the full text When the data field of retrieval writes the tables of data, inverted index shape is generated according to the description of the data field of the full-text search The index data of formula, and the index data is written to concordance list；Finally, result is returned into user, the processing knot The result that fruit has been written into for the data field of the full-text search for needing to write.
The present embodiment solves in the prior art that Hbase only supports the inquiry based on major key, does not support full-text search Problem, distributed full-text search is supported, meet the needs of user creates full-text index, improve Consumer's Experience.
Referring to Fig. 2, Fig. 2 is that the embodiment of the present invention 2 also provides a kind of distributed NewSQL Database Systems, the present embodiment bag Include：
JDCB/ODBC interface units 1, for interacting operation with user, including user's request is received, return to processing knot Fruit is to user；Wherein, user's request includes the data field for the full-text search that needs write, and the result is needs The result that the data field of the full-text search of write-in has been written into；
Master units 2, the user's request accessed for accessing JDCB/ODBC interface units 1, and coordinate multiple processing Data communication and management overall flow between device, and user request is preferentially sent to SQLPlaner units 3； Master units 2 are additionally operable to the result and return to JDCB/ODBC interface units 1；
SQLPlaner units 3, for parsing user's request, compiling and customization is asked to perform according to the user Plan；
Worker units 4, for being performed in parallel the plan, including：According to executive plan, by the full-text search Data field writes tables of data as general character string type；It is additionally operable to the result of the Hbase units being back to institute State master units 2；
Hbase units 6, for storing the tables of data；The Hbase units 6 also include the coprocessor modules 61, the coprocessor modules 61 are used for when the data field of the full-text search writes the tables of data, according to described The index data of the description generation inverted index form of the data field of full-text search, and the index data is written to index Table；
Solr units 7, for storing the concordance list；
Generally, the distributed NewSQL Database Systems of the present embodiment allow user flexible according to specific service logic Secondary index is established, user often establishes multiple secondary indexs in actual applications, when in use according to querying condition dynamic The cost using index is calculated, automatically selects most suitable index.Inquiry for rowkey is extremely efficient, therefore secondary index Implementation be using Hbase units 6 Coprocessor modules 61 and Filter modules 62 generation for data index Table.
The present embodiment supports distributed full-text search by Solr units 7, and user can be that the table of oneself creates full text rope Draw, and scanned in SQL using full-text search grammer.Which is the special extension of secondary index, also with Coprocessor modules 61 are realized, for needing the field of full-text search no longer by index datastore to other concordance list In, but by index datastore into Solr units 7, the function of full-text search is provided by Solr units 7.
Distributed transaction management device 5, it is multi-party complete for when the executive plan of worker units 4 is related to affairs, coordinating Into distributed transaction management.
Further, JDCB/ODBC interface units 1 are used to ask the SQL of conversion SQL statement form please the user Ask.
Further, the SQLPlaner units 3 are used for：
Judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then exporting The corresponding executive plan corresponding to SQL statement that prestores, if it is not, then,
Syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
Semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
View and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
Optimizer, optimizer selection result corresponding to acquisition are selected according to the transformation result；
According to data connection approach and the order of connection corresponding to optimizer selection result selection；
According to connected mode and the path of order of connection selection search；
Executive plan is generated according to searching route, and exports executive plan.
Further, the present embodiment also includes：
Monitor 8, for being responsible for metadata management, the Region of the Hbase units 6 load is monitored, and passed through The coprocessor modules 61 of the Hbase units 6 redistribute Region；The monitor 8 and the master units 2 Connection.
Further, the Region of the Hbase units 6 load is monitored, and passes through the Hbase units 6 Coprocessor modules, which redistribute Region, to be included：
The Data distribution information of the Hbase units 6 is received, the worker received in the master units 2 is mono- The load information of member 4, wherein, the load information includes the load deviation value of the worker units 4；
By the load deviation value of the worker units 4 compared with default load deviation threshold, if it is determined that described negative Carry deviation and exceed threshold values, trigger the Hbase units 6 by the Region on the higher server of hit rate and the relatively low clothes of hit rate The Region being engaged on device is carried out from new distribution；
Every Region data volume is obtained, each Region data volume and preset data amount threshold value are sentenced It is disconnected, if it is determined that the data volume of the Region exceedes threshold values, the Hbase units are triggered by more than the institute of preset data amount threshold value State Region and be cut into two.
Further, the JDCB/ODBC interface units 1 include：
JDBC application program modules 11, for receiving user's request, and JDBC object method is called to provide SQL languages Sentence, and return to user for extracting result；
JDBC driver managers module 12, for loading and calling JDBC to drive for the JDBC application program modules 11 Dynamic program module 13；
JDBC driver modules 13, for performing the calling of the JDBC object method, send corresponding to user's request Database of the SQL statement to bottom, and the result obtained from the database of the bottom is returned into JDBC application program moulds Block 11.
When it is implemented, first, user is received by JDCB/ODBC interface units 1 and asked；Then, master units 2 connect Enter user's request that JDCB/ODBC interface units 1 are accessed, and user request is preferentially sent to SQLPlaner units 3；Then, parse the user by SQLPlaner units 3 to ask, ask compiling and customization to perform meter according to the user Draw；Then, the plan is performed in parallel by worker units 4, including：According to executive plan, by the full-text search Data field writes tables of data as general character string type；The coprocessor modules 61 are used in the full-text search Data field when writing the tables of data, inverted index form is generated according to the description of the data field of the full-text search Index data, and the concordance list that the index data is written to Solr units 7 is additionally operable to the processing of the Hbase units 6 As a result the master units 2 are back to；Finally, the result of Hbase units 6 is back to the master units 2, and leads to Cross master units 2 and the result that image data has been written into is returned into JDCB/ODBC interface units 1 to return to user.
The present embodiment solves in the prior art that Hbase only supports the inquiry based on major key, does not support full-text search Problem, full-text search is established by Solr units, to support distributed full-text search, meet that user creates the need of full-text index Ask, improve Consumer's Experience.
Described above is the preferred embodiment of the present invention, it is noted that for those skilled in the art For, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improvements and modifications are also considered as Protection scope of the present invention.

Claims (9)

1. a kind of method that full-text search is established, it is characterised in that suitable for distributed NewSQL Database Systems, the distribution Formula NewSQL Database Systems include Solr units, and the method that the full-text search is established includes：
Asked with JDCB/ODBC interface mode accessing user, wherein, user's request includes the full-text search that needs write Data field；
Parse user's request, executive plan corresponding to compiling and generation；
According to the executive plan, the data field of the full-text search is write into the tables of data；
When the data field of the full-text search writes the tables of data, according to the description of the data field of the full-text search The index data of inverted index form is generated, and the index data is written to the concordance list of the solr units；
Result is returned into user, the result has been written into for the data field of the full-text search for needing to write Result.
2. the method that a kind of full-text search as claimed in claim 1 is established, it is characterised in that also include：Please by the user Seek the SQL request of conversion SQL statement form.
3. the method that a kind of full-text search as claimed in claim 2 is established, it is characterised in that the parsing user please Ask, compile and generate corresponding to executive plan include：
Judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then output is relative The executive plan corresponding to SQL statement that prestores answered, if it is not, then,
Syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
Semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
View and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
Optimizer, optimizer selection result corresponding to acquisition are selected according to the transformation result；
According to data connection approach and the order of connection corresponding to optimizer selection result selection；
According to connected mode and the path of order of connection selection search；
Executive plan is generated according to searching route, and exports executive plan.
A kind of 4. distributed NewSQL Database Systems, it is characterised in that including：
JDCB/ODBC interface units, for interacting operation with user, including user's request is received, return to result and give User；Wherein, user's request includes the data field for the full-text search that needs write, and the result is to need to write Full-text search the result that has been written into of data field；
Master units, for accessing user's request that JDCB/ODBC interface units are accessed, and coordinate between multiple processors Data communication and management overall flow, and by the user request be preferentially sent to SQLPlaner units；Master units It is additionally operable to the result and returns to JDCB/ODBC interface units；
SQLPlaner units, for parsing user's request, compiling and customization executive plan are asked according to the user；
Worker units, for being performed in parallel the plan, including：According to executive plan, by the data of the full-text search Field writes tables of data as general character string type；It is additionally operable to the result of the Hbase units being back to described Master units；
Hbase units, for storing the tables of data；The Hbase units also include the coprocessor modules, described Coprocessor modules are used for when the data field of the full-text search writes the tables of data, according to the full-text search Data field description generation inverted index form index data, and the index data is written to concordance list；
Solr units, for storing the concordance list；
Distributed transaction management device, for when the worker units executive plan is related to affairs, coordinating multi-party completion distribution Formula transaction management.
A kind of 5. distributed NewSQL Database Systems as claimed in claim 4, it is characterised in that JDCB/ODBC interface lists Member is additionally operable to ask the user SQL request of conversion SQL statement form.
6. a kind of distributed NewSQL Database Systems as claimed in claim 5, it is characterised in that the SQLPlaner is mono- Member is used for：
Judge to whether there is the prestore SQL statement corresponding with the SQL request in common buffer pool, if so, then output is relative The executive plan corresponding to SQL statement that prestores answered, if it is not, then,
Syntax check is carried out to the SQL request, if syntax error returns to error message to user, otherwise,
Semantic test is carried out to the SQL request, if semantic error returns to error message to user, otherwise,
View and expression formula conversion, conversion results corresponding to acquisition are carried out to the SQL request；
Optimizer, optimizer selection result corresponding to acquisition are selected according to the transformation result；
According to data connection approach and the order of connection corresponding to optimizer selection result selection；
According to connected mode and the path of order of connection selection search；
Executive plan is generated according to searching route, and exports executive plan.
7. a kind of distributed NewSQL Database Systems as claimed in claim 1, it is characterised in that also include：
Monitor, for being responsible for metadata management, the Region of Hbase units load is monitored, and by described The coprocessor modules of Hbase units redistribute Region；The monitor is connected with the master units.
8. a kind of distributed NewSQL Database Systems as claimed in claim 7, it is characterised in that described in the monitoring The Region of Hbase units load, and Region is redistributed by the coprocessor modules of the Hbase units Including：
The Data distribution information of the Hbase units is received, receives bearing for the worker units in the master units Information carrying ceases, wherein, the load information includes the load deviation value of the worker units；
By the load deviation value of the worker units compared with default load deviation threshold, if it is determined that the load deviation Value exceedes threshold values, triggers the Hbase units by the Region on the higher server of hit rate and the relatively low server of hit rate Region carry out from new distribution；
Every Region data volume is obtained, each Region data volume and preset data amount threshold value are judged, If it is determined that the data volume of the Region exceedes threshold values, the Hbase units are triggered by more than described in preset data amount threshold value Region is cut into two.
9. a kind of distributed NewSQL Database Systems as claimed in claim 4, it is characterised in that the JDCB/ODBC connects Mouth unit includes：
JDBC application program modules, for receiving user's request, and JDBC object method is called to be used in combination to provide SQL statement User is returned in extraction result；
JDBC driver manager modules, for JDBC driver moulds to be loaded and called for the JDBC application program modules Block；
JDBC driver modules, for performing the calling of the JDBC object method, send the corresponding SQL languages of user's request Sentence gives the database of bottom, and the result obtained from the database of the bottom is returned into JDBC application program modules.
CN201710580720.3A 2016-09-21 2017-07-17 Full-text retrieval establishing method and distributed NewSQL database system Expired - Fee Related CN107402989B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610842399.7A CN106446153A (en) 2016-09-21 2016-09-21 Distributed newSQL database system and method
CN2016108423997 2016-09-21

Publications (2)

Publication Number Publication Date
CN107402989A true CN107402989A (en) 2017-11-28
CN107402989B CN107402989B (en) 2020-10-27

Family

ID=58166840

Family Applications (24)

Application Number Title Priority Date Filing Date
CN201610842399.7A Pending CN106446153A (en) 2016-09-21 2016-09-21 Distributed newSQL database system and method
CN201710580720.3A Expired - Fee Related CN107402989B (en) 2016-09-21 2017-07-17 Full-text retrieval establishing method and distributed NewSQL database system
CN201710581256.XA Expired - Fee Related CN107391653B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and picture data storage method
CN201710580794.7A Expired - Fee Related CN107451214B (en) 2016-09-21 2017-07-17 Non-primary key query method and distributed NewSQL database system
CN201710580403.1A Expired - Fee Related CN107368575B (en) 2016-09-21 2017-07-17 Load-balanced distributed NewSQL database system
CN201710585103.2A Expired - Fee Related CN107402995B (en) 2016-09-21 2017-07-17 Distributed newSQL database system and method
CN201710581273.3A Expired - Fee Related CN107451221B (en) 2016-09-21 2017-07-17 Database interface unit device and distributed NewSQL database system
CN201710580423.9A Active CN107402987B (en) 2016-09-21 2017-07-17 Full-text retrieval method and distributed NewSQL database system
CN201710581195.7A Expired - Fee Related CN107451220B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system
CN201710581193.8A Expired - Fee Related CN107451219B (en) 2016-09-21 2017-07-17 Method for analyzing second index and distributed New SQL database
CN201710580417.3A Expired - Fee Related CN107463632B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and data query method
CN201710580456.3A Expired - Fee Related CN107402988B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and semi-structured data query method
CN201710581229.2A Expired - Fee Related CN107491345B (en) 2016-09-21 2017-07-17 Method for writing picture data and distributed NewSQ L database system
CN201710580431.3A Active CN107491485B (en) 2016-09-21 2017-07-17 Method for generating execution plan, plan unit device and distributed NewSQ L database system
CN201710580435.1A Expired - Fee Related CN107480198B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and full-text retrieval method
CN201710580739.8A Expired - Fee Related CN107402990B (en) 2016-09-21 2017-07-17 Distributed New SQL database system and semi-structured data storage method
CN201710580752.3A Expired - Fee Related CN107247808B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and picture data query method
CN201710580416.9A Expired - Fee Related CN107291947B (en) 2016-09-21 2017-07-17 Semi-structured data query method and distributed NewSQL database system
CN201710581291.1A Expired - Fee Related CN107463637B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and data storage method
CN201710580796.6A Expired - Fee Related CN107402992B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and full-text retrieval establishing method
CN201710581237.7A Expired - Fee Related CN107463635B (en) 2016-09-21 2017-07-17 Method for inquiring picture data and distributed NewSQL database system
CN201710580791.3A Active CN107291948B (en) 2016-09-21 2017-07-17 Access method of distributed newSQL database
CN201710581275.2A Active CN107329837B (en) 2016-09-21 2017-07-17 Load balancing method and unit and distributed NewSQL database system
CN201710580754.2A Expired - Fee Related CN107402991B (en) 2016-09-21 2017-07-17 Method for writing semi-structured data and distributed NewSQL database system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201610842399.7A Pending CN106446153A (en) 2016-09-21 2016-09-21 Distributed newSQL database system and method

Family Applications After (22)

Application Number Title Priority Date Filing Date
CN201710581256.XA Expired - Fee Related CN107391653B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and picture data storage method
CN201710580794.7A Expired - Fee Related CN107451214B (en) 2016-09-21 2017-07-17 Non-primary key query method and distributed NewSQL database system
CN201710580403.1A Expired - Fee Related CN107368575B (en) 2016-09-21 2017-07-17 Load-balanced distributed NewSQL database system
CN201710585103.2A Expired - Fee Related CN107402995B (en) 2016-09-21 2017-07-17 Distributed newSQL database system and method
CN201710581273.3A Expired - Fee Related CN107451221B (en) 2016-09-21 2017-07-17 Database interface unit device and distributed NewSQL database system
CN201710580423.9A Active CN107402987B (en) 2016-09-21 2017-07-17 Full-text retrieval method and distributed NewSQL database system
CN201710581195.7A Expired - Fee Related CN107451220B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system
CN201710581193.8A Expired - Fee Related CN107451219B (en) 2016-09-21 2017-07-17 Method for analyzing second index and distributed New SQL database
CN201710580417.3A Expired - Fee Related CN107463632B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and data query method
CN201710580456.3A Expired - Fee Related CN107402988B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and semi-structured data query method
CN201710581229.2A Expired - Fee Related CN107491345B (en) 2016-09-21 2017-07-17 Method for writing picture data and distributed NewSQ L database system
CN201710580431.3A Active CN107491485B (en) 2016-09-21 2017-07-17 Method for generating execution plan, plan unit device and distributed NewSQ L database system
CN201710580435.1A Expired - Fee Related CN107480198B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and full-text retrieval method
CN201710580739.8A Expired - Fee Related CN107402990B (en) 2016-09-21 2017-07-17 Distributed New SQL database system and semi-structured data storage method
CN201710580752.3A Expired - Fee Related CN107247808B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and picture data query method
CN201710580416.9A Expired - Fee Related CN107291947B (en) 2016-09-21 2017-07-17 Semi-structured data query method and distributed NewSQL database system
CN201710581291.1A Expired - Fee Related CN107463637B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and data storage method
CN201710580796.6A Expired - Fee Related CN107402992B (en) 2016-09-21 2017-07-17 Distributed NewSQL database system and full-text retrieval establishing method
CN201710581237.7A Expired - Fee Related CN107463635B (en) 2016-09-21 2017-07-17 Method for inquiring picture data and distributed NewSQL database system
CN201710580791.3A Active CN107291948B (en) 2016-09-21 2017-07-17 Access method of distributed newSQL database
CN201710581275.2A Active CN107329837B (en) 2016-09-21 2017-07-17 Load balancing method and unit and distributed NewSQL database system
CN201710580754.2A Expired - Fee Related CN107402991B (en) 2016-09-21 2017-07-17 Method for writing semi-structured data and distributed NewSQL database system

Country Status (1)

Country Link
CN (24) CN106446153A (en)

Families Citing this family (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107391744B (en) * 2017-08-10 2020-06-16 东软集团股份有限公司 Data storage method, data reading method, data storage device, data reading device and equipment
CN107480260B (en) * 2017-08-16 2021-02-23 北京奇虎科技有限公司 Big data real-time analysis method and device, computing equipment and computer storage medium
CN107688660B (en) * 2017-09-08 2020-03-13 上海达梦数据库有限公司 Parallel execution plan execution method and device
CN107766572A (en) * 2017-11-13 2018-03-06 北京国信宏数科技有限责任公司 Distributed extraction and visual analysis method and system based on economic field data
CN108228750A (en) * 2017-12-21 2018-06-29 浪潮软件股份有限公司 A kind of distributed data base and its method that data are managed
CN108038215A (en) * 2017-12-22 2018-05-15 上海达梦数据库有限公司 Data processing method and system
CN109992409B (en) * 2018-01-02 2021-07-30 中国移动通信有限公司研究院 Method, device and system for segmenting data storage area, electronic equipment and medium
CN108829507B (en) * 2018-03-30 2019-07-26 北京百度网讯科技有限公司 The resource isolation method, apparatus and server of distributed data base system
CN108664616A (en) * 2018-05-14 2018-10-16 浪潮软件集团有限公司 ROWID-based Oracle data batch acquisition method
CN108846044A (en) * 2018-05-30 2018-11-20 浪潮软件股份有限公司 A kind of map application dispositions method and device
CN108920519A (en) * 2018-06-04 2018-11-30 贵州数据宝网络科技有限公司 One-to-many data supply system and method
CN109033209B (en) * 2018-06-29 2021-12-31 新华三大数据技术有限公司 Spark storage process processing method and device
CN109241076A (en) * 2018-08-01 2019-01-18 上海依图网络科技有限公司 A kind of data query method and device
CN109271428A (en) * 2018-09-11 2019-01-25 北京市计算中心 Data pick-up method and method for exhibiting data based on geography information
CN109408591B (en) * 2018-10-12 2021-11-09 北京聚云位智信息科技有限公司 Decision-making distributed database system supporting SQL (structured query language) driven AI (Artificial Intelligence) and feature engineering
CN109298976B (en) * 2018-10-17 2022-04-12 成都索贝数码科技股份有限公司 Heterogeneous database cluster backup system and method
CN109408515A (en) * 2018-11-01 2019-03-01 郑州云海信息技术有限公司 A kind of index execution method and apparatus
CN109684412A (en) * 2018-12-25 2019-04-26 成都虚谷伟业科技有限公司 A kind of distributed data base system
CN109726250B (en) * 2018-12-27 2020-01-17 星环信息科技（上海）有限公司 Data storage system, metadata database synchronization method and data cross-domain calculation method
CN111488340B (en) * 2019-01-29 2023-09-12 菜鸟智能物流控股有限公司 Data processing method and device and electronic equipment
CN110046161A (en) * 2019-03-18 2019-07-23 平安普惠企业管理有限公司 Method for writing data and device, storage medium, electronic equipment
CN110086602B (en) * 2019-04-16 2022-02-11 上海交通大学 Rapid implementation method of SM3 password hash algorithm based on GPU
CN110110234B (en) * 2019-05-13 2020-10-16 重庆天蓬网络有限公司 Big data real-time searching system and method
CN110275901B (en) * 2019-06-25 2021-08-24 北京创鑫旅程网络技术有限公司 Cache data calling method and device
CN110457363B (en) * 2019-07-05 2023-11-21 中国平安人寿保险股份有限公司 Query method, device and storage medium based on distributed database
CN110413642B (en) * 2019-08-02 2022-05-27 北京快立方科技有限公司 Application-unaware fragmentation database parsing and optimizing method
CN110569257B (en) * 2019-09-16 2022-04-01 上海达梦数据库有限公司 Data processing method, corresponding device, equipment and storage medium
CN110704437B (en) * 2019-09-26 2022-05-20 上海达梦数据库有限公司 Method, device, equipment and storage medium for modifying database query statement
CN112688976A (en) * 2019-10-17 2021-04-20 广州迈安信息科技有限公司 Data processing transmission service system adopting JDBC/HTTP standard
CN110888919B (en) * 2019-12-04 2023-06-30 阳光电源股份有限公司 HBase-based method and device for statistical analysis of big data
CN113032479A (en) * 2019-12-24 2021-06-25 上海昂创信息技术有限公司 HBase non-primary key indexing method and HBase system
CN111309581B (en) * 2020-02-28 2023-09-12 中国工商银行股份有限公司 Application performance detection method and device in database upgrading scene
CN111651453B (en) * 2020-04-30 2024-02-06 中国平安财产保险股份有限公司 User history behavior query method and device, electronic equipment and storage medium
CN111797112B (en) * 2020-06-05 2022-04-01 武汉大学 PostgreSQL preparation statement execution optimization method
CN111930705B (en) * 2020-07-07 2023-03-14 中国电子科技集团公司电子科学研究院 Binary message protocol data processing method and device
CN112148792B (en) * 2020-09-16 2024-04-12 鹏城实验室 Partition data adjustment method, system and terminal based on HBase
CN112052347A (en) * 2020-10-09 2020-12-08 北京百度网讯科技有限公司 Image storage method and device and electronic equipment
CN112416925B (en) * 2020-11-02 2024-04-09 浙商银行股份有限公司 Query method based on ordered distributed index structure and distributed database system
CN112364033B (en) * 2021-01-13 2021-04-13 北京云真信科技有限公司 Data retrieval system
CN113760900A (en) * 2021-02-19 2021-12-07 西安京迅递供应链科技有限公司 Method and device for real-time data summarization and interval summarization
CN112905615B (en) * 2021-03-02 2023-03-24 浪潮云信息技术股份公司 Distributed consistency protocol submission method and system based on sequence verification
CN112925841B (en) * 2021-03-26 2022-11-08 瀚高基础软件股份有限公司 Distributed JDBC implementation method, device and computer-readable storage medium
CN113407662B (en) * 2021-08-19 2021-12-14 深圳市明源云客电子商务有限公司 Sensitive word recognition method, system and computer readable storage medium
CN113742370B (en) * 2021-11-02 2022-04-19 阿里云计算有限公司 Data query method and statistical information ciphertext generation method of full-encryption database
CN115129724A (en) * 2022-08-29 2022-09-30 畅捷通信息技术股份有限公司 Statistical report paging method, system, equipment and medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104731945A (en) * 2015-03-31 2015-06-24 浪潮集团有限公司 Full-text searching method and device based on HBase

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101477568A (en) * 2009-02-12 2009-07-08 清华大学 Integrated retrieval method for structured data and non-structured data
CN101567006B (en) * 2009-05-25 2012-07-04 中兴通讯股份有限公司 Database system and distributed SQL statement execution plan reuse method
CN102163195B (en) * 2010-02-22 2013-04-24 北京东方通科技股份有限公司 Query optimization method based on unified view of distributed heterogeneous database
CN102375853A (en) * 2010-08-24 2012-03-14 中国移动通信集团公司 Distributed database system, method for building index therein and query method
CN102201010A (en) * 2011-06-23 2011-09-28 清华大学 Distributed database system without sharing structure and realizing method thereof
CN102289482A (en) * 2011-08-02 2011-12-21 北京航空航天大学 Unstructured data query method
CN103150304B (en) * 2011-12-06 2016-11-23 郑红云 Cloud Database Systems
CN103577407B (en) * 2012-07-19 2016-10-12 国际商业机器公司 Querying method and inquiry unit for distributed data base
US20140074860A1 (en) * 2012-09-12 2014-03-13 Pingar Holdings Limited Disambiguator
CN102902932B (en) * 2012-09-18 2015-12-02 武汉华工安鼎信息技术有限责任公司 The using method of the outside encrypting and deciphering system of the database based on SQL rewrite
CN103092970A (en) * 2013-01-24 2013-05-08 华为技术有限公司 Database operation method and device
US9773021B2 (en) * 2013-01-30 2017-09-26 Hewlett-Packard Development Company, L.P. Corrected optical property value-based search query
CN103377292B (en) * 2013-07-02 2017-02-15 华为技术有限公司 Database result set caching method and device
US20150039587A1 (en) * 2013-07-31 2015-02-05 Oracle International Corporation Generic sql enhancement to query any semi-structured data and techniques to efficiently support such enhancements
CN103473321A (en) * 2013-09-12 2013-12-25 华为技术有限公司 Database management method and system
CN104794123B (en) * 2014-01-20 2018-07-27 阿里巴巴集团控股有限公司 A kind of method and device building NoSQL database indexes for semi-structured data
CN103984726B (en) * 2014-05-16 2017-03-29 上海新炬网络信息技术有限公司 A kind of local correction method of data base's implement plan
CN104133858B (en) * 2014-07-15 2017-08-01 武汉邮电科学研究院 Intelligence analysis system with double engines and method based on row storage
CN104503985A (en) * 2014-12-03 2015-04-08 浪潮电子信息产业股份有限公司 Method for automatically creating Solr index file by Hbase data
CN104572895B (en) * 2014-12-24 2018-02-23 天津南大通用数据技术股份有限公司 MPP databases and Hadoop company-datas interoperability methods, instrument and implementation method
CN104731922A (en) * 2015-03-26 2015-06-24 江苏物联网研究发展中心 System and method for rapidly retrieving structural data based on distributed type database HBase
CN104750815B (en) * 2015-03-30 2017-11-03 浪潮集团有限公司 The storage method and device of a kind of Lob data based on HBase
CN105389375B (en) * 2015-11-18 2018-10-02 福建师范大学 A kind of image index setting method, system and search method based on visible range
CN105740410A (en) * 2016-01-29 2016-07-06 浪潮电子信息产业股份有限公司 Data statistics method based on Hbase secondary index

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104731945A (en) * 2015-03-31 2015-06-24 浪潮集团有限公司 Full-text searching method and device based on HBase

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
@APACHEPHOENIX: "ApachePhoenix", 《APACHE.ORG》 *
JAMES TAYLOR: "Phoenix", 《APACHE.ORG》 *
LARS GEORGE: "《HBase权威指南》", 31 October 2013, 人民邮电出版社 *
吴国泉: "基于HBase的全文索引及检索技术的研究", 《万方数据知识服务平台》 *

Also Published As

Publication number Publication date
CN107402991B (en) 2020-05-19
CN107451221A (en) 2017-12-08
CN107463637A (en) 2017-12-12
CN107402989B (en) 2020-10-27
CN107402991A (en) 2017-11-28
CN107463635A (en) 2017-12-12
CN107247808B (en) 2020-01-10
CN107491485B (en) 2020-08-04
CN107402995B (en) 2020-06-09
CN106446153A (en) 2017-02-22
CN107463632A (en) 2017-12-12
CN107402992B (en) 2020-06-09
CN107247808A (en) 2017-10-13
CN107368575A (en) 2017-11-21
CN107463635B (en) 2020-09-25
CN107402987A (en) 2017-11-28
CN107451214A (en) 2017-12-08
CN107491345B (en) 2020-08-04
CN107402990B (en) 2020-06-09
CN107291947A (en) 2017-10-24
CN107329837B (en) 2020-06-09
CN107402987B (en) 2020-04-03
CN107391653B (en) 2020-05-19
CN107463637B (en) 2020-05-19
CN107451220B (en) 2020-06-09
CN107491345A (en) 2017-12-19
CN107451219B (en) 2020-06-09
CN107402992A (en) 2017-11-28
CN107329837A (en) 2017-11-07
CN107480198A (en) 2017-12-15
CN107402988A (en) 2017-11-28
CN107491485A (en) 2017-12-19
CN107402990A (en) 2017-11-28
CN107451219A (en) 2017-12-08
CN107291948A (en) 2017-10-24
CN107451221B (en) 2020-09-04
CN107291947B (en) 2020-03-10
CN107480198B (en) 2020-05-19
CN107402988B (en) 2020-01-03
CN107463632B (en) 2020-06-09
CN107451214B (en) 2020-05-19
CN107451220A (en) 2017-12-08
CN107368575B (en) 2020-06-09
CN107402995A (en) 2017-11-28
CN107291948B (en) 2020-05-19
CN107391653A (en) 2017-11-24

Similar Documents

Publication Publication Date Title
CN107402989A (en) The method and distributed NewSQL Database Systems that a kind of full-text search is established
US9959311B2 (en) Natural language interface to databases
US10042921B2 (en) Robust and readily domain-adaptable natural language interface to databases
CN110019314B (en) Dynamic data packaging method based on data item analysis, client and server
US20150269234A1 (en) User Defined Functions Including Requests for Analytics by External Analytic Engines
CN108280082A (en) A kind of extemporaneous querying method and system of statistical data
CN108334620A (en) A kind of method and device operating Cluster Database based on ORM frames
US20060190476A1 (en) Database storage system and associated method
KR20180077830A (en) Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method
CN114020852A (en) Knowledge graph display method and device
CN114596044A (en) Tool and method for project process approval
CN111143406A (en) Database data comparison method and database data comparison system
CN109117427A (en) A kind of client, server, search method and its system
Szekely et al. An efficient method for the self-consistent electro-thermal simulation and its integration into a CAD framework
CN114064862A (en) Question answering method, device and equipment
CN110019143A (en) Operational system and method based on quick data warehouse

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20201014

Address after: Room 5303, No. 1023, Gaopu Road, Tianhe Software Park, Tianhe District, Guangzhou City, Guangdong Province

Applicant after: Yunrun Da Data Service Co.,Ltd.

Address before: 510000 Yuexiu District, Guangzhou Province, north of the text of the text of the North Road, No. 68, the east wing of the text of the building on the ground floor, No. six, No. 602, No.

Applicant before: GUANGZHOU TEDAO INFORMATION TECHNOLOGY Co.,Ltd.

CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201027

Termination date: 20210717