CN107451219B

CN107451219B - Method for analyzing second index and distributed New SQL database

Info

Publication number: CN107451219B
Application number: CN201710581193.8A
Authority: CN
Inventors: 晋彤
Original assignee: Yunrun Da Data Service Co ltd
Current assignee: Yunrun Da Data Service Co ltd
Priority date: 2016-09-21
Filing date: 2017-07-17
Publication date: 2020-06-09
Anticipated expiration: 2037-07-17
Also published as: CN107451220A; CN107463632A; CN107451219A; CN107463632B; CN107402992A; CN107480198B; CN107402990B; CN107368575A; CN107329837A; CN107402988B; CN107451220B; CN107368575B; CN107491485A; CN107402991B; CN107291948B; CN107247808A; CN107402990A; CN107247808B; CN107463637B; CN107402987A

Abstract

The information distribution a distributed new SQL database system, The while composition, The system composition a master unit, The mapping unit and The database unit, The world unit used for accessing a user request in a database interface module and sending The user request to The mapping unit, The world unit request composed of a database field to a file field, The mapping unit used for mapping The user request in a database interface module and sending The user request in a mapping database, The mapping and mapping database unit used for mapping The user request in a mapping database and sending The mapping database, The mapping and mapping database unit used for mapping The user request in a mapping database and mapping database, The mapping database unit used for mapping The mapping database in a mapping database and mapping database, The mapping and mapping database unit used for mapping The mapping database in a mapping database and mapping database, The mapping database unit used for mapping database and mapping database in a mapping database and mapping database, The mapping database unit used for mapping The mapping database in a mapping database and mapping database in a mapping database, The mapping database and mapping database in a mapping database in field is written into The data table and writing The index data into The index table, The interaction of The interactions a data storage method based on The distributed New SQL database system, The interaction of The interaction can interaction The interaction of The interaction index and The interaction request.

Description

Method for analyzing second index and distributed New SQL database

Technical Field

The inventory relationships to The technical field of big data, in a specific to a method for evaluating a second database index and a distributed New SQL database.

Backsound

An Hbase present one of The most well-distributed SQL databases in The Hadoop analysis, The design concept present derived from The Bible of The Google, The Hbase main component comprising an HMmaster and an HRegistrer, an a type data model is provided for an a user, an a main key range is provided for a two dimensional input of registers, The HMmaster is provided for a managing and distributing The registers, and The HRegistrer is provided for a mapping and mapping of The registers, and The Hbase mapping and mapping of The registers, The Hbase present one of The least well-distributed SQL databases and mapping of The Hbase domains: in reactive application, a user, soft needles to form multi-dimensional query, and the existing Hbase cancer feedback non-primary query.

Discloseness of invasion

The environmental aspect of The innovation to The product a method for analyzing The absolute second index and The differential New SQL database system, The while can be used to find The absolute second index and The thermal absolute solution of The non-primary key request.

In order to achieve the above object, an embodiment of the present invention provides a modified New SQL database system, including:

Accessing a user request in an interface mode of JDCB/ODBC, where the user request comprises a data field Accessing to be write;

Adapting the user request, the adapting and generating a correcting execution plan;

Writing the data field requested by the user inter a data table recording to an execution plan;

When the data fields are written into the data a data table, generating a precision of index data in an inverted index form for the data fields to be written, and writing the precision of index data in an index table;

And returning the processing result of the refining writing the data to the user.

Further, the method lso comprises the following steps of and converting the user request into an SQL request in an SQL status form.

Further, the analysing the user request, the formulating, and the generating the correcting the executing plan include:

Judging of the previous stored SQL statement correlation to the SQL request exists in the shared cache pool, if so, output of the execution plan correlation to the previous stored SQL statement, other,

An And clipping out syntax check on the SQL request, if the syntax error information to a user, other wise,

(ii) a Semantic check is carried out on the SQL request, if the Semantic error return information to the user, other,

Carrying out view and expression conversion on the SQL request to object a correlation conversion result;

Selecting an optimizing recording to the conversion result to the object a correcting optimizing Selecting result;

Selecting a correcting data connecting mode and a correcting connecting sequence recording to the Selecting result of the optimizer;

Selecting a selected path accessing to the connection mode and the connection sequence;

And outputting the execution plan.

Coresponsondingly, an embodiment of the present introduction of flame compressor a distributed New SQL database system, including:

The JDCB/ODBC interface unit used for registering out The interactive operation with a user, a receiving user request and a returning processing result, a receiving The user request comprising a field to be write, a processing result is a processing result write in The field;

The system components a master unit, an SQLPLaner unit and a data processing unit, where The master unit is used for accessing a user request in a JDCB/ODBC interface mode, associating data communication amplitude of processors and manipulating The book process, and presenting The user request in The SQLPLanner unit;

The SQLPLaner unit being used for adapting The user request, The formulating and conditioning an execution plan accessing to The user request;

A worker unit, configured to execute the plate in parallel, and write the data requested by the user inter a data table of an Hbase unit, the Hbase unit being used for returning the processing result of the Hbase unit to the master unit;

The Hbase unit used for storing The data table and The index table, and The flame compressor a processor module used for generating a compressing of an index data in an inverted index for compressing from The data field to The write machine of The data field is written into The data table, and compressing The compressing of The index data in The index table;

The used of the distributed transaction manager for coordinating multiple partitions to fine distributed transaction manager while the work unit execution plates to a transaction.

Further, the JDCB/ODBC interface unit is Further configured to convert the user request to an SQL request in the form of an SQL status.

Further, the SQLPLanner unit is configured to:

And outputting the execution plan.

Further, the method lso comprises the following steps:

A monitor for marking a charge of metadata management, monitoring the load of the Region of the hbase unit, and registering the Region of the processor module of the hbase unit, the monitor being connected with the master unit.

(iii) the monitoring the load of the Region of the hbase unit and the redistributing the Region of the hbase unit module of the hbase unit include:

A retrieving data distribution information of the hbase unit, and Receiving load information of the worker unit in the master unit, the world load information comprising a load removal value of the worker unit;

Comparing the load removal value of the work unit with a preset load removal threshold, and if the load removal value is calculated to the exceeded the threshold, triggering the hbase unit to a performance second distribution on the Region on the server with a high shift rate and the Region on the server with a low shift rate;

Acquiring the data volume of each of the reach registers, determining the data volume of each of the reach registers and a preset data volume threshold, and triggering the hbase unit to the preset data volume of each of the reach registers and the preset data volume of each of the reach registers.

Further, the JDCB/ODBC interface unit include:

The JDBC application program module being used for retrieving The user request, The JDBC object method to The field an SQL status and extracting a result to The user;

The JDBC driver manager module is used for loading and locking The JDBC driver module for The JDBC application program module;

The JDBC driver module is used for executing The locking of The JDBC object method, sensing The SQL state correcting to The user request to The bottom database and returning The result of The reset operation from The bottom database to The JDBC application module.

The method for reflecting the second information index and the distributed new SQL database distributed by the information having the presence of the additional information such as the added information of the added information index and the added information index, the information of the user request, the information of The printer art can be used to print a non-primary key request, objects of the business effects of the user to a flexible assay index access to a specific business logic and a recording the multi-dimensional request of the user in a positive application.

Drawings

FIG. 1 is a schema flow chart of a data storage method to implementation 1 of the present introduction;

FIG. 2 is a schematic structural diagram of a distributed New SQL database protocol in implementation 2 of the present introduction.

Detailed Description

The technical solutions in The description of The present information with less detail and less description with reference to The drawing in The description of The present information, and The present of The description of The present embodiment with a part of The description of The present information, and The not of The present information with other embodiments, The present of The present invention with a part of The description of The present invention, and The not of The present invention with The use of The present invention in The description of The present invention.

Referring to fig. 1, fig. 1 is a schematic flow chart of a method for creating a second index recording to embodiment 1 of the present information, including the steps of:

S1, accessing a user request in an interface mode of JDCB/ODBC, where the user request comprises a data field requiring to be written;

S2, analyzing the user request, compiling and generating a correcting execution plan;

S3, writing the data field requested by the user inter a data table recording to an execution plan;

S4, the writing of the data fields are written to the data table, the generating of index data in an inverted index for the data fields to the write, and the writing of the index data in the index table;

And S5, returning the processing result of the data writing completion to the user.

Further, step S1 include and convert the user request into an SQL request in an SQL status form.

Further, the step S2 of matching the user request, the configuring and generating a correcting execution plan include:

S21, judging the heat of the shared cache pool has the pre-stored SQL sensor correlation to the SQL request, if yes, outputting the execution plan correlation to the pre-stored SQL sensor, if not,

S22, synchronizing the SQL request, if the synchronizing error return information to the user, otherwise,

S23, semantic checking the SQL request, if the semantic error return information to the user, other,

S24, a harvesting out view and expression conversion on the SQL request to object a correlation conversion result;

S25, selecting an optimizing recording to the conversion result to the object a correcting optimizing selection result;

S26, selecting a correlation data connection mode and a connection sequence recording to the result of the timer selection;

S27, selecting the searched path accessing to the connection mode and the connection sequence;

And D S28, generating an execution plan recording to the search path And outputting the execution plan.

The computer program product may include a computer program, a computer readable medium, a computer readable program, a computer readable storage medium, a computer readable program, an instruction readable program, a computer readable program, a.

The user is allowed to have a non-flexible basic a complexity of a second index of a second specific service so as to have a second hierarchical query of The second specific service of The second specific application.

Referring to fig. 2, fig. 2 is a structured schema diagram of a distributed New SQL database system acquisition to implementation 2 of the present information, the world of the implementation information:

The JDCB/ODBC interface unit 1 is used for sharing out The interactive operation with a user, receiving a user request and returning a processing result, The user request comprising a data field to be write, The processing result is a processing result write in The data field;

The master unit 2 being used for accessing a user request in a mode of The JDCB/ODBC interface unit 1, coordinating data communication a plurality of processors and managing The thresholds, and coordinating The user request to The SQLPLanner unit 3;

The SQLPLaner unit 3 is used for adapting The user request, adapting and customizing an execution place recording to The user request;

A worker unit 4, configured to execute the plate in parallel, and write the data field requested by the user inter a data table of an Hbase unit, and is used for returning the processing result of the Hbase unit 6 to the master unit;

The Hbase unit 6 is configured to store The data table and The index table, and The Hbase unit 6 background inputs a processor module 61, configured to generate a complexity of an index data in an inverted index for The data field to be write The data field is write in The data table, and write The complexity of The index data in The index table;

In addition, the Hbase unit 6 conveyor include a filtering module 62, an a filtering module 62 and a co-processor module 61 for generating an index table for data. her, the filtering module relays to a Filter build In the Hbase unit 6.

The coprocessors module 61 and The filtering module 62 generate an index table for The data, and thermally analyze The estimation of a secondary index.

The distributed transaction manager 5 is used for registering multiple copies to complete distributed transaction manager while the processor unit 4 execution of transaction, the processor unit 2 is used for registering multiple copies to complete distributed transaction manager, the processor unit 2 is used for registering multiple copies of the distributed transaction manager 5, And the processor unit is used for executing multiple copies of the distributed transaction manager 5.

Further, the JDCB/ODBC interface unit 1 is Further configured to convert the user request to an SQL request in the form of an SQL status.

Further, the SQLPLaner unit 3 is configured to:

And outputting the execution plan.

Further, the method lso comprises the following steps:

A monitor 8 for managing metadata, monitoring the load of Region 6, and responding Region of Region 61 of Region 6, the monitor 8 is connected to the master unit 2.

(iii) the monitoring of the load of the Region of the hbase unit 6 and the redistribution of the Region by the processor module 61 of the hbase unit 6 include:

(ii) a receivingtadiention data distribution information of the hbase unit 6, and a Receiving load information of the worker unit 4 in the master unit, the Receiving load information of the load information comprising a load removal value of the worker unit 4;

(ii) Comparing the load removal value of the worker unit 4 with a preset load removal threshold, and if the load removal value is calculated to be the exceeded the threshold, verifying the hbase unit 6 to the new distribution the Region on the server with the high threshold rate and the Region on the server with the low threshold rate;

Acquiring the data volume of each of the reach registers, merging the data volume of each of the reach registers and a preset data volume threshold, and if the data volume of each of the reach registers is merged to the exceeded the threshold, and merging the hbase unit 6 to the derivative the exceeded the preset data volume threshold into the two registers.

Further, the JDCB/ODBC interface unit 1 include:

The JDBC application program module 11 is used for receiving a user request, The closing a JDBC object method to The field an SQL status, and The extracting a result to The return to a user;

A JDBC driver manager module 12, configured to load and call a JDBC driver module 13 for the JDBC application module 11;

The JDBC driver module 13 is configured to execute The invocation of The JDBC object method, send an SQL state correction to The user request to The interpreting database, and return a result from The interpreting JD database to The BC application module.

The user request is associated with the program execution unit 2 and the related execution program is associated with the program execution unit 5 and the related execution program is associated with the program execution unit 1 and the related execution program is associated with the program execution unit 5 and the related execution program is associated with the program execution unit 5 and the related execution program execution unit 1 and the related execution program is associated with the program execution unit 5 and the related execution program 1 and execution program 5 and execution program, the coprocessors module 61 of the Hbase unit 6 generators the index data in the form of inverted index for the field to be write, and writes the index data in the index table, fine, the processing result of the Hbase unit 6 is returned to the master unit 2, and the master unit 2 is returned to the user.

The user is allowed to be transmitted to The flexible update The second index to The specific service location, and The multimedia request requirement of The user in The specific application method.

While the following is directed to the predicted embodiment of the present input, it will be understood by the following will be slid in the art of the present changes and modifications, may be made by the following of the macro of the present with the following removal of the from the scope and scope of the input.

Claims

1. A method for evaluating a two-level index is applicable to a distributed New SQL database system and comprises the following steps:

Accessing a user request in an interface mode of JDCB/ODBC, and converting the user request in an SQL status form, and updating the user request from a data field requiring to a write;

Analysis of the user request, compensation and generation of a correlation execution plan, specific, where a pre-stored SQL statement correlation to the SQL request executions in the shared cache pool is calculated, if yes, an execution plan correlation to the pre-stored SQL statement is output, if not,

Generating an execution plan recording to the search path and outputting the execution plan;

When the data fields are written into the data table, generating a plurality of indexes in an inverted index form for the data fields to be written into the data table, and writing the index data into the index table;

2. A distributed New SQL database system, comprising:

The JDCB/ODBC interface unit used for registering out The interactive operation with a user, and completing The steps of receiving a user request, converting The user request into an SQL request in an SQL status for, and completing a processing result;

The SQLPLaner unit being used for adapting The user request, The configuring and The storing an execution plan accessing The user request, The method of The following steps of,

The Hbase unit used for storing The data table and The index table, and The flame compressor a processor module used for generating a compressing a parameter of The index data in an inverted index for The data field to be written in The data table, and writing The compressing of The index data in The index table;

3. The distributed New SQL database system recording to claim 2, fuser composing:

A monitor for marking a charge of metadata management, a loading of a Region of the Hbase unit, and a registering the Region through a processor module of the Hbase unit, the monitor being connected with the master unit.

4. The distributed New SQL database system recording to claim 3, The detecting of The loading of The Hbase unit and The distributing of The Region by The processors module of The Hbase unit:

A receivingtainingdata distribution information of the Hbase unit, and a Receiving load information of the worker unit in the master unit, the Receiving load information comprising a load removal value of the worker unit;

(ii) Comparing the load removal value of the work unit with a preset load removal threshold, and if the load removal value is calculated to be the exceeded the threshold, triggering the Hbase unit to a formed second distribution on the Region on the server with a high project rate and the Region on the server with a low project rate;

Acquiring the data volume of each of the reach registers, discarding the data volume of each of the reach registers and a preset data volume threshold, and triggering the Hbase unit to di the registration data volume of each of the reach registers and the preset data volume of each of the reach registers.

5. The distributed New SQL database system of claim 2, where The JDCB/ODBC interface unit composites: