CN106991190A - A kind of database automatically creates subdata base system - Google Patents

A kind of database automatically creates subdata base system Download PDF

Info

Publication number
CN106991190A
CN106991190A CN201710232919.7A CN201710232919A CN106991190A CN 106991190 A CN106991190 A CN 106991190A CN 201710232919 A CN201710232919 A CN 201710232919A CN 106991190 A CN106991190 A CN 106991190A
Authority
CN
China
Prior art keywords
data
database
coding
subdata
primary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710232919.7A
Other languages
Chinese (zh)
Inventor
梁启新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Inspur Smart Computing Technology Co Ltd
Original Assignee
Guangdong Inspur Big Data Research Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Inspur Big Data Research Co Ltd filed Critical Guangdong Inspur Big Data Research Co Ltd
Priority to CN201710232919.7A priority Critical patent/CN106991190A/en
Publication of CN106991190A publication Critical patent/CN106991190A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275Synchronous replication

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention provides a kind of database and automatically creates subdata base system, it is related to database technology, when user approaches, equaled or exceeded the load threshold of the primary database to the visit capacity of a primary database, load equalizer creates one, two or more subdata bases according to visit capacity elasticity, so as to realize the extension of database, share visit capacity.

Description

A kind of database automatically creates subdata base system
Technical field
Subdata base system is automatically created the present invention relates to database technology, more particularly to a kind of database.
Background technology
Database is that the data acquisition system in second-level storage is organized and deposited according to certain data model, this data Set has following features:Do not repeat as far as possible, be in an optimal manner a variety of application services of some particular organization, its data knot Structure increasing to data, is deleted, changed, looking into and be managed and controlled by unified software independently of using its application program.From development History sees that database is the advanced stage of data management, and it is grown up by file management system.
With using and demand database, the technical problem to be solved is to provide a kind of Database Systems, as user couple When the visit capacity of one primary database approaches, equaled or exceeded the load threshold of the primary database, load equalizer is according to the visit The amount of asking elasticity creates one, two or more subdata bases, so as to realize the extension of database, shares visit capacity.
The content of the invention
In order to solve the problem, the present invention proposes a kind of database and automatically creates subdata base system.When user is to one When the visit capacity of individual primary database approaches, equaled or exceeded the load threshold of the primary database, load equalizer is according to the access Amount elasticity creates one, two or more subdata bases, so as to realize the extension of database, shares visit capacity.
The technical scheme is that:
A kind of database automatically creates subdata base system,
Including load equalizer, primary database and subdata base, the quantity of subdata base is not less than primary database quantity, Mei Gezhu Database is the database for allowing user to carry out data read-write operation, and each primary database only preserves a type of data, Each primary database corresponds to preserve different types of data respectively;
When user approaches, equaled or exceeded the load threshold of the primary database to the visit capacity of a primary database, load is equal Weighing apparatus creates more than one subdata base according to visit capacity elasticity, and each subdata base is only to allow user to carry out data The database of read operation.
After subdata base is created, load equalizer sends primary data to the corresponding primary database of subdata base and synchronously referred to Order, the master data receives primary data synchronic command, primary data synchronic command is deferred to, at the beginning of the data that the primary database is stored Beginning is synchronized to the corresponding whole subdata bases of the master data.
Load equalizer receives data write request, and load equalizer judges according to received data write request Data type, selects from whole primary databases the primary database for preserving the type data to be according to Database Systems access strategy Target database, data write instruction is sent to target database, and the data asked are write into target database, and load is equal Weighing apparatus sends later data synchronic command to the target database, so that the target database is by newest write data post It is synchronized to the corresponding whole subdata bases of the target database;
When newest write data post is synchronized to the corresponding whole subdata bases of the target database by target database During:
It is binary log information, binary log packet that data are updated into operation note when target database writes data Include data and update operation species and data renewal operation content, include data increase operation, data deletion according to operation species are updated Operation and data change operation, and data, which update operation content, includes table name, field name, field attribute information and literary name segment length Degree;
The target database sends binary log information and gives the target database corresponding whole subdata bases, the target data The corresponding whole subdata bases in storehouse receive the binary log information that target database is sent, based on the binary log received Information, subdata base generates three kinds of functional modules, is respectively:Realize that data increase the SQL SQL of operating function Module, the SQL SQL modules for realizing data deletion action function and the knot for realizing data change operating function Structure query language SQL modules, extract the data in binary log information and update operation species and data renewal operation content, If it is that data increase operation that the data extracted, which update the corresponding data of operation content and update operation species, the data are updated and grasped It is embedded into as content in the SQL SQL modules for realizing data increase operating function, data increase behaviour is realized in execution Make the SQL SQL modules of function, the data of subdata storehouse same memory location carry out data increase operation, real Existing data increase operation;If it is data deletion action that the data extracted, which update the corresponding data of operation content and update operation species, Then data renewal operation content is embedded into the SQL SQL modules for realizing data deletion action function, held Row realizes the SQL SQL modules of data deletion action function, and the data of subdata storehouse same memory location are entered Row data deletion action, realizes data deletion action;If the data extracted update operation content, corresponding data update operation kind Class is that data change operation, then data renewal operation content is embedded into the structuralized query for realizing that data change operating function In language SQL modules, the SQL SQL modules for realizing that data change operating function are performed, subdata storehouse is identical The data of storage location carry out data and change operation, realize that data change operation.
Load equalizer receives data read request, the type of the reading data is judged, according to the type of the reading data It is determined that preserve the primary database and whole subdata bases of the type data, it is determined that preserving the primary databases of the type data and complete In portion's subdata base, Database Systems access speed strategy of the load equalizer based on Domain Name System carries out intelligence Matching, selects nearest database, the nearest database is referred to as matching database, to matched data to send the user asked Storehouse sends data read command, and matching database receives the data read command and provides digital independent according to data read command The data of request.
When the nearest database of user's selection to send request is primary database, only it is determined that preserving the type data Whole subdata bases in, Database Systems access speed strategy of the load equalizer based on Domain Name System is carried out Intelligent Matching, selects nearest subdata base, the nearest subdata base is referred to as matching database to send the user asked, to Matching database sends data read command, and matching database receives the data read command and provided according to data read command The data of data read request.
Subdata base structure is:
Based on coding standard set in advance, unique encodings are carried out to field, unique encodings are carried out to table, between table and table Data relationship carries out unique encodings, generates node cluster coding structure table, and the content of node cluster coding structure table storage is included to word Duan Jinhang unique coding, to table carry out unique coding and between table and table data relationship carry out it is unique Coding, forms coding nodes, based on the subordinate relation between the data relationship between table, field and table and table, in above-mentioned section Subordinate mapping in point cluster coding structure table between the coding of the coding of correspondence establishment table, the coding of field and data relationship, Map the title corresponding to each coding in the node cluster value comparison table of establishment, the title include the title of table, the title of field or The title of data relationship between table and table, creates the coding section with unique encodings under the corresponding coding nodes of field The corresponding field data of each coding child node is stored in point, each coding child node one memory space of correspondence, memory space.
Brief description of the drawings
Fig. 1 is the workflow schematic diagram of the present invention.
Embodiment
More detailed illustrate is carried out to present disclosure below:
A kind of Database Systems, including load equalizer, multiple primary databases and multiple subdata bases, quantity of subdata base etc. In or more than primary database quantity, each primary database is the database for allowing user to carry out data read-write operation, Mei Gezhu Database only preserves a type of data, and each primary database corresponds to preserve different types of data respectively;
When user approaches, equaled or exceeded the load threshold of the primary database to the visit capacity of a primary database, load is equal Weighing apparatus creates one, two or more subdata bases according to visit capacity elasticity, and each subdata base is only to allow user to enter The database of row data reading operation;
After subdata base is created, load equalizer sends primary data synchronic command to the corresponding primary database of subdata base, The master data receives primary data synchronic command, defers to primary data synchronic command, the data that the primary database is stored are initial The corresponding whole subdata bases of the master data are synchronized to, primary database and whole subdata bases corresponding with the primary database are kept Data consistency;
Load equalizer receives data write request, and load equalizer judges data according to received data write request Type, the primary database for selecting to preserve the type data from whole primary databases according to Database Systems access strategy is target Database, data write instruction is sent to target database, and the data asked are write into target database, load equalizer Later data synchronic command is sent to the target database, so that the target database is synchronous by newest write data post To the corresponding whole subdata bases of the target database;
Load equalizer receives data read request, judges the type of the reading data, is determined according to the type of the reading data The primary database and whole subdata bases of the type data are preserved, it is determined that preserving the primary database and all sons of the type data In database, Database Systems access speed strategy of the load equalizer based on Domain Name System carries out intelligent Match somebody with somebody, the nearest database of user's selection to send request, the nearest database is referred to as matching database, to matching database Data read command is sent, matching database receives the data read command and provides digital independent according to data read command and asks The data asked.
Above-mentioned Database Systems, when the nearest database of user's selection to send request is primary database, only true Surely in the whole subdata bases for preserving the type data, Database Systems of the load equalizer based on Domain Name System Access speed strategy carries out intelligent Matching, the nearest subdata base of user's selection to send request, the nearest subdata base Referred to as matching database, data read command is sent to matching database, and matching database receives the data read command and root The data of data read request are provided according to data read command.
Above-mentioned Database Systems, the target database pair is synchronized in target database by newest write data post During during the whole subdata bases answered:
It is binary log information, binary log packet that data are updated into operation note when target database writes data Include data and update operation species and data renewal operation content, include data increase operation, data deletion according to operation species are updated Operation and data change operation, and data, which update operation content, includes table name, field name, field attribute information and literary name segment length Degree;
The target database sends binary log information and gives the target database corresponding whole subdata bases, the target data The corresponding whole subdata bases in storehouse receive the binary log information that target database is sent, based on the binary log received Information, subdata base generates three kinds of functional modules, is respectively:Realize that data increase the SQL SQL of operating function Module, the SQL SQL modules for realizing data deletion action function and the knot for realizing data change operating function Structure query language SQL modules, extract the data in binary log information and update operation species and data renewal operation content, If it is that data increase operation that the data extracted, which update the corresponding data of operation content and update operation species, the data are updated and grasped It is embedded into as content in the SQL SQL modules for realizing data increase operating function, data increase behaviour is realized in execution Make the SQL SQL modules of function, the data of subdata storehouse same memory location carry out data increase operation, real Existing data increase operation;If it is data deletion action that the data extracted, which update the corresponding data of operation content and update operation species, Then data renewal operation content is embedded into the SQL SQL modules for realizing data deletion action function, held Row realizes the SQL SQL modules of data deletion action function, and the data of subdata storehouse same memory location are entered Row data deletion action, realizes data deletion action;If the data extracted update operation content, corresponding data update operation kind Class is that data change operation, then data renewal operation content is embedded into the structuralized query for realizing that data change operating function In language SQL modules, the SQL SQL modules for realizing that data change operating function are performed, subdata storehouse is identical The data of storage location carry out data and change operation, realize that data change operation.
Above-mentioned Database Systems, subdata base structure is:
Based on coding standard set in advance, unique encodings are carried out to field, unique encodings are carried out to table, between table and table Data relationship carries out unique encodings, generates node cluster coding structure table, and the content of node cluster coding structure table storage is included to word Duan Jinhang unique coding, to table carry out unique coding and between table and table data relationship carry out it is unique Coding, forms coding nodes, based on the subordinate relation between the data relationship between table, field and table and table, in above-mentioned section Subordinate mapping in point cluster coding structure table between the coding of the coding of correspondence establishment table, the coding of field and data relationship, Map the title corresponding to each coding in the node cluster value comparison table of establishment, the title include the title of table, the title of field or The title of data relationship between table and table, creates the coding section with unique encodings under the corresponding coding nodes of field The corresponding field data of each coding child node is stored in point, each coding child node one memory space of correspondence, memory space.
The data syn-chronization of primary database and whole subdata bases corresponding with the primary database, it is ensured that the uniformity of data. Each primary database is the database for allowing user to carry out data read-write operation, and each primary database only preserves a type of Data, each primary database corresponds to preserve different types of data respectively, is easy to management and the extended operation of database.

Claims (7)

1. a kind of database automatically creates subdata base system, it is characterised in that
Including load equalizer, primary database and subdata base, the quantity of subdata base is not less than primary database quantity, Mei Gezhu Database is the database for allowing user to carry out data read-write operation, and each primary database only preserves a type of data, Each primary database corresponds to preserve different types of data respectively;
When user approaches, equaled or exceeded the load threshold of the primary database to the visit capacity of a primary database, load is equal Weighing apparatus creates more than one subdata base according to visit capacity elasticity, and each subdata base is only to allow user to carry out data The database of read operation.
2. system according to claim 1, it is characterised in that
After subdata base is created, load equalizer sends primary data synchronic command to the corresponding primary database of subdata base, The master data receives primary data synchronic command, defers to primary data synchronic command, the data that the primary database is stored are initial It is synchronized to the corresponding whole subdata bases of the master data.
3. system according to claim 2, it is characterised in that
Load equalizer receives data write request, and load equalizer judges data according to received data write request Type, the primary database for selecting to preserve the type data from whole primary databases according to Database Systems access strategy is target Database, data write instruction is sent to target database, and the data asked are write into target database, load equalizer Later data synchronic command is sent to the target database, so that the target database is synchronous by newest write data post To the corresponding whole subdata bases of the target database.
4. system according to claim 3, it is characterised in that
When newest write data post is synchronized to the corresponding whole subdata bases of the target database by target database During:
It is binary log information, binary log packet that data are updated into operation note when target database writes data Include data and update operation species and data renewal operation content, include data increase operation, data deletion according to operation species are updated Operation and data change operation, and data, which update operation content, includes table name, field name, field attribute information and literary name segment length Degree;
The target database sends binary log information and gives the target database corresponding whole subdata bases, the target data The corresponding whole subdata bases in storehouse receive the binary log information that target database is sent, based on the binary log received Information, subdata base generates three kinds of functional modules, is respectively:Realize that data increase the SQL SQL of operating function Module, the SQL SQL modules for realizing data deletion action function and the knot for realizing data change operating function Structure query language SQL modules, extract the data in binary log information and update operation species and data renewal operation content, If it is that data increase operation that the data extracted, which update the corresponding data of operation content and update operation species, the data are updated and grasped It is embedded into as content in the SQL SQL modules for realizing data increase operating function, data increase behaviour is realized in execution Make the SQL SQL modules of function, the data of subdata storehouse same memory location carry out data increase operation, real Existing data increase operation;If it is data deletion action that the data extracted, which update the corresponding data of operation content and update operation species, Then data renewal operation content is embedded into the SQL SQL modules for realizing data deletion action function, held Row realizes the SQL SQL modules of data deletion action function, and the data of subdata storehouse same memory location are entered Row data deletion action, realizes data deletion action;If the data extracted update operation content, corresponding data update operation kind Class is that data change operation, then data renewal operation content is embedded into the structuralized query for realizing that data change operating function In language SQL modules, the SQL SQL modules for realizing that data change operating function are performed, subdata storehouse is identical The data of storage location carry out data and change operation, realize that data change operation.
5. system according to claim 2, it is characterised in that
Load equalizer receives data read request, judges the type of the reading data, is determined according to the type of the reading data The primary database and whole subdata bases of the type data are preserved, it is determined that preserving the primary database and all sons of the type data In database, Database Systems access speed strategy of the load equalizer based on Domain Name System carries out intelligent Match somebody with somebody, the nearest database of user's selection to send request, the nearest database is referred to as matching database, to matching database Data read command is sent, matching database receives the data read command and provides digital independent according to data read command and asks The data asked.
6. system according to claim 5, it is characterised in that
When the nearest database of user's selection to send request is primary database, only it is determined that preserving the complete of the type data In portion's subdata base, Database Systems access speed strategy of the load equalizer based on Domain Name System carries out intelligence Matching, selects nearest subdata base, the nearest subdata base is referred to as matching database, to matching to send the user asked Database sends data read command, and matching database receives the data read command and provides data according to data read command The data of read requests.
7. system according to claim 2, it is characterised in that
Subdata base structure is:
Based on coding standard set in advance, unique encodings are carried out to field, unique encodings are carried out to table, between table and table Data relationship carries out unique encodings, generates node cluster coding structure table, and the content of node cluster coding structure table storage is included to word Duan Jinhang unique coding, to table carry out unique coding and between table and table data relationship carry out it is unique Coding, forms coding nodes, based on the subordinate relation between the data relationship between table, field and table and table, in above-mentioned section Subordinate mapping in point cluster coding structure table between the coding of the coding of correspondence establishment table, the coding of field and data relationship, Map the title corresponding to each coding in the node cluster value comparison table of establishment, the title include the title of table, the title of field or The title of data relationship between table and table, creates the coding section with unique encodings under the corresponding coding nodes of field The corresponding field data of each coding child node is stored in point, each coding child node one memory space of correspondence, memory space.
CN201710232919.7A 2017-04-11 2017-04-11 A kind of database automatically creates subdata base system Pending CN106991190A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710232919.7A CN106991190A (en) 2017-04-11 2017-04-11 A kind of database automatically creates subdata base system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710232919.7A CN106991190A (en) 2017-04-11 2017-04-11 A kind of database automatically creates subdata base system

Publications (1)

Publication Number Publication Date
CN106991190A true CN106991190A (en) 2017-07-28

Family

ID=59415947

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710232919.7A Pending CN106991190A (en) 2017-04-11 2017-04-11 A kind of database automatically creates subdata base system

Country Status (1)

Country Link
CN (1) CN106991190A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107563219A (en) * 2017-08-28 2018-01-09 深圳市华傲数据技术有限公司 A kind of data base management method and device
CN108763300A (en) * 2018-04-19 2018-11-06 北京奇艺世纪科技有限公司 A kind of data query method and device
CN110647347A (en) * 2019-09-05 2020-01-03 广州市蜗牛互动科技有限公司 Mobile application configuration method and equipment
WO2020224098A1 (en) * 2019-05-08 2020-11-12 平安科技(深圳)有限公司 Cloud access method and apparatus based on global server load balance, and storage medium
WO2021068521A1 (en) * 2019-10-12 2021-04-15 浪潮电子信息产业股份有限公司 Data processing method, apparatus and device for local storage engine system
CN116166639A (en) * 2023-03-03 2023-05-26 武汉骏信达信息咨询有限公司 Big data processing method and system based on artificial intelligence

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412860A (en) * 2012-10-25 2013-11-27 华为技术有限公司 Method and device for extending database and database system
CN103559323A (en) * 2013-11-22 2014-02-05 盛杰 Database implementation method
CN106021593A (en) * 2016-06-07 2016-10-12 浪潮电子信息产业股份有限公司 Replication processing method in takeover process of first database and second database

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412860A (en) * 2012-10-25 2013-11-27 华为技术有限公司 Method and device for extending database and database system
CN103559323A (en) * 2013-11-22 2014-02-05 盛杰 Database implementation method
CN106021593A (en) * 2016-06-07 2016-10-12 浪潮电子信息产业股份有限公司 Replication processing method in takeover process of first database and second database

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107563219A (en) * 2017-08-28 2018-01-09 深圳市华傲数据技术有限公司 A kind of data base management method and device
CN107563219B (en) * 2017-08-28 2020-04-24 深圳市华傲数据技术有限公司 Database management method and device
CN108763300A (en) * 2018-04-19 2018-11-06 北京奇艺世纪科技有限公司 A kind of data query method and device
CN108763300B (en) * 2018-04-19 2020-07-31 北京奇艺世纪科技有限公司 Data query method and device
WO2020224098A1 (en) * 2019-05-08 2020-11-12 平安科技(深圳)有限公司 Cloud access method and apparatus based on global server load balance, and storage medium
CN110647347A (en) * 2019-09-05 2020-01-03 广州市蜗牛互动科技有限公司 Mobile application configuration method and equipment
WO2021068521A1 (en) * 2019-10-12 2021-04-15 浪潮电子信息产业股份有限公司 Data processing method, apparatus and device for local storage engine system
CN116166639A (en) * 2023-03-03 2023-05-26 武汉骏信达信息咨询有限公司 Big data processing method and system based on artificial intelligence
CN116166639B (en) * 2023-03-03 2024-04-26 西安快鱼科技有限公司 Big data processing method and system based on artificial intelligence

Similar Documents

Publication Publication Date Title
CN106991190A (en) A kind of database automatically creates subdata base system
US11580070B2 (en) Utilizing metadata to prune a data set
US11816126B2 (en) Large scale unstructured database systems
JP7309005B2 (en) Database tenant migration system and method
EP3304356B1 (en) Table partitioning within distributed database systems
CN100399327C (en) Managing file system versions
CN104657459B (en) A kind of mass data storage means based on file granularity
US20200334207A1 (en) Synchronization of metadata in a distributed storage system
WO2021169113A1 (en) Data management method and apparatus, and computer device and storage medium
CN102708165B (en) Document handling method in distributed file system and device
CN103488685B (en) Fragmented-file storage method based on distributed storage system
CN104133882A (en) HDFS (Hadoop Distributed File System)-based old file processing method
CN101169785A (en) Clustered database system dynamic loading balancing method
CN105608228B (en) A kind of efficient distributed RDF data storage method
JP2003522344A (en) Database synchronization / organization system and method
US20130254240A1 (en) Method of processing database, database processing apparatus, computer program product
CN104298760A (en) Data processing method and data processing device applied to data warehouse
US10712964B2 (en) Pre-forking replicas for efficient scaling of a distributed data storage system
CN102483768A (en) Policy-based storage structure distribution
CN104281717B (en) A kind of method for setting up magnanimity ID mapping relations
CN109684282A (en) A kind of method and device constructing metadata cache
CN110347651A (en) Method of data synchronization, device, equipment and storage medium based on cloud storage
US20220035786A1 (en) Distributed database management system with dynamically split b-tree indexes
WO2022095366A1 (en) Redis-based data reading method and apparatus, device, and readable storage medium
CN102202087A (en) Method for identifying storage equipment and system thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170728

RJ01 Rejection of invention patent application after publication