CN112000825A - Method and system for establishing electronic license storage model based on sub-warehouse and sub-table - Google Patents
Method and system for establishing electronic license storage model based on sub-warehouse and sub-table Download PDFInfo
- Publication number
- CN112000825A CN112000825A CN202010861398.3A CN202010861398A CN112000825A CN 112000825 A CN112000825 A CN 112000825A CN 202010861398 A CN202010861398 A CN 202010861398A CN 112000825 A CN112000825 A CN 112000825A
- Authority
- CN
- China
- Prior art keywords
- sub
- data
- module
- electronic license
- establishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000005012 migration Effects 0.000 claims description 19
- 238000013508 migration Methods 0.000 claims description 19
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000013500 data storage Methods 0.000 abstract description 5
- 230000002776 aggregation Effects 0.000 abstract description 4
- 238000004220 aggregation Methods 0.000 abstract description 4
- 238000007726 management method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000003780 insertion Methods 0.000 description 4
- 230000037431 insertion Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a system for establishing an electronic license storage model based on sub-base and sub-table, belonging to the field of data storage; the method comprises the following specific steps: s1, disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries; s2, selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene; s3, establishing an index database separately to inquire the non-hard key field; s4 adding branch tables to migrate the data; the electronic license database meeting the national standard can be established through the established storage model of the electronic license based on the sub-database and the sub-table, the electronic license system can support the aggregation of a large amount of license data, and efficient license sharing service can be provided for the outside in various service scenes.
Description
Technical Field
The invention discloses a method and a system for establishing an electronic license storage model based on sub-base and sub-table, relating to the technical field of data storage.
Background
The issuing of 6 electronic certificate national standards, such as GB/T36901-.
In addition, the access specifications of national government affairs service platforms such as 'C0126 + 2018 national government affairs service platform data exchange and sharing interface standard' provide a standard basis for the cross-layer data exchange of the electronic certificate, provide standard support for the data sharing application of the electronic certificate across departments and regions, and accelerate the integration and sharing of certificate type government affair information resources.
The establishment and the perfection of the electronic license standard specification accelerate the data aggregation of the electronic license library, the data volume of the electronic license library continuously increases, and meanwhile, the query requirements of users increase, which puts higher requirements on the data storage capacity of the electronic license library and the capacity of providing retrieval service for the outside.
The traditional mode of single storehouse list table stores the electron certificate, when the electron certificate assembles in a large number, has certain problem. First, under the influence of objective conditions of the database, when the data amount of a single table reaches a certain threshold, the data retrieval capability gradually decreases. Secondly, when a single record has large data, only relatively few rows can be stored in the same storage space, which may cause additional I/O overhead and reduce data access performance. Finally, different types of electronic certificates have large difference in service information content, and if single-table storage is adopted, the service information can only be stored as unstructured data, which is not beneficial to subsequent application.
In summary, the storage model of the single library and the single table is difficult to adapt to the requirement of large-scale aggregation of the electronic certificate data, and the capability of providing retrieval service to the outside is obviously reduced along with the increase of the data volume.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a system for establishing an electronic license storage model based on sub-library and sub-table, and the adopted technical scheme is as follows: a method for establishing an electronic license storage model based on sub-base and sub-table comprises the following specific steps:
s1, disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
s2, selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
s3, establishing an index database separately to inquire the non-hard key field;
s4 adds the branch table to migrate the data.
The S1 specific steps of disassembling the electronic license data and distributing the electronic license data to the corresponding vertical sub-libraries are as follows:
s101, establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
s102, the basic information and the business information of the electronic license are stored in a database.
The specific operation steps of S2 for selecting the shreddkey and the sub-table algorithm according to the electronic license data application scene are as follows:
s201, judging the data type of the electronic license;
s202, selecting a hash license identifier as a shreddkey;
s203, according to the hash certificate identification mod sub-table number, sub-table is conducted.
The specific steps of the S3 for separately establishing the index library to query the non-hard key field are as follows:
s301, establishing an index table of the license number and the license number of the holder;
s311, establishing an index table of the designated field;
s302, copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
s303, performing table division according to the value of the hash index field.
And the S4 migrates the data by adding the partial table in an expansion mode of 2^ (n +1) by adopting the data of the partial table of the hash remainder algorithm.
A system for establishing an electronic license storage model based on sub-base and sub-table specifically comprises a disassembling and distributing module, a scene selecting module, a field inquiring module and a data migration module:
disassembling the distribution module: disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
a scene selection module: selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
a field query module: establishing an index library separately to inquire the non-hard key field;
a data migration module: and adding the sub-table to migrate the data.
The disassembling and distributing module specifically comprises an index library dividing module and a library dividing and storing module:
an index base dividing module: establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
a database storage module: and performing library-based storage on the basic information and the service information of the electronic certificate.
The scene selection module specifically comprises a data judgment module, an identification selection module and a sub-table determination module:
a data judgment module: judging the data type of the electronic certificate;
identifying a selected module: selecting a hash license identifier as a shreddkey;
a sub-table determining module: and performing table division according to the hash certificate identification mod table division number.
The field query module specifically comprises a first establishing module, a second establishing module, a sub-table judging module and a sub-table operating module:
a first establishing module: establishing an index table of the certificate number and the certificate number of the holder;
a second establishing module: establishing an index table of the designated field;
a sub-table judging module: copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
a sub-meter operation module: and performing table division according to the value of the hash index field.
The data migration module migrates the data by adding the sublist in an expansion mode of 2^ (n +1) through adopting the data of the hash remainder algorithm sublist.
The invention has the beneficial effects that:
1. and a large amount of convergence of the electronic certificates is supported.
In the model, the electronic license data is stored in a vertical database and a horizontal table, so that the storage capacity of the electronic license is improved. The vertical database division enables the storage structure of the electronic license system to be clearer, management, maintenance, expansion and the like are convenient to carry out according to the characteristics of different business data, the bottleneck of single-machine hardware resources is avoided as much as possible, the number of database connections is reduced, and the pressure is shared among a plurality of servers. And the tables are divided horizontally, so that the data volume of the single table can be kept in the data volume with better performance supported by the database, the performance problem caused by overlarge data volume of the single table is solved, I/O (input/output) competition is avoided, and the table locking probability is reduced.
2. And flexible query of the electronic license is supported.
In the model, through the structured storage of the electronic license service information item, a user can customize and query specific types of licenses based on the service information item according to actual needs (such as business licenses, query based on a registration place and the like), so that the rapid retrieval of the electronic licenses under more service scenes can be supported, and the flexibility of the application of the electronic licenses is improved.
3. An electronic certificate sublist expansion method is provided.
In the model, the problems that the electronic license data is increased, the number of the single tables reaches a threshold value, and the sub-tables are expanded after the performance is reduced again are considered. A double-division table mode is provided, the data migration ratio is about 50% when the division table is expanded every time, and the division table expansion and switching are supported under the condition that the electronic license system keeps running.
The electronic license database meeting the national standard can be established through the established storage model of the electronic license based on the sub-database and the sub-table, the electronic license system can support the aggregation of a large amount of license data, and efficient license sharing service can be provided for the outside in various service scenes.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention; FIG. 2 is a schematic diagram of the system of the present invention; FIG. 3 is a diagram of an index table data structure; FIG. 4 is a license information splitting diagram; FIG. 5 is a diagram of an electronic certificate database and sub-table model.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The first embodiment is as follows:
a method for establishing an electronic license storage model based on sub-base and sub-table comprises the following specific steps:
s1, disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
s2, selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
s3, establishing an index database separately to inquire the non-hard key field;
s4 adding branch tables to migrate the data;
when the method is used for establishing an electronic license storage model, the characteristics of electronic license data are analyzed by combining national standard specifications and actual services, the electronic license data are disassembled according to S1 and are distributed into different corresponding vertical sub-libraries, then, based on the application scene of each part of electronic license data, a guard key and a sub-table algorithm are selected according to S2, then, an index library is independently established according to S3 to inquire non-guard key fields, sub-tables are added according to S4 to migrate the data, and sub-table expansion is realized;
the storage model established by the method supports mass convergence of the electronic certificates: in the model, the electronic license data is stored in a vertical database and a horizontal table, so that the storage capacity of the electronic license is improved. The vertical database division enables the storage structure of the electronic license system to be clearer, management, maintenance, expansion and the like are convenient to carry out according to the characteristics of different business data, the bottleneck of single-machine hardware resources is avoided as much as possible, the number of database connections is reduced, and the pressure is shared among a plurality of servers. The table is divided horizontally, so that the data volume of the single table can be kept in the data volume with better performance supported by the database, the performance problem caused by overlarge data volume of the single table is solved, I/O (input/output) competition is avoided, and the table locking probability is reduced;
the storage model established by the method supports flexible query of the electronic certificate: in the model, through the structured storage of the electronic license service information item, a user can customize and query specific types of licenses based on the service information item according to actual needs (such as business licenses, query based on a registration place and the like), so that the rapid retrieval of the electronic licenses under more service scenes can be supported, and the flexibility of the application of the electronic licenses is improved;
the storage model established by the method of the invention supports an electronic certificate sublist extension method: in the model, the problems that the electronic license data is increased, the number of the single tables reaches a threshold value, and the sub-tables are expanded after the performance is reduced again are considered. A double-divide-table mode is provided, the data migration ratio is about 50% when the sub-table is expanded each time, and sub-table expansion and switching are supported under the condition that the electronic license system keeps running;
the electronic license storage model based on the sub-base and sub-table established by the method can establish an electronic license base which accords with the national standard specification, and the electronic license system can support the convergence of a large amount of license data and can provide high-efficiency license sharing service to the outside in various service scenes;
further, the specific steps of the S1 disassembling the electronic license data and distributing the electronic license data to the corresponding vertical sub-libraries are as follows:
s101, establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
s102, performing sub-base storage on the basic information and the service information of the electronic license;
the electronic license index library is built according to S101 as shown in fig. 4 for the following reasons:
after the subsequent horizontal sub-tables are considered, if the license is searched by using the non-sharp key field, a plurality of tables are simultaneously inquired in the service, and the inquiry result is combined to obtain the required license data, so that the searching efficiency is reduced.
The electronic license index database is separately provided, and a user can customize and establish electronic license index information according to actual business requirements, so that the flexibility of the model in use is improved.
In addition, in the electronic license library, besides the basic information and the service information of the electronic license, the needs of electronic license management are considered, so that the 'used trace and trace tracing' needs to be realized, and log information such as sharing, changing, annual inspection, logout and the like of the electronic license needs to be stored; the electronic certificate sub-database and sub-table model is as shown in fig. 5, and an electronic certificate log database is established in the storage model and stores electronic certificate log information;
different license types and service information structures have larger difference, if the service information and the basic information are put in one record, the service information can only be stored in a form of unstructured data and is stored in a large field, and the I/O efficiency is reduced.
In the current national standard system, the retrieval of the electronic license is mainly carried out on the basis of the basic information of the electronic license, so that the retrieval frequency of the basic information of the electronic license is higher than that of the basic information of the general service, and the sub-database is convenient for selecting hardware resources according to actual requirements.
I/O contention is avoided, a user checking detailed information of the license and a user retrieving the license information do not mutually influence each other for reasons based on similarity, and electronic license file information is also independently built and stored in a database in the model;
further, the specific operation steps of selecting the shreddkey and the sub-table algorithm according to the electronic license data application scene in the S2 are as follows:
s201, judging the data type of the electronic license;
s202, selecting a hash license identifier as a shreddkey;
s203, according to the hash certificate identification mod sub-table number, sub-table is carried out;
in the model, an electronic license library is divided into: the system comprises an electronic certificate basic information base, an electronic certificate business information base, an electronic certificate file information base, an electronic certificate index base and an electronic certificate log base; the basic information base, the business information base and the file information base are established by vertically dividing the electronic certificate information;
according to the definition of national standard of electronic license, for an electronic license, the information includes the basic information, service information, file information and the like of the license; the basic information of the electronic certificate is the common attribute of all types of electronic certificates, and has similar data characteristics and retrieval requirements; the electronic license service information, different types of licenses and different service information are provided, in practical application, different retrieval requirements are possible, and the cards need to be combed by combining specific license types; the electronic license file is an OFD (office file format) file, and an original file or an added file of the license file is obtained usually based on a license identifier in basic information or a license type plus a license number of a license holder;
firstly, judging whether the data type of the electronic certificate is an electronic certificate basic information base, an electronic certificate business information base, an electronic certificate file information base, an electronic certificate index base and an electronic certificate log base according to S201;
the query of the basic information of the electronic certificate is usually based on the certificate identification/the certificate number of the holder/the serial number of the certificate, the actual service significance of the certificate is considered, the certificate identification of the electronic certificate is directly adopted as a shard key according to S202 to be sorted, and then the sorting is carried out according to the number of the sorted hash certificate identification mod according to S203;
the electronic license service information base is firstly listed according to the license types, the service information items of different license types naturally have differences, and the list can be established according to the combing results of the service information of each license type; secondly, the data volume and the application frequency difference of different types of electronic certificates are large, and for the types of the certificates with large data volume and large application, the classification can be further carried out according to the classification number mod of the hash certificate identifications according to S203, and the certificate data of the same type are dispersed into a plurality of tables;
the sub-table of the electronic license file information base is in a binary stream form, when a user retrieves files according to the non-license identification field, the user can firstly inquire the files through a license index base, a basic information base, a business information base and the like, and then acquire the identification information of the license and then acquire the license file by using the license identification; therefore, for the electronic certificate file library, direct query based on the certificate identification can be provided, and the electronic certificate file information library is sorted according to the hash certificate identification mod sorting number in S203;
the establishment of the electronic license index library is convenient for the retrieval of electronic license information, and the specific index table structure and sub-table related information can refer to the content of the next part of 'non-shrard key field query problem';
the electronic certificate log library is sorted, and the most concerned of the user is the log in the latest time period, so that the electronic certificate log information can be sorted according to the time range; for log information, only two operations of query and insertion exist, the most recent log table is usually operated, the past log table only relates to query operation, and no insertion operation exists;
further, the specific steps of separately establishing the index library by the S3 to query the non-hard key field are as follows:
s301, establishing an index table of the license number and the license number of the holder;
s311, establishing an index table of the designated field;
s302, copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
s303, performing table division according to the value of the hash index field;
the method comprises the steps that a storage model established by the method adopts a mode of independently establishing an index base to process the query problem of non-shredded key fields, according to the requirement of general services, firstly, an index table of license numbers and holder license numbers is established according to S301, the shredded keys are the license numbers and the holder license numbers respectively, a user can also copy data of related fields into the index table according to S311 as shown in figure three according to the index fields of a certain kind of specifically appointed licenses according to S311 according to the requirement, whether the index table is divided or not is selected according to the number of the licenses, and if the division is required, the division is carried out according to the value of a hash index field according to S303;
further, the S4 migrates the data by adding the partial table in an expansion mode of 2^ (n +1) by adopting the data of the partial table of the hash remainder algorithm.
When the data volume of a single table of a basic information table or a business data table and the like reaches a database threshold value and the data storage and query efficiency is reduced, the continuous addition of sub-tables needs to be considered, and the data access efficiency is guaranteed;
in the storage model, a mode of 2^ (n +1) is adopted during expansion according to the data of the hash remainder algorithm sublist adopted by S4.
Assuming that m tables (table 0, table 1, …, and table m-1) in the database are now expanded to 2m tables, the specific process is as follows:
firstly, newly building a table (table m, table m +1, …, table 2 m-1);
secondly, marking the values of the migration identification fields F in the tables 0 to m-1 as null;
third, the data in "Table 0" to "Table m-1" are migrated. Taking "table x" (0< x < m-1) as an example, the value of each piece of data "hash (shreddkey) mod 2 m" is calculated: record with result still equal to x, field F is marked 0; the calculation result is equal to the data of x + m, the field F is marked as 1, and the data is copied into the table x + m;
fourthly, adjusting the records of the sub-table number in the electronic license system from m to 2 m;
fifthly, deleting the data with the value of 1 in the 'table 0' to the 'table m-1', and completing the expansion;
the step has the advantages that only half of the data needs to be migrated during each expansion, and the electronic license system can still provide service to the outside as usual during the migration.
Example two:
a system for establishing an electronic license storage model based on sub-base and sub-table specifically comprises a disassembling and distributing module, a scene selecting module, a field inquiring module and a data migration module:
disassembling the distribution module: disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
a scene selection module: selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
a field query module: establishing an index library separately to inquire the non-hard key field;
a data migration module: adding a sub-table to migrate the data;
when the system is used for establishing an electronic license storage model, the characteristics of electronic license data are analyzed by combining national standard specifications and actual services, a disassembling and distributing module disassembles the electronic license data and distributes the electronic license data to different corresponding vertical sub-libraries, then based on the application scene of each part of electronic license data, a scene selection module selects a guard key and a sub-table algorithm, then an index library is established independently through a field query module to query non-guard key fields, and a data migration module is used for increasing sub-tables to migrate the data, so that sub-table expansion is realized;
the storage model established by the system of the invention supports mass convergence of the electronic certificates: in the model, the electronic license data is stored in a vertical database and a horizontal table, so that the storage capacity of the electronic license is improved. The vertical database division enables the storage structure of the electronic license system to be clearer, management, maintenance, expansion and the like are convenient to carry out according to the characteristics of different business data, the bottleneck of single-machine hardware resources is avoided as much as possible, the number of database connections is reduced, and the pressure is shared among a plurality of servers. The table is divided horizontally, so that the data volume of the single table can be kept in the data volume with better performance supported by the database, the performance problem caused by overlarge data volume of the single table is solved, I/O (input/output) competition is avoided, and the table locking probability is reduced;
the storage model established by the system supports flexible query of the electronic certificate: in the model, through the structured storage of the electronic license service information item, a user can customize and query specific types of licenses based on the service information item according to actual needs (such as business licenses, query based on a registration place and the like), so that the rapid retrieval of the electronic licenses under more service scenes can be supported, and the flexibility of the application of the electronic licenses is improved;
the electronic certificate sublist extension method is provided by the support of the storage model established by the system of the invention: in the model, the problems that the electronic license data is increased, the number of the single tables reaches a threshold value, and the sub-tables are expanded after the performance is reduced again are considered. A double-divide-table mode is provided, the data migration ratio is about 50% when the sub-table is expanded each time, and sub-table expansion and switching are supported under the condition that the electronic license system keeps running;
the electronic license storage model based on the sub-base and sub-table established by the system can establish an electronic license base which accords with the national standard specification, and the electronic license system can support the convergence of a large amount of license data and can provide high-efficiency license sharing service to the outside in various service scenes;
further, the disassembling and distributing module specifically includes an index library dividing module and a library dividing and storing module:
an index base dividing module: establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
a database storage module: performing database storage on the basic information and the service information of the electronic certificate;
as shown in fig. 4, the electronic license index library is built through the index library partitioning module for the following reasons:
after the subsequent horizontal sub-tables are considered, if the license is searched by using the non-sharp key field, a plurality of tables are simultaneously inquired in the service, and the inquiry result is combined to obtain the required license data, so that the searching efficiency is reduced.
The electronic license index database is separately provided, and a user can customize and establish electronic license index information according to actual business requirements, so that the flexibility of the model in use is improved.
In addition, in the electronic license library, besides the basic information and the service information of the electronic license, the needs of electronic license management are considered, so that the 'used trace and trace tracing' needs to be realized, and log information such as sharing, changing, annual inspection, logout and the like of the electronic license needs to be stored; the electronic certificate sub-database and sub-table model is as shown in fig. 5, and an electronic certificate log database is established in the storage model and stores electronic certificate log information;
different license types and service information structures have larger difference, if the service information and the basic information are put in one record, the service information can only be stored in a form of unstructured data and is stored in a large field, and the I/O efficiency is reduced.
In the current national standard system, the retrieval of the electronic license is mainly carried out on the basis of the basic information of the electronic license, so that the retrieval frequency of the basic information of the electronic license is higher than that of the basic information of the general service, and the sub-database is convenient for selecting hardware resources according to actual requirements.
I/O contention is avoided, a user checking detailed information of the license and a user retrieving the license information do not mutually influence each other for reasons based on similarity, and electronic license file information is also independently built and stored in a database in the model;
further, the scene selection module specifically includes a data judgment module, an identifier selection module, and a sub-table determination module:
a data judgment module: judging the data type of the electronic certificate;
identifying a selected module: selecting a hash license identifier as a shreddkey;
a sub-table determining module: according to the hash certificate identification mod sub-table number, sub-table is carried out;
in the model, an electronic license library is divided into: the system comprises an electronic certificate basic information base, an electronic certificate business information base, an electronic certificate file information base, an electronic certificate index base and an electronic certificate log base; the basic information base, the business information base and the file information base are established by vertically dividing the electronic certificate information;
according to the definition of national standard of electronic license, for an electronic license, the information includes the basic information, service information, file information and the like of the license; the basic information of the electronic certificate is the common attribute of all types of electronic certificates, and has similar data characteristics and retrieval requirements; the electronic license service information, different types of licenses and different service information are provided, in practical application, different retrieval requirements are possible, and the cards need to be combed by combining specific license types; the electronic license file is an OFD (office file format) file, and an original file or an added file of the license file is obtained usually based on a license identifier in basic information or a license type plus a license number of a license holder;
firstly, judging whether the data type of the electronic certificate is an electronic certificate basic information base, an electronic certificate business information base, an electronic certificate file information base, an electronic certificate index base and an electronic certificate log base by a data judgment module;
the query of the basic information of the electronic certificate is usually based on the certificate identification/the certificate number of the holder/the serial number of the certificate, the actual service significance of the certificate is considered, the certificate identification of the electronic certificate is directly used as a shard key by an identification selection module to be subjected to sub-tabulation, and then the sub-tabulation determination module is used for performing tabulation according to the sub-tabulation number of the hash certificate identification mod;
the electronic license service information base is firstly listed according to the license types, the service information items of different license types naturally have differences, and the list can be established according to the combing results of the service information of each license type; secondly, the data volume and the application frequency difference of different types of electronic certificates are large, and for the types of the certificates with large data volume and large application, the sub-tables can be further divided by a sub-table determining module according to the sub-table number of the hash certificate identifications mod, so that the certificate data of the same type are dispersed into a plurality of tables;
the sub-table of the electronic license file information base is in a binary stream form, when a user retrieves files according to the non-license identification field, the user can firstly inquire the files through a license index base, a basic information base, a business information base and the like, and then acquire the identification information of the license and then acquire the license file by using the license identification; therefore, for the electronic certificate file library, direct query based on the certificate identification can be provided, and the electronic certificate file information library is subjected to sub-tabulation according to the sub-tabulation determining module according to the hash certificate identification mod sub-tabulation number;
the establishment of the electronic license index library is convenient for the retrieval of electronic license information, and the specific index table structure and sub-table related information can refer to the content of the next part of 'non-shrard key field query problem';
the electronic certificate log library is sorted, and the most concerned of the user is the log in the latest time period, so that the electronic certificate log information can be sorted according to the time range; for log information, only two operations of query and insertion exist, the most recent log table is usually operated, the past log table only relates to query operation, and no insertion operation exists;
further, the field query module specifically includes a first establishing module, a second establishing module, a sub-table judging module and a sub-table operating module:
a first establishing module: s301, establishing an index table of the license number and the license number of the holder;
a second establishing module: s311, establishing an index table of the designated field;
a sub-table judging module: s302, copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
a sub-meter operation module: s303, performing table division according to the value of the hash index field;
the storage model established by the system processes the query problem of the non-shreddable key field by adopting a mode of independently establishing an index library, firstly establishing an index table of a license number and a holder license number through a first establishing module according to the requirement of general services, wherein the shreddable key is the license number and the holder license number respectively, a user can also establish an index table according to the requirement, a second establishing module is shown as a third drawing according to the specifically appointed index field of a certain type of license, then a sub-table judging module copies the data of the relevant field into the index table, and selects whether the index table is sub-table or not according to the number of the licenses, if so, the sub-table operation module is used for sub-table according to the value of a hash index field;
furthermore, the data migration module migrates the data by adding the branch table in an expansion mode of 2^ (n +1) by adopting the data of the hash remainder algorithm branch table;
when the data volume of a single table of a basic information table or a business data table and the like reaches a database threshold value and the data storage and query efficiency is reduced, the continuous addition of sub-tables needs to be considered, and the data access efficiency is guaranteed;
in the storage model, the data migration module adopts a 2^ (n +1) mode during expansion for the data adopting the 'hash remainder' algorithm sublist.
Assuming that m tables (table 0, table 1, …, and table m-1) in the database are now expanded to 2m tables, the specific process is as follows:
firstly, newly building a table (table m, table m +1, …, table 2 m-1);
secondly, marking the values of the migration identification fields F in the tables 0 to m-1 as null;
third, the data in "Table 0" to "Table m-1" are migrated. Taking "table x" (0< x < m-1) as an example, the value of each piece of data "hash (shreddkey) mod 2 m" is calculated: record with result still equal to x, field F is marked 0; the calculation result is equal to the data of x + m, the field F is marked as 1, and the data is copied into the table x + m;
fourthly, adjusting the records of the sub-table number in the electronic license system from m to 2 m;
fifthly, deleting the data with the value of 1 in the 'table 0' to the 'table m-1', and completing the expansion;
the advantage of using the data migration module is that only half of the data needs to be migrated each time of expansion, and the electronic license system can still provide service to the outside as usual during the migration process.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for establishing an electronic license storage model based on sub-base and sub-table is characterized by comprising the following specific steps:
s1, disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
s2, selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
s3, establishing an index database separately to inquire the non-hard key field;
s4 adds the branch table to migrate the data.
2. The method for establishing an electronic license storage model based on sub-base and sub-table as claimed in claim 1, wherein the step of S1 disassembling the electronic license data and distributing the electronic license data to the corresponding vertical sub-base comprises the following steps:
s101, establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
s102, the basic information and the business information of the electronic license are stored in a database.
3. The method for establishing the electronic license storage model based on the sub-library and sub-table as claimed in claim 2, wherein the specific operation steps of the S2 selecting the shard key and the sub-table algorithm according to the electronic license data application scenario are as follows:
s201, judging the data type of the electronic license;
s202, selecting a hash license identifier as a shreddkey;
s203, according to the hash certificate identification mod sub-table number, sub-table is conducted.
4. The method for establishing the electronic license storage model based on the sub-library and the sub-table as claimed in claim 3, wherein the specific steps of independently establishing the index library by the S3 to inquire the non-hard key field are as follows:
s301, establishing an index table of the license number and the license number of the holder;
s311, establishing an index table of the designated field;
s302, copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
s303, performing table division according to the value of the hash index field.
5. The method for establishing an electronic license storage model based on sub-library and sub-table as claimed in claim 4, wherein the S4 migrates the data by adding sub-tables in an extension manner of 2^ (n +1) by adopting the data of the sub-tables of the hash remainder algorithm.
6. A system for establishing an electronic license storage model based on sub-base and sub-table is characterized by specifically comprising a disassembling and distributing module, a scene selecting module, a field inquiring module and a data migration module:
disassembling the distribution module: disassembling the electronic license data and distributing the electronic license data to corresponding vertical sub-libraries;
a scene selection module: selecting a shreddkey and a sub-table algorithm according to the electronic license data application scene;
a field query module: establishing an index library separately to inquire the non-hard key field;
a data migration module: and adding the sub-table to migrate the data.
7. The system for establishing an electronic license storage model based on sub-base and sub-table as claimed in claim 6, wherein the disassembling and allocating module specifically comprises an index base dividing module and a sub-base storage module:
an index base dividing module: establishing a partitioned electronic license index library according to the vertical segmentation of the electronic license information;
a database storage module: and performing library-based storage on the basic information and the service information of the electronic certificate.
8. The system for establishing an electronic license storage model based on sub-base and sub-table as claimed in claim 7, wherein the scene selection module specifically comprises a data judgment module, an identification selection module and a sub-table determination module:
a data judgment module: judging the data type of the electronic certificate;
identifying a selected module: selecting a hash license identifier as a shreddkey;
a sub-table determining module: and performing table division according to the hash certificate identification mod table division number.
9. The system for establishing an electronic license storage model based on sub-base and sub-table as claimed in claim 8, wherein the field query module specifically comprises a first establishing module, a second establishing module, a sub-table judging module and a sub-table operating module:
a first establishing module: establishing an index table of the certificate number and the certificate number of the holder;
a second establishing module: establishing an index table of the designated field;
a sub-table judging module: copying data of related fields into an index table, and selecting whether the index table is divided into tables according to the number of licenses;
a sub-meter operation module: and performing table division according to the value of the hash index field.
10. The system for establishing an electronic license storage model based on sub-base and sub-table as claimed in claim 9, wherein the data migration module migrates data by adding sub-tables in an extension manner of 2^ (n +1) by adopting data of a hash remainder algorithm sub-table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010861398.3A CN112000825A (en) | 2020-08-25 | 2020-08-25 | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010861398.3A CN112000825A (en) | 2020-08-25 | 2020-08-25 | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112000825A true CN112000825A (en) | 2020-11-27 |
Family
ID=73471672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010861398.3A Pending CN112000825A (en) | 2020-08-25 | 2020-08-25 | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000825A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112650865A (en) * | 2021-01-27 | 2021-04-13 | 南威软件股份有限公司 | Method and system for solving multi-region license data conflict based on flexible rule |
| CN113407811A (en) * | 2021-07-06 | 2021-09-17 | 广州汇思信息科技股份有限公司 | Table dividing method and device for learning platform, computer equipment and storage medium |
-
2020
- 2020-08-25 CN CN202010861398.3A patent/CN112000825A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112650865A (en) * | 2021-01-27 | 2021-04-13 | 南威软件股份有限公司 | Method and system for solving multi-region license data conflict based on flexible rule |
| WO2022160373A1 (en) * | 2021-01-27 | 2022-08-04 | 南威软件股份有限公司 | Method and system for solving multi-region certificate data conflict based on flexible rules |
| CN113407811A (en) * | 2021-07-06 | 2021-09-17 | 广州汇思信息科技股份有限公司 | Table dividing method and device for learning platform, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427684B (en) | Data query method and device and computing equipment | |
| CN108563923B (en) | Distributed storage method and system for genetic variation data | |
| US6438562B1 (en) | Parallel index maintenance | |
| CN102122285B (en) | Data cache system and data inquiry method | |
| CN101354726B (en) | Method for managing memory metadata of cluster file system | |
| CN108460041B (en) | Data processing method and device | |
| CN104794123A (en) | Method and device for establishing NoSQL database index for semi-structured data | |
| CN110147407B (en) | Data processing method and device and database management server | |
| CN111008521B (en) | Method, device and computer storage medium for generating wide table | |
| CN103473239A (en) | Method and device for updating data of non relational database | |
| CN110399368B (en) | Method for customizing data table, data operation method and device | |
| JP6928677B2 (en) | Data processing methods and equipment for performing online analysis processing | |
| CN112000825A (en) | Method and system for establishing electronic license storage model based on sub-warehouse and sub-table | |
| CN103488687A (en) | Searching system and searching method of big data | |
| US7162478B2 (en) | System and method for correlated fragmentations in databases | |
| CN111552710B (en) | Query optimization method for distributed database | |
| CN112163048A (en) | Method and device for realizing OLAP analysis based on ClickHouse | |
| CN110955704A (en) | Data management method, device, equipment and storage medium | |
| CN113742343A (en) | Data splitting method, device and storage medium based on large amount of service data scenes | |
| CN112148680B (en) | File system metadata management method based on distributed graph database | |
| WO2017118335A1 (en) | Mapping method and device | |
| US20220253419A1 (en) | Multi-record index structure for key-value stores | |
| US20080097971A1 (en) | Peer-to-peer based secondary key search method and system for cluster database | |
| CN111488340A (en) | Data processing method and device and electronic equipment | |
| CN102006156B (en) | Method and system for synchronizing configuration data among boards |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |