CN103778133A - Database object changing method and device - Google Patents
Database object changing method and device Download PDFInfo
- Publication number
- CN103778133A CN103778133A CN201210398278.XA CN201210398278A CN103778133A CN 103778133 A CN103778133 A CN 103778133A CN 201210398278 A CN201210398278 A CN 201210398278A CN 103778133 A CN103778133 A CN 103778133A
- Authority
- CN
- China
- Prior art keywords
- database
- database object
- dependence
- attribute information
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database object changing method and device. The database object changing method comprises the steps of collecting the dependence relation of a database dependence system and a database object, wherein the dependence relation includes the correspondence relation of the dependence system and the dependent database object and attribute information of the database object; receiving and resolving changing operation on the database object to obtain the attribute information of the database object required to be changed; judging whether the dependence relation includes the attribute information of the database object required to be changed or not; sending a change prompt message to the corresponding database dependence system and executing the changing operation on the database object if the dependence relation includes the attribute information of the database object required to be changed. By adopting the database object changing method and device, a corresponding data sheet in the dependence system can be changed when a data sheet structure in a database changes.
Description
Technical field
The application relates to database technical field, particularly relates to a kind of variation and device of database object.
Background technology
Database has had application widely in areas of information technology, and particularly in Internet firm, data-base cluster scale is along with the expansion of business is increasing; In database scale development, produce the dependence system (as data warehouse, search system) of various databases, to meet the demand of various data analyses and data search.
Data warehouse need to the line based in database on business datum analyze, be then converted to and be suitable for the content that Various types of data form extracts; Search system need to be analyzed based on business datum on line, is then converted to the data content for being suitable for searching service.
Dependence system is extracted data and is obtained by table, if the structure of table changes, the process of extracting data just there will be mistake, thereby causes dependence system to go wrong.For example: dependence system needs a user message table " USER_INFO ", extract once every day, there is people that the table name of this table is revised as to " USER_BASE_INFO " when daytime, if down-stream system does not know that table name claims to change, or just can not extract desired data according to former table name.
The database of some catenet companies at present, only the online hundreds of thousands that just reaches using is shown at present.Simultaneously, these systems that depend on database have again huge and numerous and diverse storage content separately, and can adopt more cloud computing technology, therefore, dependence between database and dependence system is very complicated, in the time that list structure in database occurs to change, artificially cannot judge that the tables of data that need to rely on system to which does corresponding change, thereby cause the normal operation of the system that relies on to go wrong.
Therefore, need at present the technical matters that those skilled in the art solve to be exactly, a kind of change mechanism of database object is provided, when data list structure in database is changed, can change the corresponding tables of data in dependence system.
Summary of the invention
The application's technical matters to be solved is to provide a kind of variation of database object, when data list structure in database is changed, can change the corresponding tables of data in dependence system.
The application also provides a kind of change device of database object, in order to guarantee said method application and realization in practice.
In order to address the above problem, the application discloses a kind of variation of database object, comprising:
Collect database and rely on the dependence of system and database object, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Receive and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Judge in described dependence whether comprise the attribute information of the database object of request change;
If so, send and change information to corresponding database dependence system, and carry out the described alter operation to database object.
Preferably, described database object is tables of data, and described alter operation is data list structure operation, and described database dependence system comprises data warehouse and search system.
Preferably, described attribute information comprises Hostname, port, database-name and data table name.
Preferably, the step that described collection database relies on the dependence of system and database object is,
In data warehouse and search system, extract respectively the dependence record to database object.
Preferably, the dependence of described collection is stored in the Hash table of internal memory.
Preferably, described method also comprises:
Data table name in described attribute information is carried out to format conversion;
According to default hash algorithm, the data table name after format conversion is got to hash value as key, using Hostname, database-name and port as value, with the data structure storage of key-value in Hash table.
Preferably, the alter operation of described parsing to database object, the step of the attribute information of the database object that the request that obtains is changed comprises:
Described alter operation is split as to multiple child-operations, and extract to database object modify, the child-operation of structure modify and deletion;
Utilize regular expression, each child-operation extracting is carried out to canonical coupling, draw the attribute information that each child-operation is corresponding, wherein said Hostname is submitted in submitting described alter operation to.
Preferably, the database name in described attribute information is called master library title, the alter operation of described parsing to database object, and the step of the attribute information of the database object that the request that obtains is changed also comprises:
In database monitoring platform, extract the correspondence table in master library and standby storehouse;
According to described corresponding table and master library title, draw Hostname, port, database-name and data table name that each child-operation is corresponding, wherein, database-name comprises master library title and standby library name.
Preferably, the alter operation of described parsing to database object, the step of the attribute information of the database object that the request that obtains is changed also comprises:
Revise the target character in described alter operation.
Preferably, described collection database relies on the step of the dependence of system and database object and carries out according to predeterminated frequency, and upgrades according to up-to-date dependence of once collecting.
Preferably, described method also comprises:
If not, carry out the described alter operation to database object.
Preferably, in the described dependence of described judgement, the step that whether comprises the attribute information of the database object of request change comprises:
Data table name in the attribute information that extraction parses, according to default hash algorithm, is converted to hash value by described data table name;
In Hash table, the search key identical with described hash value, if exist, judges all the other attribute informations in described alter operation, whether comprises the value that this key is corresponding;
If consistent, judge the attribute information that comprises the database object of asking change in described dependence.
Preferably, described change information comprises the attribute information of the database object of request change, and the link of described alter operation.
The application also provides a kind of change device of database object, comprising:
Dependence collection module, relies on the dependence of system and database object for collecting database, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Alter operation parsing module, for receiving and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Whether judge module, for judging described dependence, comprise the attribute information of the database object of request change, if so, carries out information sending module and change module;
Information sending module, sends and changes information for the database dependence system to corresponding;
Change module, for carrying out the described alter operation to database object.
Compared with prior art, the application has the following advantages:
According to the application, collect in advance the dependence of database dependence system and tables of data, in dependence, comprise the attribute of tables of data, then in the database object alter operation that judgement is submitted to, whether also comprise these information, if comprise, illustrate and can affect dependence system to the retouching operation of this dependence table, can send information to relying on the dependence system that table is corresponding, when data list structure in database is changed, automatic analysis changes the involved dependence system of task, can change the corresponding tables of data in dependence system.
Secondly, the application is kept at the dependence of collection in hash table with the data structure of key-value, take key as mark fast finding is to corresponding tables of data, then contrasts value and can complete analysis, has greatly improved analysis efficiency.
Certainly, arbitrary product of enforcement the application not necessarily needs to reach above-described all advantages simultaneously.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the variation embodiment 1 of a kind of database object of the application;
Fig. 2 is the process flow diagram of the variation embodiment 2 of a kind of database object of the application;
Fig. 3 is the structured flowchart of the change device embodiment 1 of a kind of database object of the application;
Fig. 4 is the structured flowchart of the change device embodiment 2 of a kind of database object of the application.
Embodiment
For the above-mentioned purpose, the feature and advantage that make the application can become apparent more, below in conjunction with the drawings and specific embodiments, the application is described in further detail.
The application proposes a kind of variation of database object, has collected in advance the dependence of database dependence system and database, and business personnel submits the alter operation to database object at altering system, and notification database task management platform; Database tasks management platform is passed through dispatching system, set up also each task of scheduled for executing database, database tasks platform receives after notice, set up analysis task by dispatching system, analysis task can be scheduled according to default scheduling rule, and which dependence system analyzes alter operation according to dependence can exert an influence to.And then can change the corresponding tables of data in dependence system.
With reference to figure 1, the process flow diagram that it shows the variation embodiment 1 of a kind of database object of the application, specifically can comprise the following steps:
In the application, described database can be OLTP(Online Transaction Processing, online issued transaction) database, OLTP database is the background data base of operation system on line, generally has higher concurrent; The application also can be applied to the database of other types, for example, DSS(Decision Suport System, decision support system (DSS)) database.
Database object can be the most basic tables of data and view, and constraint, sequence, function, storing process, bag, trigger etc.If described database object is tables of data, described alter operation is data list structure operation, and described attribute information can comprise Hostname, port, database-name and data table name.
Described database relies on system can comprise data warehouse and search system, all needs to use database object and does data analysis.
Preferably, described step 101 can comprise:
Sub-step S11, in data warehouse and search system, extract respectively the dependence record of database object.
The tables of data of data warehouse and search system depends on database, can be take the data of database as basis, and the unit by database name, database object name, host name etc. as extraction data, so its dependence is also take these as basis.In data warehouse and search system, recorded the database object extracting in database, the time of extracting, the attribute information of database object, can be by the corresponding relation of dependence system and the database object relying on, and the attribute information of described database object arranges out for subsequent use.
In concrete realization, described collection database relies on the step of the dependence of system and database object and can carry out according to predeterminated frequency, and upgrades according to up-to-date dependence of once collecting.The dependence of collecting can be kept in public database.
Database table structure is changed based on business demand, business personnel submits Request for Change operation at altering system, meeting notification database task management platform after altering system reception alter operation, database tasks platform receives the alter operation that altering system sends over, set up analysis task, and scheduled for executing analysis task, alter operation is resolved.
In the time that database object is tables of data, be the DDL statement (Data Description Language, database schema definitional language) that carries out list structure change to the alter operation of database object.The content of alter operation mainly can comprise concrete main frame, port, database name, the data table name of execution; Wherein main frame navigates to a database instance, library name navigates on concrete database, DDL statement is the statement of revising the one or more list structures in correspondence database, and this statement can be carried out normally in a database, after regular expression conversion, can be performed.
In concrete realization, Hostname may be not comprised in DDL statement, but in the time submitting database object alter operation to, together submits to.Concrete, can, in the time that altering system is filled in change, in option " change equipment ", fill in Hostname.
Following two DDL statements: 172.23.100.17:3306sns_01 carries out in storehouse:
DDL statement 1, Create table sns_blogs (id number, name varchar2 (32));
DDL statement 2, Alter table sns_groups add column group_desc varchar2 (32).
While filling in changefree change, extending this as Hostname is A.
Previous DDL statement list is shown in the operation of carrying out establishment table sns_blogs in sns_01 database;
A rear DDL statement list is shown in sns_01 database and modifies and show the operation of sns_groups.
The attribute information that DDL statement is above extracted to the database object that obtains request change is as follows:
DDL statement 1: Hostname 172.23.100.17, port 3306, database name sns_01, table sns_blogs
DDL statement 2: Hostname 172.23.100.17, port 3306, database name sns_01, table sns_groups.
In a preferred embodiment of the present application, described step 102 can comprise:
Sub-step S21, described alter operation is split as to multiple child-operations, and extract to database object modify, the child-operation of structure modify and deletion;
Sub-step S22, utilize regular expression, each child-operation extracting is carried out to canonical coupling, draw the attribute information that each child-operation is corresponding, wherein, described Hostname is submitted in submitting described alter operation to.
Because alter operation may relate to the multiple operations of changing to multiple different database objects, therefore, also need alter operation to be split as multiple child-operations.For tables of data, be the DDL statement that change statement is split as to single operation, then extract wherein database object is modified, the statement of structure modify and deletion, because these statements just can impact dependence system.
Because each DDL statement likely can bring impact to dependence system, so the data that split are out cycle criterion.First judge in each DDL statement the whether operation of alter table type, be exactly that his-and-hers watches will be modified so, then see in this statement whether comprise keyword add, drop or rename, if so, represent that this statement changes list structure really; If this statement is not altertable type, so just see whether statement comprises keyword drop table or turncate, if so, the deletion of showing or data deletion action.
Alter operation generally can include multiple statements, and these statements may operate the multiple tables in the multiple storehouses of multiple main frames, in concrete execution, they is split to the execution of coming, in the following example:
Use db1;
Create table test1(id bigint);
Create table test2(id bigint,name varchar(32));
drop table test3。
According to the difference of operand, above-mentioned alter operation can be split as 4 DDL statements successively, respectively corresponding following operation:
The first step: be switched to the operation of db1 storehouse;
Second step: create table test1, comprise row id type bigint, other attribute acquiescences;
Step 3: create table test2, comprise row id and name, attribute is respectively bigint and varchar, and name length is 32;
Step 4: abandon table test3.
In the process of judgement, first judge in above-mentioned DDL statement to be not the operation of alter table type, therefore can not contain any in keyword add, drop or rename yet; Then judge in above-mentioned DDL statement and comprise keyword drop table, therefore, the deletion action that has comprised his-and-hers watches in above-mentioned DDL statement, can extract this statement of drop table test3, further analyzes.
The application utilizes regular expression to mate for each child-operation splitting out, utilizes canonical coupling effectively DDL statement in enormous quantities to be divided and filtered, and can precisely extract the attribute information in each child-operation.
Database is to have active and standby relation, can think that active and standby structure is the same with data.Alter operation must change for the database object in master library, after master library information is modified, can automatically upgrade standby library information, but because dependence system both can also can information extraction from standby storehouse from master library, not necessarily information extraction from master library, therefore, for the dependence system that relies on standby storehouse, in the attribute information of dependence, also can comprise standby library name, so need further to extract comparatively complete main and standby relation, draw master library and a standby library information that changes corresponding impact.In this application, the corresponding relation in master library and standby storehouse can be kept in database monitoring platform.
Described step 102 can also comprise:
Sub-step S23, in database monitoring platform, extract the correspondence table in master library and standby storehouse;
Sub-step S24, the described corresponding table of foundation and master library title, draw Hostname, port, database-name and data table name that each child-operation is corresponding, and wherein, database-name comprises master library title and standby library name.
According to Hostname corresponding to each child-operation drawing in above-mentioned steps, port, master library title and data table name, and the corresponding relation in master library and standby storehouse in database monitoring platform, can further draw the corresponding relation of Hostname, port, master library title, standby library name and data table name that each child-operation is corresponding.
In concrete realization, described step 102 can also comprise:
Sub-step S25, revise the target character in described alter operation.
In alter operation, may exist some special characters to need to process, using these special characters as target character, before fractionation alter operation is multiple child-operations, if need to analyze and have target character for carrying out in script, need to revise, obtain the comparatively statement of standard.Special character comprises following two kinds of situations:
1, branch
In concrete realization, because multiple DDL statements are submitted to simultaneously, for example, so just have list separator between multiple statement,, SQL(Structured Query Language, Structured Query Language (SQL)) list separator of statement acquiescence be "; ", if itself also exist in DDL statement content "; " so splitting when statement according to separator, just may split mistake.
In structure modify statement (comprising the keywords such as create table, alter table, drop table), if there is branch in statement content, after detecting, can be by this branch, replace with comma or other symbols, make to split the problem of makeing mistakes and solved.
In the following example:
create table abc(
id bigint,
ss_desc varchar(400)comment‘1:XXX;2ddd;DD”D’
)
Branch wherein can be modified to comma:
create table abc(
id bigint,
ss_desc varchar(400)comment‘1:XXX,2ddd;DD”D’
)
For correcting statement (comprising the keywords such as insert, update, delete), the statement of exactly database data content being modified, now can not revise statement content originally, can carry out escape to statement, transfer branch to corresponding ascii code value, the ascii code of for example branch is 59.
For example:
Insert into abc(id,ss_desc)
Values(1,‘abc;ddd’);
If at mysql(associated data base management system) in can be by escape:
Insert into abc(id,ss_desc)
Values(1,concat(‘abc’,char(59),’ddd’));
In oracle database, can be by escape:
Insert into abc(id,ss_desc)
Values(1,‘abc’||chr(59)||’ddd’);
2, other
Because the alter operation of database object can move simultaneously on many machines, middle exist as/* */--the data that this class symbol of # comprises, can be considered to annotation information, these contents are not the contents in statement, mainly conveniently examine and check, and system independence, neither to carry out content by DDL statement, the application can directly do to delete for this type of symbol and process, and carries out content and put together and can make to carry out content and report an error.
Collect the corresponding relation of dependence system with the database object relying on by step 101, and the attribute information of database object; By step 102 obtained alter operation for the attribute information of database object.If there is the related database object of alter operation in dependence, illustrate this alter operation for database object there is corresponding dependence system, so this time change and will exert an influence to dependence system.
Concrete, can judge according to the attribute information of database object, in dependence, comprise the attribute information of the database object that dependence system relies on, if in the dependence of collecting, the attribute information of the database object that has comprised request change, illustrates that this change meeting exerts an influence to dependence system.
The dependence of collecting in the following example:
DB main frame: 172.23.100.17, DB port: 3306, DB library name: sns_01, DB table name: sns_blog_[0000-0255], rely on storehouse: ABC, rely on object table: XXX.
This dependence represents to rely on storehouse ABC and relies on object table XXX, depends on the tables of data sns_blog_[0000-0255 in database sns_01], the DB main frame of this tables of data is that 172.23.100.17, DB port are 3306.
The alter operation of submitting to is as follows:
Carry out in 172.23.100.17:3306sns_01 storehouse: Create table sns_blogs (id number, name varchar2 (32));
Resolve alter operation and show that the attribute information of the database object of asking change is as follows:
Hostname 172.23.100.17, port 3306, database name sns_01, table sns_blogs
The attribute information that parsing is obtained contrasts with dependence, can find Hostname, port, database name and table name, is all included in above-mentioned dependence.
Owing to having preserved the corresponding relation of dependence system and its database object relying in dependence, therefore, can send information to corresponding dependence system.Concrete, described change information can comprise the attribute information of the database object of request change, and the link of described alter operation, dependence system receives and changes after information, can be according to the attribute information of database object, find corresponding database object, and according to the link of alter operation, carry out corresponding alter operation.
With reference to figure 2, the process flow diagram that it shows the variation embodiment 2 of a kind of database object of the application, specifically can comprise the following steps:
Step 201, the dependence of collecting database and rely on system and database object, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object.
Step 202, the data table name in described attribute information is carried out to format conversion.
In database, there is a large amount of point storehouse submeters, each table can be split as several thousand tables, no matter so writing DDL script or during at configuration data, if every day, statement did not process, will become several thousand statements, it is all very high writing like this cost and evaluating cost, because the structure of these submeters is all to touch the same (only having title suffix to change), thus in the time configuring and Request for Change writes, can adopt some simple and easy literary styles, for example:
Create table user_info_[0000-1023](.....)
So just can represent the establishment of 1024 tables, reviewer can be very clear, also have to write one by one, and great-jump-forward as:
Create table user_info_[0,3,5,7,9](.....)
Etc. literary style differ, but finally take while execution in database, corresponding database-name can and database in table name be consistent.Therefore, need to further be converted to the form that can carry out the real data of Rapid matching.
For example,, to Create table test_[0000-0255] data table name in (id bigint) carries out can obtaining 256 statements after format conversion:
Create table test_0000(id bigint);
Create table test_0001(id bigint);
Create table test_0002(id bigint);
Create table test_0255(id bigint)。
No matter what format write is, after conversion, being all converted to a kind of unified standard format leaves in internal memory, in judging that whether attribute information is consistent, just need not consider it is what format write, be converted to same form, no matter be that submeter or single table can be mated accurately, greatly improve the efficiency of analyzing.
Step 203, according to default hash algorithm, the data table name after format conversion is got to hash value as key, using Hostname, database-name and port as value, with the data structure storage of key-value in Hash table.
Hash table (Hashmap) is a kind of structured data repository, possesses the feature of efficiently searching.For example, if each article have a number designation ID, since 1 automatic increase, now, search the content information of article according to the ID of article, if mate one by one since 1, if data volume is larger, matching process will consume performance very slowly and extremely.Hash algorithm carries out hash by these ID exactly, for example, take array 10 as radix, now ID is that 1,11,21,31 etc. data will be placed in first array index, Hashmap inside is generally to use storage of linked list, can usable range store the location of refinement more if just will further optimize.
In the application, the dependence of collection can be stored in the Hash table (HashMap) of internal memory.Improve the performance of coupling with predefined internal memory hash, all matching processs all do not rely on file system and database itself, and this has improved the speed of coupling greatly.
Data in HashMap are deposited with the data structure of Key-Value, and wherein, Key is by default hash algorithm, and the data table name after format transformation is calculated.
For example, the ascii value of character string is spliced, get up by the numbers pile up of a string character string of each table name, then carry out delivery with the basic array size of a preset Hash, obtain the subscript of stored Hash array (being Hash bucket).For example: Hash bucket number is 100, numbering is from 0-99, and Any Digit and 100 deliverys just can obtain the numeral between 0-99 so, and corresponding is exactly the numbering of Hash bucket, seek scope directly can be dwindled to 100 times.And along with data volume increases, each scope of dwindling of calculating also can and increase along with the number increase of bucket, namely can be along with data volume increases.
The dependence of following example illustrates the process of calculating key according to data table name:
DB main frame: 172.23.100.17, DB port: 3306, DB library name: sns_01, DB table name: abc_table, dependence storehouse: ABC, dependence object table: XXX.
Table name comprises 12 characters, and the ascii list of corresponding each character is:
| Character | a | b | c | - | t | a | b | l | e |
| Ascii code | 97 | 98 | 99 | 95 | 116 | 97 | 98 | 108 | 101 |
Stacked up as key=97+98+99+95+116+97+98+108+10=909.If now Hash array has 10, obtain being down designated as under 9(by 909%10 so and be designated as 0-9), so now just under be designated as in 9 Hash bucket and can find corresponding value value, comprise DB main frame 172.23.100.17, DB port 3306, DB library name sns_01.
Using Hostname, database-name and port as value, the value in upper example comprises DB main frame 172.23.100.17, DB port 3306, DB library name sns_01.In concrete realization, in dependence, can also comprise and rely on the recalls information of storehouse to tables of data, for example, to being the last service time of relied on tables of data that 2012-12-21, affiliated type are that DW|SEARCH, wheel synchronization type are the information such as datax, dbsync, drc, these also can be used as a part of value.
Step 204, receive and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Step 205, judge in described dependence whether to comprise the attribute information of the database object that request changes, if so, perform step 206 and 207, if not, perform step 207.
Step 206, send and change information to corresponding database dependence system;
Step 207, carry out the described alter operation to database object.
In a preferred embodiment of the present application, described step 103 can comprise:
Sub-step S31, for extracting the data table name of the attribute information parsing, according to default hash algorithm, described data table name is converted to hash value;
Sub-step S32, in Hash table the search key identical with described hash value, if exist, judge all the other attribute informations in described alter operation, whether comprise the value that this key is corresponding, if consistent, judge the attribute information that comprises the database object of asking change in described dependence.
In the time of the attribute information contrasting in dependence and the alter operation of collecting, need in dependence, search same data, utilize the application, use same algorithm to get hash value to data table name in attribute information, just can obtain fast below which array index, and then dwindle very large scope, whether unanimously then directly contrast value, seek rate is very fast, and then can improve the efficiency of analysis.
HashMap is preset in advance, put in internal memory, key correspondence the table likely affecting, because the renewal of dependence is not frequent especially, thus the little change of this internal storage structure, therefore, as long as can find in Hashmap, be influential, search less than be do not have influential.According to the application, alter operation is analyzed, average each analysis on altered project time can remain in 30ms.
The java language that the corresponding program of the application can be used machine word to call the turn, need to coordinate java virtual machine environment, OS and hardware environment do not need too many requirement, only need to this main frame support be installed by java virtual machine, program is owing to using java to write, so can accomplish cross-platform analysis.
In sum, according to the application, collect in advance the dependence of database dependence system and tables of data, in dependence, comprise the attribute of tables of data, then in the database object alter operation that judgement is submitted to, whether also comprise these information, if comprise, illustrate and can affect dependence system to the retouching operation of this dependence table, can send information to relying on the dependence system that table is corresponding, when data list structure in database is changed, automatic analysis changes the involved dependence system of task, can change the corresponding tables of data in dependence system.
Secondly, the application is kept at the dependence of collection in hash table with the data structure of key-value, take key as mark fast finding is to corresponding tables of data, then contrasts value and can complete analysis, has greatly improved analysis efficiency.
For embodiment of the method, for simple description, therefore it is all expressed as to a series of combination of actions, but those skilled in the art should know, the application is not subject to the restriction of described sequence of movement, because according to the application, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in instructions all belongs to preferred embodiment, and related action and module might not be that the application is necessary.
With reference to figure 3, it shows the structured flowchart of the change device embodiment 1 of a kind of database object of the application, specifically can comprise with lower module:
Dependence collection module 301, relies on the dependence of system and database object for collecting database, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Alter operation parsing module 302, for receiving and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Whether judge module 303, for judging described dependence, comprise the attribute information of the database object of request change, if so, carries out information sending module and change module;
Information sending module 304, sends and changes information for the database dependence system to corresponding;
Change module 305, for carrying out the described alter operation to database object.
In concrete realization, altering system and data base administration platform can be positioned at same equipment or share same server, and altering system can be stored in this locality by content, is then analyzed by database tasks management platform; Because altering system and database tasks management platform are separate, therefore, also can submit the alter operation to database object at long-range altering system, altering system notification database task management platform requires to analyze, database tasks management platform is carried out reading of long-range DDL statement, concrete, can use and in HttpClient, read changed content; If DDL phrase data amount is larger, the mode that can adopt annex to upload, the content of carrying out remote accessory with URLConnection reads.
In a preferred embodiment of the present application, described database object can be tables of data, and described alter operation is data list structure operation, and described database relies on system can comprise data warehouse and search system.Described attribute information can comprise Hostname, port, database-name and data table name.
In a preferred embodiment of the present application, described dependence collection module can comprise:
Dependence record extracts submodule, in data warehouse and search system, extracts respectively the dependence record to database object.
The dependence of described collection can be stored in the Hash table of internal memory.
In a preferred embodiment of the present application, described alter operation parsing module can comprise:
Split submodule, for described alter operation is split as to multiple child-operations, and extract to database object modify, the child-operation of structure modify and deletion;
Canonical matched sub-block, for utilizing regular expression, carries out canonical coupling to each child-operation extracting, and draws the attribute information that each child-operation is corresponding, and wherein said Hostname is submitted in submitting described alter operation to.
In a preferred embodiment of the present application, the database name in described attribute information is called master library title, and described parsing module can also comprise:
Active and standby corresponding table extraction module, for extracting the correspondence table in master library and standby storehouse at database monitoring platform;
Submodule is determined in active and standby storehouse, for according to described corresponding table and master library title, draws Hostname, port, database-name and data table name that each child-operation is corresponding, and wherein, database-name comprises master library title and standby library name.
In a preferred embodiment of the present application, described alter operation parsing module can also comprise:
Target character correcting module, for revising the target character of described alter operation.
In concrete realization, described dependence collection module can be carried out according to predeterminated frequency.
With reference to figure 4, it shows the structured flowchart of the change device embodiment 2 of a kind of database object of the application, specifically can comprise with lower module:
Dependence collection module 401, relies on the dependence of system and database object for collecting database, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Format converting module 402, carries out format conversion for the data table name to described attribute information;
Memory module 403, for according to default hash algorithm, gets hash value as key to the data table name after format conversion, using Hostname, database-name and port as value, with the data structure storage of key-value in Hash table.
Alter operation parsing module 404, for receiving and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Whether judge module 405, for judging described dependence, comprise the attribute information of the database object of request change, if so, carries out information sending module 406 and change module 407, if not, carries out and change module 407;
Information sending module 406, sends and changes information for the database dependence system to corresponding;
Change module 407, for carrying out the described alter operation to database object.
In a preferred embodiment of the present application, described judge module 405 can comprise:
Hash changes submodule, for extracting the data table name of the attribute information parsing, according to default hash algorithm, described data table name is converted to hash value;
Search submodule, for searching for the key identical with described hash value at Hash table, if exist, judge all the other attribute informations in described alter operation, whether comprise the value that this key is corresponding, if consistent, judge the attribute information that comprises the database object of asking change in described dependence.
In a preferred embodiment of the present application, described change information can comprise the attribute information of the database object of request change, and the link of described alter operation.
Because described device embodiment is substantially corresponding to the embodiment of the method shown in earlier figures 1 and Fig. 2, therefore not detailed part in the description of the present embodiment can, referring to the related description in previous embodiment, just not repeat at this.
The application can be used in numerous general or special purpose computingasystem environment or configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multicomputer system, system, set top box, programmable consumer-elcetronics devices, network PC, small-size computer, mainframe computer based on microprocessor, comprise distributed computing environment of above any system or equipment etc.
The application can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises and carries out particular task or realize routine, program, object, assembly, data structure of particular abstract data type etc.Also can in distributed computing environment, put into practice the application, in these distributed computing environment, be executed the task by the teleprocessing equipment being connected by communication network.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium including memory device.
In this article, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby the process, method, article or the equipment that make to comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or be also included as the intrinsic key element of this process, method, article or equipment.The in the situation that of more restrictions not, the key element being limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
The variation of a kind of database object above the application being provided, and, a kind of change device of database object is described in detail, applied principle and the embodiment of specific case to the application herein and set forth, the explanation of above embodiment is just for helping to understand the application's method and core concept thereof; , for one of ordinary skill in the art, according to the application's thought, all will change in specific embodiments and applications, in sum, this description should not be construed as the restriction to the application meanwhile.
Claims (14)
1. a variation for database object, is characterized in that, comprising:
Collect database and rely on the dependence of system and database object, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Receive and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Judge in described dependence whether comprise the attribute information of the database object of request change;
If so, send and change information to corresponding database dependence system, and carry out the described alter operation to database object.
2. the method for claim 1, is characterized in that, described database object is tables of data, and described alter operation is data list structure operation, and described database dependence system comprises data warehouse and search system.
3. method as claimed in claim 2, is characterized in that, described attribute information comprises Hostname, port, database-name and data table name.
4. method as claimed in claim 3, is characterized in that, the step that described collection database relies on the dependence of system and database object is,
In data warehouse and search system, extract respectively the dependence record to database object.
5. method as claimed in claim 4, is characterized in that, the dependence of described collection is stored in the Hash table of internal memory.
6. method as claimed in claim 5, is characterized in that, also comprises:
Data table name in described attribute information is carried out to format conversion;
According to default hash algorithm, the data table name after format conversion is got to hash value as key, using Hostname, database-name and port as value, with the data structure storage of key-value in Hash table.
7. method as claimed in claim 3, is characterized in that, the alter operation of described parsing to database object, and the step of the attribute information of the database object that the request that obtains is changed comprises:
Described alter operation is split as to multiple child-operations, and extract to database object modify, the child-operation of structure modify and deletion;
Utilize regular expression, each child-operation extracting is carried out to canonical coupling, draw the attribute information that each child-operation is corresponding, wherein said Hostname is submitted in submitting described alter operation to.
8. method as claimed in claim 7, is characterized in that, the database name in described attribute information is called master library title, the alter operation of described parsing to database object, and the step of the attribute information of the database object that the request that obtains is changed also comprises:
In database monitoring platform, extract the correspondence table in master library and standby storehouse;
According to described corresponding table and master library title, draw Hostname, port, database-name and data table name that each child-operation is corresponding, wherein, database-name comprises master library title and standby library name.
9. method as claimed in claim 7 or 8, is characterized in that, the alter operation of described parsing to database object, and the step of the attribute information of the database object that the request that obtains is changed also comprises:
Revise the target character in described alter operation.
10. the method for claim 1, is characterized in that, described collection database relies on the step of the dependence of system and database object and carries out according to predeterminated frequency, and upgrades according to up-to-date dependence of once collecting.
11. the method for claim 1, is characterized in that, also comprise:
If not, carry out the described alter operation to database object.
12. methods as claimed in claim 6, is characterized in that, in the described dependence of described judgement, the step that whether comprises the attribute information of the database object of request change comprises:
Data table name in the attribute information that extraction parses, according to default hash algorithm, is converted to hash value by described data table name;
In Hash table, the search key identical with described hash value, if exist, judges all the other attribute informations in described alter operation, whether comprises the value that this key is corresponding;
If consistent, judge the attribute information that comprises the database object of asking change in described dependence.
13. the method for claim 1, is characterized in that, described change information comprises the attribute information of the database object of request change, and the link of described alter operation.
The change device of 14. 1 kinds of database objects, is characterized in that, comprising:
Dependence collection module, relies on the dependence of system and database object for collecting database, described dependence comprises dependence system and the corresponding relation of the database object that relies on, and the attribute information of described database object;
Alter operation parsing module, for receiving and resolve the alter operation to database object, the attribute information of the database object that the request that obtains is changed;
Whether judge module, for judging described dependence, comprise the attribute information of the database object of request change, if so, carries out information sending module and change module;
Information sending module, sends and changes information for the database dependence system to corresponding;
Change module, for carrying out the described alter operation to database object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210398278.XA CN103778133A (en) | 2012-10-18 | 2012-10-18 | Database object changing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210398278.XA CN103778133A (en) | 2012-10-18 | 2012-10-18 | Database object changing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103778133A true CN103778133A (en) | 2014-05-07 |
Family
ID=50570377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210398278.XA Pending CN103778133A (en) | 2012-10-18 | 2012-10-18 | Database object changing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103778133A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112016A (en) * | 2014-07-18 | 2014-10-22 | 北京京东尚科信息技术有限公司 | Method for changing data sheet structure |
| CN104699811A (en) * | 2015-03-23 | 2015-06-10 | 重庆广播电视大学 | Method and device for recording and backtracking variation of form data in non-invasion manner |
| WO2016192583A1 (en) * | 2015-06-04 | 2016-12-08 | 阿里巴巴集团控股有限公司 | Data processing method and device for data warehouse |
| CN107180053A (en) * | 2016-03-11 | 2017-09-19 | 中国移动通信集团河北有限公司 | A kind of data warehouse optimization method and device |
| CN107357910A (en) * | 2017-07-18 | 2017-11-17 | 郑州云海信息技术有限公司 | A kind of method and device of data comparison |
| CN108415998A (en) * | 2018-02-24 | 2018-08-17 | 平安科技（深圳）有限公司 | Using dependence update method, terminal, equipment and storage medium |
| CN109697068A (en) * | 2017-10-24 | 2019-04-30 | 中移(苏州)软件技术有限公司 | One kind dividing logic SQL statement interpretation method and device under the table mode of library |
| CN110262803A (en) * | 2019-06-30 | 2019-09-20 | 潍柴动力股份有限公司 | A kind of generation method and device of dependence |
| CN110879801A (en) * | 2019-11-21 | 2020-03-13 | 中国银行股份有限公司 | Data reading method and device |
| CN110968569A (en) * | 2019-12-19 | 2020-04-07 | 广州品唯软件有限公司 | Database management method, database management device, and storage medium |
| CN111104409A (en) * | 2019-12-25 | 2020-05-05 | 上海米哈游天命科技有限公司 | Database processing method and device, storage medium and electronic equipment |
| CN111651431A (en) * | 2020-06-09 | 2020-09-11 | 中国雄安集团数字城市科技有限公司 | Database service oriented management flow standardization method |
| CN112016982A (en) * | 2020-08-19 | 2020-12-01 | 云账户技术(天津)有限公司 | Billing amount calculation method and device |
| CN112115125A (en) * | 2020-09-27 | 2020-12-22 | 北京人大金仓信息技术股份有限公司 | Database access object name resolution method and device and electronic equipment |
| CN112347202A (en) * | 2020-11-27 | 2021-02-09 | 中国工商银行股份有限公司 | Method and device for notifying data change message |
-
2012
- 2012-10-18 CN CN201210398278.XA patent/CN103778133A/en active Pending
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112016B (en) * | 2014-07-18 | 2017-10-03 | 北京京东尚科信息技术有限公司 | The method for changing data list structure |
| CN104112016A (en) * | 2014-07-18 | 2014-10-22 | 北京京东尚科信息技术有限公司 | Method for changing data sheet structure |
| CN104699811A (en) * | 2015-03-23 | 2015-06-10 | 重庆广播电视大学 | Method and device for recording and backtracking variation of form data in non-invasion manner |
| CN104699811B (en) * | 2015-03-23 | 2018-02-23 | 重庆广播电视大学 | A kind of no intrusive mood form data change record and the method and device of backtracking |
| CN106294478B (en) * | 2015-06-04 | 2019-11-08 | 阿里巴巴集团控股有限公司 | The data processing method and device of data warehouse |
| WO2016192583A1 (en) * | 2015-06-04 | 2016-12-08 | 阿里巴巴集团控股有限公司 | Data processing method and device for data warehouse |
| CN106294478A (en) * | 2015-06-04 | 2017-01-04 | 阿里巴巴集团控股有限公司 | The data processing method of data warehouse and device |
| CN107180053A (en) * | 2016-03-11 | 2017-09-19 | 中国移动通信集团河北有限公司 | A kind of data warehouse optimization method and device |
| CN107180053B (en) * | 2016-03-11 | 2020-10-20 | 中国移动通信集团河北有限公司 | Data warehouse optimization method and device |
| CN107357910A (en) * | 2017-07-18 | 2017-11-17 | 郑州云海信息技术有限公司 | A kind of method and device of data comparison |
| CN109697068A (en) * | 2017-10-24 | 2019-04-30 | 中移(苏州)软件技术有限公司 | One kind dividing logic SQL statement interpretation method and device under the table mode of library |
| WO2019161620A1 (en) * | 2018-02-24 | 2019-08-29 | 平安科技（深圳）有限公司 | Application dependency update method, terminal and device, and storage medium |
| CN108415998A (en) * | 2018-02-24 | 2018-08-17 | 平安科技（深圳）有限公司 | Using dependence update method, terminal, equipment and storage medium |
| CN110262803B (en) * | 2019-06-30 | 2023-04-18 | 潍柴动力股份有限公司 | Method and device for generating dependency relationship |
| CN110262803A (en) * | 2019-06-30 | 2019-09-20 | 潍柴动力股份有限公司 | A kind of generation method and device of dependence |
| CN110879801A (en) * | 2019-11-21 | 2020-03-13 | 中国银行股份有限公司 | Data reading method and device |
| CN110968569A (en) * | 2019-12-19 | 2020-04-07 | 广州品唯软件有限公司 | Database management method, database management device, and storage medium |
| CN110968569B (en) * | 2019-12-19 | 2023-12-29 | 广州品唯软件有限公司 | Database management method, database management device, and storage medium |
| CN111104409A (en) * | 2019-12-25 | 2020-05-05 | 上海米哈游天命科技有限公司 | Database processing method and device, storage medium and electronic equipment |
| CN111651431A (en) * | 2020-06-09 | 2020-09-11 | 中国雄安集团数字城市科技有限公司 | Database service oriented management flow standardization method |
| CN112016982A (en) * | 2020-08-19 | 2020-12-01 | 云账户技术(天津)有限公司 | Billing amount calculation method and device |
| CN112115125A (en) * | 2020-09-27 | 2020-12-22 | 北京人大金仓信息技术股份有限公司 | Database access object name resolution method and device and electronic equipment |
| CN112115125B (en) * | 2020-09-27 | 2024-04-26 | 北京人大金仓信息技术股份有限公司 | Database access object name resolution method and device and electronic equipment |
| CN112347202A (en) * | 2020-11-27 | 2021-02-09 | 中国工商银行股份有限公司 | Method and device for notifying data change message |
| CN112347202B (en) * | 2020-11-27 | 2023-12-29 | 中国工商银行股份有限公司 | Method and device for notifying data change message |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103778133A (en) | Database object changing method and device | |
| CN107451225B (en) | Scalable analytics platform for semi-structured data | |
| US10983967B2 (en) | Creation of a cumulative schema based on an inferred schema and statistics | |
| CN108874971B (en) | Tool and method applied to mass tagged entity data storage | |
| CN106897322B (en) | A kind of access method and device of database and file system | |
| US10528540B2 (en) | Dynamic aggregate generation and updating for high performance querying of large datasets | |
| US10565208B2 (en) | Analyzing multiple data streams as a single data object | |
| US10268645B2 (en) | In-database provisioning of data | |
| CN111767303A (en) | Data query method and device, server and readable storage medium | |
| CN107408114B (en) | Identifying join relationships based on transactional access patterns | |
| CN104881424A (en) | Regular expression-based acquisition, storage and analysis method of power big data | |
| CN106708993A (en) | Spatial data storage processing middleware framework realization method based on big data technology | |
| CN104866580A (en) | Method for quickly detecting impact caused by database modification to current service | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| CN103970902A (en) | Method and system for reliable and instant retrieval on situation of large quantities of data | |
| CN114116716A (en) | Hierarchical data retrieval method, device and equipment | |
| CN105493028A (en) | Data consistency and rollback for cloud analytics | |
| Jewell et al. | Performance and capacity implications for big data | |
| CN106055678A (en) | Hadoop-based panoramic big data distributed storage method | |
| US20190311051A1 (en) | Virtual columns to expose row specific details for query execution in column store databases | |
| CN104714974A (en) | Method and device for parsing and reprocessing query statement | |
| CN114297173A (en) | Knowledge graph construction method and system for large-scale mass data | |
| CN113821573A (en) | Mass data rapid retrieval service construction method, system, terminal and storage medium | |
| KR101955376B1 (en) | Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method | |
| CN111125045B (en) | Lightweight ETL processing platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140507 |