CN110019118A - The system and method for database O&M efficiency are promoted based on DevOps - Google Patents

The system and method for database O&M efficiency are promoted based on DevOps Download PDF

Info

Publication number
CN110019118A
CN110019118A CN201710901776.4A CN201710901776A CN110019118A CN 110019118 A CN110019118 A CN 110019118A CN 201710901776 A CN201710901776 A CN 201710901776A CN 110019118 A CN110019118 A CN 110019118A
Authority
CN
China
Prior art keywords
node
database
mysql
devops
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710901776.4A
Other languages
Chinese (zh)
Inventor
刘海忠
何广柏
张伟
刘亚光
曾培植
杨超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Eshore Technology Co Ltd
Original Assignee
Guangdong Eshore Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Eshore Technology Co Ltd filed Critical Guangdong Eshore Technology Co Ltd
Priority to CN201710901776.4A priority Critical patent/CN110019118A/en
Publication of CN110019118A publication Critical patent/CN110019118A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/248Presentation of query results

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention relates to the system and method that database O&M efficiency is promoted based on DevOps, which includes the node application unit for tenant's application node example;Monitoring nodes unit is written to MySQL, reads monitoring data and graphical representation from MySQL using jqPlot chart for acquiring the monitoring data of monitored node in the node application unit;DDL audits unit, for carrying out the analysis of syntax and semantics, being fed back based on the analysis results according to the DDL and DML statement that need node to be changed and input is chosen in node application unit, generates flashback sentence to all operations.The present invention improves database administrator productivity, database service is more efficient, reliable and stable, it is also beneficial to batch operation and upgrading, more efficient inter-department cooperation, promote team collaboration's efficiency, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.

Description

The system and method for database O&M efficiency are promoted based on DevOps
Technical field
The present invention relates to enterprise database operation platforms, more specifically refer to and promote database O&M effect based on DevOps The system and method for rate.
Background technique
In Project Running Process, database performance is beneath, Frequent Troubles are configured often from operating system and database Item missing, table structure, SQL design are lack of standardization;Also because permission is chaotic, sentence writes the reasons such as fault leads to database maloperation Constantly.Therefore, to ensure that database stability, performance maximize, probability of malfunction is reduced, database administrator needs were being researched and developed Following item is completed in unified control in journey, support: database service deployment, DDL manual examination and verification publication, DML maloperation are corrected, slowly Query optimization, optimiged index, rights management, troubleshooting ... arrive when the database varying duty of projects is parallel, a side Face data library manager link can become the bottleneck of whole flow process;On the other hand it requires to abide by from audit is deployed to due to database Standardization is followed, between the complexity of sentence and environment, manual examination and verification and operation are difficult to ensure high-accuracy.
Generally there are artificial change and semi-automatic change mode for the change work in business data at present, wherein such as Shown in Fig. 1, for manually change method, all change work is completed by manually walking process, the friendship of this method Stream mode is usually mail, and when needing to be implemented change, DEV/QA writes mail and initiates to change to database administrator, in data depositary management Reason person receives and probably completes following process: business request for data base resource later, after database administrator application hardware resource, hand It is dynamic to dispose and access various monitoring, high-availability system, standby system one by one;Research staff applies for that access resource, personnel change, number Server change permission is frequently logged on according to library manager personnel;Developer submits databases comparison to require, database administrator Advise after audit, the process of walking again for not meeting specification rewrites;For satisfactory, the direct line of database administrator Upper execution, but manual examination and verification are easy error, mistake are just found after online practice, and necessary reply is to exploitation.For semi-automatic For change, as shown in Fig. 2, assisting individual artificial lengthy and tedious affairs using Open-Source Tools or script, exchange way is still postal Part or flow system, probably there is a following process: business request for data base resource, after database administrator application hardware resource, step on Record server operation automatized script deployment;There is a powerful review procedure druid resolver, process is still and manual examination and verification Equally;Database administrator still needs login service device to change permission;The work of server disposition plan target, it is automatic daily to receive Slow SQL on line concentration, hair mail optimize to each developer after analysis summary, developer to log in test environment check index with Executive plan.
But it for artificial change mode, has the following problems: the daily repeated work of database administrator, especially It in DDL audit, execution and link, recycles and error-prone, 100% identification not can guarantee for syntax error, semantic error naked eyes, It just to practice and just find on line;For defined rule, there is loose unevenness in every database administrator, most Cause rule not at rule eventually, is difficult to carry out down;Before execution, database administrator will check environment on line, consider language Stability on sentence pair line has an impact, and tradeoff executes the time;After execution, it is also necessary to notify DEV, QA department to verify, data depositary management Reason person could complete the change, waste many time costs.It for semi-automatic change mode, has the following problems: every time Change, database administrator still want login service device remote batch to operate;Druid resolver still remains that there may be do not have The loophole audited brings very big risk to database on line;Review procedure first has to execute once in test environment, If it find that can just go to execute on line after out of question, then this test just needs moment and synchronous, the data of environment holding on line It needs equally, it has to be ensured that identical, execute just in Beta has referring to property in this way the metadata of database and table.One frequently The node of change, test environment need environment on continuous line synchro, and cost is very high;Since backup export data and execution are not Inside same affairs, not can guarantee the accuracy of Backup Data, and after change on line data covering, can not rollback, this The backup of sample has little significance.In conclusion either artificial change or semi-automatic change, there are development difficulty height and O&Ms The high problem of complexity.
Therefore, it is necessary to design a kind of system for promoting database O&M efficiency based on DevOps, realizing reduces exploitation hardly possible Degree and O&M difficulty ensure lasting, efficient, the stable output service of database.
Summary of the invention
It is an object of the invention to overcome the deficiencies of existing technologies, provides and database O&M efficiency is promoted based on DevOps System and method.
To achieve the above object, the invention adopts the following technical scheme: promoting database O&M efficiency based on DevOps System, the system comprises:
Node application unit, for for tenant's application node example;
Monitoring nodes unit is written to for acquiring the monitoring data of the monitored node in the node application unit MySQL reads monitoring data and graphical representation from MySQL using jqPlot chart;
DDL audits unit, for according to chosen in node application unit the DDL for needing node to be changed and input and DML statement carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback language to all operations Sentence.
Its further technical solution are as follows: the node application unit includes node details input module, example generation module And installation module;
The node details input module, for inputting applied all node instance information for tenant;
The example generation module, the resource of information for being inputted according to the node details input module from MySQL Qualified example is read in the table of pond, change example state is deployable state;
The installation module for installing to the example that tenant applies, and is added to the monitoring nodes unit, Example state is changed as use state.
Its further technical solution are as follows: the system also includes slow log collection units, for analyzing slow SQL statement, and Inverted sequence is carried out to show.
Its further technical solution are as follows: the system also includes account management units, the company created for managing tenant Connect the application account of MySQL.
Its further technical solution are as follows: the system also includes SQL optimiged index units, for optimizing for used in tenant SQL index.
The present invention also provides the methods for promoting database O&M efficiency based on DevOps, which comprises
Good installable node is planned in the resource pool table of MySQL;
Input node application condition;
Node is transferred from the resource pool table of MySQL according to application condition, forms qualified example, by the example shape State is set as in deployable state;
The example is installed;
The example is added in monitoring nodes, and sets use state for example state;
Monitoring nodes are carried out to example;
According to the need node to be changed in node application condition and the DDL and DML statement of input, grammer and language are carried out Justice is analyzed, and is fed back based on the analysis results;
Flashback sentence is generated to all operations.
Its further technical solution are as follows: the step of monitoring nodes are carried out to example, comprising the following specific steps
Obtain the monitoring data of monitored node in example;
The monitoring data is written in MySQL;
Monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
Its further technical solution are as follows: according to the need node to be changed in node application condition and the DDL of input and DML statement carries out the step of syntax and semantics are analyzed, fed back based on the analysis results, comprising the following specific steps
Obtain example domestic demand node, DDL and DML statement to be changed;
Obtain customized configuration rule, using interior band syntax analyzer DDL and DML statement can be carried out syntactic analysis and Semantic analysis;
Whether discriminatory analysis result is to meet auditing standards;
If so, selection corresponding node, executes corresponding change, and enters and flashback sentence is generated to all operations Step;
If it is not, then returning to error information.
Compared with the prior art, the invention has the advantages that: it is of the invention that database O&M efficiency is promoted based on DevOps System, unit is audited by setting node application unit, monitoring nodes unit and DDL, to node instance it is automatic apply, Monitoring nodes and the audit for changing node are freed from database administrator, form the automation for mixing multiple Open-Source Tools System improves database administrator productivity, and database service is more efficient, reliable and stable, is also beneficial to batch operation and liter Grade, more efficient inter-department cooperation promote team collaboration's efficiency, without frittering away energy on flow system or e-mail communications, It is easily accomplished the work of the intersection of Ops and Dev, QA department, realizing reduces development difficulty and O&M difficulty, ensures database Lasting, efficient, stable output service.
The invention will be further described in the following with reference to the drawings and specific embodiments.
Detailed description of the invention
Fig. 1 is the flow chart of the artificial change mode of the prior art;
Fig. 2 is the flow chart of the semi-automatic change mode of the prior art;
Fig. 3 is the structural frames for the system that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides Figure;
Fig. 4 is the structural block diagram for the node application unit that the specific embodiment of the invention provides;
Fig. 5 is the functional block for the system that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides Frame figure;
Fig. 6 is the process for the method that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides Figure;
Fig. 7 is the flow chart that monitoring nodes are carried out to example that the specific embodiment of the invention provides;
Fig. 8 be the specific embodiment of the invention provide according in node application condition need node to be changed and input DDL and DML statement carry out the flow chart analyzed of syntax and semantics.
Specific embodiment
In order to more fully understand technology contents of the invention, combined with specific embodiments below to technical solution of the present invention into One step introduction and explanation, but not limited to this.
The specific embodiment as shown in Fig. 3~8, it is provided in this embodiment that database O&M efficiency is promoted based on DevOps System can be used in and open database service and automatically access monitoring and carry out all daily changes and subsequent operation operation During, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
DevOps (Development Operations) is the general designation of one group of process, method and system, is opened for promoting Communication, cooperation and integration between hair, technology operation and quality assurance (QA) department.Under traditional On-Premise mode, respectively The working condition of departmental staff is often interrupted to support the work to cross, this is unfavorable for the work for being absorbed in oneself, if by this A little regularized operations are simultaneously encapsulated, and provide unified operation interface to Dev, QA, autonomous solve the problems, such as may be implemented completely.
As shown in figure 3, present embodiments providing the system for promoting database O&M efficiency based on DevOps, the system packet It includes:
Node application unit 1, for for tenant's application node example;
Monitoring nodes unit 2 is written for acquiring the monitoring data of the monitored node in the node application unit 1 To MySQL, monitoring data and graphical representation are read from MySQL using jqPlot chart;
DDL audits unit 3, for according to chosen in node application unit 1 DDL for needing node to be changed and input and DML statement carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback language to all operations Sentence.
Specifically, the above-mentioned system for promoting database O&M efficiency based on DevOps is specifically to utilize ansible+ Python+php mixes the Open-Source Tools such as inception, sqladvisor, percona toolkit, jqPlot, accomplishes to be based on The creation of MySQL resource pool and the procedure of monitoring are handled, and self-service operation largely saves process spending.
According to the condition for tenant's application node example that node application unit 1 is got, the calling for carrying out node forms reality Example, and monitoring nodes unit 2 is added, if enumerating change node in condition, is then analyzed, returned by DDL audit unit 3 Return change as a result, and will operation generate flashback sentence, be used for rollback;And monitoring nodes unit 2 is then responsible for carrying out node Monitoring data is obtained, and is presented to tenant, improves research and development, database administrator efficiency, ensures that database is lasting, efficient, steady Fixed output service.
In certain embodiments, above-mentioned node application unit 1 includes node details input module 11, example generation module 12 and installation module 13.
Node details input module 11, for inputting applied all node instance information for tenant.
Example generation module 12, the resource of information for being inputted according to the node details input module 11 from MySQL Qualified example is read in the table of pond, change example state is deployable state.
Module 13 is installed, for being installed to the example that tenant applies, and is added to the monitoring nodes unit 2, Example state is changed as use state.
For above-mentioned node details input module 11, essential information, configuration information and availability information have been specifically included, Wherein essential information includes nodename, place region, intranet and extranet ip address port and creation time;Configuration information includes version The hardware informations such as sheet, CPU/ memory/resources usage amount;Availability information includes copy mode and deployment side;Section can be passed through Whether the point input of details input module 11 changes node or selection monitoring node etc., autonomous to select, and system is according to the choosing of tenant It selects and carries out the generation of corresponding node example, and feed back accordingly as a result, realizing and Dev, QA, Ops are put into same system exploitation, note The self-service completion of weight department's meet work, provides a kind of quickly and effectively mode, avoids unnecessary inter-sectional process, Shortening business on-line time, and ensured the market competitiveness of business, it is more suitable for internet+epoch.
For above-mentioned example generation module 12 and installation module 13, it is mainly used for the process of automatic application node example In, the generation of corresponding node example is carried out to the node application condition (i.e. node instance information) of tenant's input, on condition that DB is needed Good installable MySQL node is planned inside MySQL resource pool table in advance, in order to according to the application condition of tenant at any time It calls node and installs, form node instance, reach self-service operation, largely save the effect of process spending.
For above-mentioned monitoring nodes unit 2, specifically from health monitoring, monitoring resource, the monitoring of key caching, Innodb prison Many-sided monitoring for carrying out node instance such as control, duplication monitoring and table space analysis, wherein health monitoring, which is mainly to look at, works as Preceding connection process, number of concurrent, network flow, tps, qps;Monitoring resource is to monitor connection resource, file resource, table resource to make With rate and maximum permissible value;Current cache utilization rate and hit rate are specifically checked in key caching monitoring;Innodb monitoring is specifically to look into It sees that buffer pool, io Thread Count size and the additions and deletions of innodb change and looks into magnitude;It is multiple that duplication monitoring specifically obtains current principal and subordinate The monitor state of system;Table space analysis is specifically analytical database table space size.
In addition, auditing unit 3 for above-mentioned DDL, specifically databases comparison unifies entrance, and tenant, which chooses, to need to change Node, DDL and DML statement are submitted in input frame, system can be according to rule be custom-configured, and interior band parser can be to its grammer It is analyzed with semanteme, is not met the node instance of auditing standards, directly return to corresponding error information;For meeting audit mark Quasi- node instance can be executed directly in the inline system chosen, and the change can audit and can trace to the source;Meanwhile to all operations Flashback sentence is generated, rollback is facilitated, accomplishes that change is controllable.Link is changed in audit, by repairing thus according to MySQL source code Change, corresponding error message can all be returned to careful to the syntactic analysis of all sentences of submission, if grammer is problematic Core person, also offer semantic analysis, reports an error when the information such as table, library, a column are incorrect or do not meet auditing standards, or Person reports an error etc. when having used an object being not present.Additionally provide many functions for the constraint of SQL normalization, database Administrator can be configured by system parameter, provided only audit and do not executed function, tenant no longer needs to do inline system Be mirrored to test environment, when executing DML obtain flashback sentence also according to Binlog, provide and generate rollback sentence Function, accuracy rate are protected, while all operations Automatic dispatching all operationss after line examination & approval can audit, can trace to the source.
In certain embodiments, above-mentioned system further includes that slow log collection unit is gone forward side by side for analyzing slow SQL statement Row inverted sequence shows, and has automatically generated optimum indexing suggestion for slow SQL, tenant can directly debug SQL with reference to the suggestion.It will count According to library manager from the permission modification of log database server, audit execution, data change, the passive situations solution such as slow query optimization It has put and has come out, had more time studies study, further to the more intelligent direction of automation operation platform go to develop, improve Database administrator productivity.
In certain embodiments, above-mentioned system further includes account management unit, the connection created for managing tenant The application account of MySQL.Account only allows additions and deletions to change to look into, and connects database for program, if any change request, then need from DDL audit unit 3 changes.
In certain embodiments, above-mentioned system further includes backup management unit, all interior for being created to tenant Appearance is backed up, including all information in account, plays the role of safety management.
In addition, in certain embodiments, above-mentioned system further includes error log download unit, it to be used for node error log Download link facilitates tenant to pay close attention to the output of oneself node mistake.
The above-mentioned system for promoting database O&M efficiency based on DevOps is provided based on the MySQL resource pool planned Basic, normal, high three kinds of hardware grades configuration has asynchronous, half synchronization, synchronizes a variety of copy modes selections by force, provides Shen for tenant Please, audit is handled, and ultimately produces the visualized operation of MySQL database node, meanwhile, tenant can select according to business demand Single node or cluster distributed cloud database.After node is created, database monitoring, slow log Macro or mass analysis, backup plan It is slightly included in database administrator management platform automatically, subsequent 99% database manipulation tenant can pass through the self-service solution of the system Certainly.
The above-mentioned system for promoting database O&M efficiency based on DevOps is supervised by setting node application unit 1, node It controls unit 2 and DDL and audits unit 3, to the automatic application of node instance, monitoring nodes and change the audit of node from data Library manager frees, and forms the automated system for mixing multiple Open-Source Tools, improves database administrator productivity, data Library service is more efficient, reliable and stable, is also beneficial to batch operation and upgrading, and more efficient inter-department cooperation promotes association, team Make efficiency, without frittering away energy on flow system or e-mail communications, is easily accomplished the intersection of Ops and Dev, QA department Work, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
As shown in fig. 6, the present embodiment additionally provides the method for promoting database O&M efficiency based on DevOps, this method packet It includes:
S1, good installable node is planned in the resource pool table of MySQL;
S2, input node application condition;
S3, node is transferred from the resource pool table of MySQL according to application condition, qualified example is formed, by the reality Example state is set as in deployable state;
S4, the example is installed;
S5, the example is added in monitoring nodes, and sets use state for example state;
S6, monitoring nodes are carried out to example;
S7, according to the DDL and DML statement of the need node to be changed in node application condition and input, carry out grammer It is analyzed with semanteme, is fed back based on the analysis results;
S8, flashback sentence is generated to all operations.
For above-mentioned S1 step, specifically there is database administrator first to plan in the resource pool table of MySQL database The MySQL node that can be installed well, in order to be called at any time when subsequent tenant's application node example.
For above-mentioned S2 step, node application condition includes essential information, configuration information and availability information, wherein Essential information includes nodename, place region, intranet and extranet ip address port and creation time;Configuration information include version, The hardware informations such as CPU/ memory/resources usage amount;Availability information includes copy mode and deployment side;It can be detailed by node Whether the input of feelings input module 11 changes node or selection monitoring node etc., autonomous to select, system according to the selection of tenant into Row corresponding node example generates, and feeds back accordingly as a result, realizing and Dev, QA, Ops are put into same system exploitation, emphasis portion The self-service completion of door meet work, provides a kind of quickly and effectively mode, avoids unnecessary inter-sectional process, shorten Business on-line time, and ensured the market competitiveness of business, it is more suitable for internet+epoch.
For above-mentioned S6 step, the step of monitoring nodes are carried out to example, comprising the following specific steps
S61, the monitoring data for obtaining monitored node in example;
S62, the monitoring data is written in MySQL;
S63, monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
Above-mentioned S61 step specifically carries out monitoring nodes by monitoring nodes unit 2 to S63 step;From health monitoring, Many-sided prisons for carrying out node instance such as monitoring resource, the monitoring of key caching, Innodb monitoring, duplication monitoring and table space analysis Control, wherein health monitoring is mainly to look at current connection process, number of concurrent, network flow, tps, qps;Monitoring resource is monitoring Connection resource, file resource, the utilization rate of table resource and maximum permissible value;Current cache utilization rate is specifically checked in key caching monitoring And hit rate;Buffer pool, io Thread Count size and the additions and deletions that Innodb monitoring is specifically to look at innodb change the amount of looking into Grade;Duplication monitoring specifically obtains the monitor state of current leader follower replication;Table space analysis is specifically that analytical database table space is big It is small.
For above-mentioned S7 step, according to the need node to be changed in node application condition and the DDL and DML of input Sentence carries out the step of syntax and semantics are analyzed, fed back based on the analysis results, comprising the following specific steps
S71, example domestic demand node, DDL and DML statement to be changed are obtained;
S72, customized configuration rule is obtained, syntactic analysis can be carried out to DDL and DML statement using interior band syntax analyzer And semantic analysis;
Whether S73, discriminatory analysis result are to meet auditing standards;
If so, S74, selection corresponding node, execute corresponding change, and enter S8 step;
S75, if it is not, then returning to error information.
Above-mentioned S71 step is completed, tenant, which chooses, to be needed to S75 step and S8 step especially by DDL audit unit 3 Node to be changed submits DDL and DML statement in input frame, and system can be according to rule be custom-configured, and interior band parser can be right Its syntax and semantics is analyzed, and the node instance of auditing standards is not met, and directly returns to corresponding error information;For meeting The node instance of auditing standards can be executed directly in the inline system chosen, and the change can audit and can trace to the source;Meanwhile to institute There is operation to generate flashback sentence, facilitate rollback, accomplishes that change is controllable.Link is changed in audit, by thus according to MySQL Codes Revised all can return corresponding error message to the syntactic analysis of all sentences of submission, if grammer is problematic Back to auditor, also offer semantic analysis, when the information such as table, library, a column are incorrect or do not meet auditing standards Report an error when reporting an error, or having used an object being not present etc..Additionally provide many function for the constraint of SQL normalization Can, database administrator can all be configured by system parameter, provided only audit and do not executed function, tenant no longer needs to do Inline system is mirrored to test environment, obtains flashback sentence also according to Binlog when executing DML, provides and generate back The function of sentence is rolled, accuracy rate is protected, while all operations Automatic dispatching all operationss after line examination & approval can audit, can trace back Source.
The above-mentioned method that database O&M efficiency is promoted based on DevOps, by based on the MySQL resource pool planned, Basic, normal, high three kinds of hardware grades configuration is provided, has asynchronous, half synchronization, synchronize a variety of copy modes selections by force, mentioned for tenant For applying, audit processing, ultimately produce the visualized operation of MySQL database node, meanwhile, tenant can according to business demand, Select single node or cluster distributed cloud database.It is database monitoring, slow log Macro or mass analysis, standby after node is created Part strategy is included in database administrator management platform automatically, and subsequent 99% database manipulation tenant can be self-service by the system It solves.
The above-mentioned method for promoting database O&M efficiency based on DevOps, improves database administrator productivity, data Library service is more efficient, reliable and stable, is also beneficial to batch operation and upgrading, and more efficient inter-department cooperation promotes association, team Make efficiency, without frittering away energy on flow system or e-mail communications, is easily accomplished the intersection of Ops and Dev, QA department Work, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
It is above-mentioned that technology contents of the invention are only further illustrated with embodiment, in order to which reader is easier to understand, but not It represents embodiments of the present invention and is only limitted to this, any technology done according to the present invention extends or recreation, by of the invention Protection.Protection scope of the present invention is subject to claims.

Claims (8)

1. the system for promoting database O&M efficiency based on DevOps, which is characterized in that the system comprises:
Node application unit, for for tenant's application node example;
Monitoring nodes unit is written to for acquiring the monitoring data of the monitored node in the node application unit MySQL reads monitoring data and graphical representation from MySQL using jqPlot chart;
DDL audits unit, for according to DDL the and DML language for choosing need node to be changed and input in node application unit Sentence, carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback sentence to all operations.
2. the system according to claim 1 for promoting database O&M efficiency based on DevOps, which is characterized in that the section Point application unit includes node details input module, example generation module and installation module;
The node details input module, for inputting applied all node instance information for tenant;
The example generation module, the resource pool table of information for being inputted according to the node details input module from MySQL Middle to read qualified example, change example state is deployable state;
The installation module for installing to the example that tenant applies, and is added to the monitoring nodes unit, change Example state is use state.
3. the system according to claim 1 or 2 for promoting database O&M efficiency based on DevOps, which is characterized in that institute The system of stating further includes slow log collection unit, for analyzing slow SQL statement, and carries out inverted sequence and shows.
4. the system according to claim 3 for promoting database O&M efficiency based on DevOps, which is characterized in that the system System further includes account management unit, for managing the application account for the connection MySQL that tenant is created.
5. the system according to claim 4 for promoting database O&M efficiency based on DevOps, which is characterized in that the system System further includes SQL optimiged index unit, for optimizing for SQL index used in tenant.
6. the method for promoting database O&M efficiency based on DevOps, which is characterized in that the described method includes:
Good installable node is planned in the resource pool table of MySQL;
Input node application condition;
Node is transferred from the resource pool table of MySQL according to application condition, qualified example is formed, the example state is set It is set in deployable state;
The example is installed;
The example is added in monitoring nodes, and sets use state for example state;
Monitoring nodes are carried out to example;
According to the need node to be changed in node application condition and the DDL and DML statement of input, carry out syntax and semantics into Row analysis, is fed back based on the analysis results;
Flashback sentence is generated to all operations.
7. the method according to claim 6 for promoting database O&M efficiency based on DevOps, which is characterized in that example The step of carrying out monitoring nodes, comprising the following specific steps
Obtain the monitoring data of monitored node in example;
The monitoring data is written in MySQL;
Monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
8. the method according to claim 7 for promoting database O&M efficiency based on DevOps, which is characterized in that according to section The DDL and DML statement of need node to be changed and input in point application condition, carry out syntax and semantics and are analyzed, root The step of being fed back according to analysis result, comprising the following specific steps
Obtain example domestic demand node, DDL and DML statement to be changed;
Customized configuration rule is obtained, syntactic analysis and semanteme can be carried out to DDL and DML statement using interior band syntax analyzer Analysis;
Whether discriminatory analysis result is to meet auditing standards;
If so, selection corresponding node, executes corresponding change, and enter the step of generating flashback sentence to all operations;
If it is not, then returning to error information.
CN201710901776.4A 2017-09-28 2017-09-28 The system and method for database O&M efficiency are promoted based on DevOps Pending CN110019118A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710901776.4A CN110019118A (en) 2017-09-28 2017-09-28 The system and method for database O&M efficiency are promoted based on DevOps

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710901776.4A CN110019118A (en) 2017-09-28 2017-09-28 The system and method for database O&M efficiency are promoted based on DevOps

Publications (1)

Publication Number Publication Date
CN110019118A true CN110019118A (en) 2019-07-16

Family

ID=67186417

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710901776.4A Pending CN110019118A (en) 2017-09-28 2017-09-28 The system and method for database O&M efficiency are promoted based on DevOps

Country Status (1)

Country Link
CN (1) CN110019118A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110795420A (en) * 2019-10-30 2020-02-14 浪潮云信息技术有限公司 Automatic MySQL database backup method based on infrastructure
CN110795450A (en) * 2019-09-12 2020-02-14 连连银通电子支付有限公司 SQL processing method and device
CN111651431A (en) * 2020-06-09 2020-09-11 中国雄安集团数字城市科技有限公司 Database service oriented management flow standardization method

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1464434A (en) * 2002-06-26 2003-12-31 联想(北京)有限公司 Method for realizing modular query language interpreter in the flush type data base system
CN103617176A (en) * 2013-11-04 2014-03-05 广东电子工业研究院有限公司 Method for achieving automatic synchronization of multi-source heterogeneous data resources
CN103631924A (en) * 2013-12-03 2014-03-12 Tcl集团股份有限公司 Application method and system for distributive database platform
CN103678532A (en) * 2013-12-02 2014-03-26 中国移动(深圳)有限公司 Alternation statement reverse analysis method, database alternating and backspacing method and database alternating and backspacing system
CN104333512A (en) * 2014-10-30 2015-02-04 北京思特奇信息技术股份有限公司 Distributed memory database access system and method
CN104391995A (en) * 2014-12-15 2015-03-04 北京趣拿软件科技有限公司 SQL (Structured Query Language) statement auditing method, and database operation and maintenance method and system
CN104951524A (en) * 2015-06-11 2015-09-30 上海新炬网络信息技术有限公司 Mobile platform for database operation and maintenance and using method of mobile platform
CN105302697A (en) * 2015-11-13 2016-02-03 中国建设银行股份有限公司 Running state monitoring method and system of density data model database
CN105631602A (en) * 2016-01-11 2016-06-01 中国移动通信集团广东有限公司 Business grade operation platform based on DevOps
US20170169059A1 (en) * 2015-12-15 2017-06-15 Mongodb, Inc. Systems and methods for automating management of distributed databases

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1464434A (en) * 2002-06-26 2003-12-31 联想(北京)有限公司 Method for realizing modular query language interpreter in the flush type data base system
CN103617176A (en) * 2013-11-04 2014-03-05 广东电子工业研究院有限公司 Method for achieving automatic synchronization of multi-source heterogeneous data resources
CN103678532A (en) * 2013-12-02 2014-03-26 中国移动(深圳)有限公司 Alternation statement reverse analysis method, database alternating and backspacing method and database alternating and backspacing system
CN103631924A (en) * 2013-12-03 2014-03-12 Tcl集团股份有限公司 Application method and system for distributive database platform
CN104333512A (en) * 2014-10-30 2015-02-04 北京思特奇信息技术股份有限公司 Distributed memory database access system and method
CN104391995A (en) * 2014-12-15 2015-03-04 北京趣拿软件科技有限公司 SQL (Structured Query Language) statement auditing method, and database operation and maintenance method and system
CN104951524A (en) * 2015-06-11 2015-09-30 上海新炬网络信息技术有限公司 Mobile platform for database operation and maintenance and using method of mobile platform
CN105302697A (en) * 2015-11-13 2016-02-03 中国建设银行股份有限公司 Running state monitoring method and system of density data model database
US20170169059A1 (en) * 2015-12-15 2017-06-15 Mongodb, Inc. Systems and methods for automating management of distributed databases
CN105631602A (en) * 2016-01-11 2016-06-01 中国移动通信集团广东有限公司 Business grade operation platform based on DevOps

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
邹筱菁等: ""基于DevOps的软件开发管理"", 《数字技术与应用》 *
陈咏秋等: ""面向云服务的DevOps知识获取与应用"", 《计算机系统应用》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110795450A (en) * 2019-09-12 2020-02-14 连连银通电子支付有限公司 SQL processing method and device
CN110795420A (en) * 2019-10-30 2020-02-14 浪潮云信息技术有限公司 Automatic MySQL database backup method based on infrastructure
CN111651431A (en) * 2020-06-09 2020-09-11 中国雄安集团数字城市科技有限公司 Database service oriented management flow standardization method

Similar Documents

Publication Publication Date Title
US20220253298A1 (en) Systems and methods for transformation of reporting schema
Mens et al. Evolving software architecture descriptions of critical systems
US7580946B2 (en) Smart integration engine and metadata-oriented architecture for automatic EII and business integration
CN105119757A (en) Method and system for operation and maintenance automation of enterprise servers
CN102789415A (en) Modular script designer for next generation testing system
CN102375731A (en) Coding-free integrated application platform system
CN111651431A (en) Database service oriented management flow standardization method
US20130339933A1 (en) Systems and methods for quality assurance automation
CN112527774A (en) Data center building method and system and storage medium
CN115374102A (en) Data processing method and system
CN110942155A (en) Research method of machine learning engine
CN110019118A (en) The system and method for database O&M efficiency are promoted based on DevOps
CN112395317B (en) Method for collaborative design and achievement management of data model
CN103412745A (en) Development and application platform
CN108876283A (en) A kind of automatic auditing system and method
Berenbach et al. Metrics for model driven requirements development
Rabbi et al. YAWL2DVE: An automated translator for workflow verification
CN108287897A (en) A kind of visual distributed data fix tool and its method
CN112130849B (en) Code automatic generation method and device
CN111984826B (en) XML-based data automatic warehousing method, system, device and storage medium
Ma et al. Microservice Migration Using Strangler Fig Pattern and Domain-Driven Design.
CN108647018A (en) Can live preview code generating system
CN114911773A (en) Universal meta-model design method
ElGamal et al. An Architecture-Oriented Data Warehouse Testing Approach.
CN112308508A (en) Information unified management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190716