CN110019118A - The system and method for database O&M efficiency are promoted based on DevOps - Google Patents
The system and method for database O&M efficiency are promoted based on DevOps Download PDFInfo
- Publication number
- CN110019118A CN110019118A CN201710901776.4A CN201710901776A CN110019118A CN 110019118 A CN110019118 A CN 110019118A CN 201710901776 A CN201710901776 A CN 201710901776A CN 110019118 A CN110019118 A CN 110019118A
- Authority
- CN
- China
- Prior art keywords
- node
- database
- mysql
- devops
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012544 monitoring process Methods 0.000 claims abstract description 71
- 238000004458 analytical method Methods 0.000 claims abstract description 35
- 238000012550 audit Methods 0.000 claims abstract description 25
- 230000008859 change Effects 0.000 claims description 41
- 230000001737 promoting effect Effects 0.000 claims description 18
- 238000007726 management method Methods 0.000 claims description 10
- 238000009434 installation Methods 0.000 claims description 6
- 238000011161 development Methods 0.000 abstract description 7
- 230000002045 lasting effect Effects 0.000 abstract description 7
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 16
- 238000000275 quality assurance Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000036541 health Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000007792 addition Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000012508 change request Methods 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007257 malfunction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to the system and method that database O&M efficiency is promoted based on DevOps, which includes the node application unit for tenant's application node example；Monitoring nodes unit is written to MySQL, reads monitoring data and graphical representation from MySQL using jqPlot chart for acquiring the monitoring data of monitored node in the node application unit；DDL audits unit, for carrying out the analysis of syntax and semantics, being fed back based on the analysis results according to the DDL and DML statement that need node to be changed and input is chosen in node application unit, generates flashback sentence to all operations.The present invention improves database administrator productivity, database service is more efficient, reliable and stable, it is also beneficial to batch operation and upgrading, more efficient inter-department cooperation, promote team collaboration's efficiency, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
Description
Technical field
The present invention relates to enterprise database operation platforms, more specifically refer to and promote database O&M effect based on DevOps
The system and method for rate.
Background technique
In Project Running Process, database performance is beneath, Frequent Troubles are configured often from operating system and database
Item missing, table structure, SQL design are lack of standardization；Also because permission is chaotic, sentence writes the reasons such as fault leads to database maloperation
Constantly.Therefore, to ensure that database stability, performance maximize, probability of malfunction is reduced, database administrator needs were being researched and developed
Following item is completed in unified control in journey, support: database service deployment, DDL manual examination and verification publication, DML maloperation are corrected, slowly
Query optimization, optimiged index, rights management, troubleshooting ... arrive when the database varying duty of projects is parallel, a side
Face data library manager link can become the bottleneck of whole flow process；On the other hand it requires to abide by from audit is deployed to due to database
Standardization is followed, between the complexity of sentence and environment, manual examination and verification and operation are difficult to ensure high-accuracy.
Generally there are artificial change and semi-automatic change mode for the change work in business data at present, wherein such as
Shown in Fig. 1, for manually change method, all change work is completed by manually walking process, the friendship of this method
Stream mode is usually mail, and when needing to be implemented change, DEV/QA writes mail and initiates to change to database administrator, in data depositary management
Reason person receives and probably completes following process: business request for data base resource later, after database administrator application hardware resource, hand
It is dynamic to dispose and access various monitoring, high-availability system, standby system one by one；Research staff applies for that access resource, personnel change, number
Server change permission is frequently logged on according to library manager personnel；Developer submits databases comparison to require, database administrator
Advise after audit, the process of walking again for not meeting specification rewrites；For satisfactory, the direct line of database administrator
Upper execution, but manual examination and verification are easy error, mistake are just found after online practice, and necessary reply is to exploitation.For semi-automatic
For change, as shown in Fig. 2, assisting individual artificial lengthy and tedious affairs using Open-Source Tools or script, exchange way is still postal
Part or flow system, probably there is a following process: business request for data base resource, after database administrator application hardware resource, step on
Record server operation automatized script deployment；There is a powerful review procedure druid resolver, process is still and manual examination and verification
Equally；Database administrator still needs login service device to change permission；The work of server disposition plan target, it is automatic daily to receive
Slow SQL on line concentration, hair mail optimize to each developer after analysis summary, developer to log in test environment check index with
Executive plan.
But it for artificial change mode, has the following problems: the daily repeated work of database administrator, especially
It in DDL audit, execution and link, recycles and error-prone, 100% identification not can guarantee for syntax error, semantic error naked eyes,
It just to practice and just find on line；For defined rule, there is loose unevenness in every database administrator, most
Cause rule not at rule eventually, is difficult to carry out down；Before execution, database administrator will check environment on line, consider language
Stability on sentence pair line has an impact, and tradeoff executes the time；After execution, it is also necessary to notify DEV, QA department to verify, data depositary management
Reason person could complete the change, waste many time costs.It for semi-automatic change mode, has the following problems: every time
Change, database administrator still want login service device remote batch to operate；Druid resolver still remains that there may be do not have
The loophole audited brings very big risk to database on line；Review procedure first has to execute once in test environment,
If it find that can just go to execute on line after out of question, then this test just needs moment and synchronous, the data of environment holding on line
It needs equally, it has to be ensured that identical, execute just in Beta has referring to property in this way the metadata of database and table.One frequently
The node of change, test environment need environment on continuous line synchro, and cost is very high；Since backup export data and execution are not
Inside same affairs, not can guarantee the accuracy of Backup Data, and after change on line data covering, can not rollback, this
The backup of sample has little significance.In conclusion either artificial change or semi-automatic change, there are development difficulty height and O&Ms
The high problem of complexity.
Therefore, it is necessary to design a kind of system for promoting database O&M efficiency based on DevOps, realizing reduces exploitation hardly possible
Degree and O&M difficulty ensure lasting, efficient, the stable output service of database.
Summary of the invention
It is an object of the invention to overcome the deficiencies of existing technologies, provides and database O&M efficiency is promoted based on DevOps
System and method.
To achieve the above object, the invention adopts the following technical scheme: promoting database O&M efficiency based on DevOps
System, the system comprises:
Node application unit, for for tenant's application node example；
Monitoring nodes unit is written to for acquiring the monitoring data of the monitored node in the node application unit
MySQL reads monitoring data and graphical representation from MySQL using jqPlot chart；
DDL audits unit, for according to chosen in node application unit the DDL for needing node to be changed and input and
DML statement carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback language to all operations
Sentence.
Its further technical solution are as follows: the node application unit includes node details input module, example generation module
And installation module；
The node details input module, for inputting applied all node instance information for tenant；
The example generation module, the resource of information for being inputted according to the node details input module from MySQL
Qualified example is read in the table of pond, change example state is deployable state；
The installation module for installing to the example that tenant applies, and is added to the monitoring nodes unit,
Example state is changed as use state.
Its further technical solution are as follows: the system also includes slow log collection units, for analyzing slow SQL statement, and
Inverted sequence is carried out to show.
Its further technical solution are as follows: the system also includes account management units, the company created for managing tenant
Connect the application account of MySQL.
Its further technical solution are as follows: the system also includes SQL optimiged index units, for optimizing for used in tenant
SQL index.
The present invention also provides the methods for promoting database O&M efficiency based on DevOps, which comprises
Good installable node is planned in the resource pool table of MySQL；
Input node application condition；
Node is transferred from the resource pool table of MySQL according to application condition, forms qualified example, by the example shape
State is set as in deployable state；
The example is installed；
The example is added in monitoring nodes, and sets use state for example state；
Monitoring nodes are carried out to example；
According to the need node to be changed in node application condition and the DDL and DML statement of input, grammer and language are carried out
Justice is analyzed, and is fed back based on the analysis results；
Flashback sentence is generated to all operations.
Its further technical solution are as follows: the step of monitoring nodes are carried out to example, comprising the following specific steps
Obtain the monitoring data of monitored node in example；
The monitoring data is written in MySQL；
Monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
Its further technical solution are as follows: according to the need node to be changed in node application condition and the DDL of input and
DML statement carries out the step of syntax and semantics are analyzed, fed back based on the analysis results, comprising the following specific steps
Obtain example domestic demand node, DDL and DML statement to be changed；
Obtain customized configuration rule, using interior band syntax analyzer DDL and DML statement can be carried out syntactic analysis and
Semantic analysis；
Whether discriminatory analysis result is to meet auditing standards；
If so, selection corresponding node, executes corresponding change, and enters and flashback sentence is generated to all operations
Step；
If it is not, then returning to error information.
Compared with the prior art, the invention has the advantages that: it is of the invention that database O&M efficiency is promoted based on DevOps
System, unit is audited by setting node application unit, monitoring nodes unit and DDL, to node instance it is automatic apply,
Monitoring nodes and the audit for changing node are freed from database administrator, form the automation for mixing multiple Open-Source Tools
System improves database administrator productivity, and database service is more efficient, reliable and stable, is also beneficial to batch operation and liter
Grade, more efficient inter-department cooperation promote team collaboration's efficiency, without frittering away energy on flow system or e-mail communications,
It is easily accomplished the work of the intersection of Ops and Dev, QA department, realizing reduces development difficulty and O&M difficulty, ensures database
Lasting, efficient, stable output service.
The invention will be further described in the following with reference to the drawings and specific embodiments.
Detailed description of the invention
Fig. 1 is the flow chart of the artificial change mode of the prior art；
Fig. 2 is the flow chart of the semi-automatic change mode of the prior art；
Fig. 3 is the structural frames for the system that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides
Figure；
Fig. 4 is the structural block diagram for the node application unit that the specific embodiment of the invention provides；
Fig. 5 is the functional block for the system that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides
Frame figure；
Fig. 6 is the process for the method that database O&M efficiency is promoted based on DevOps that the specific embodiment of the invention provides
Figure；
Fig. 7 is the flow chart that monitoring nodes are carried out to example that the specific embodiment of the invention provides；
Fig. 8 be the specific embodiment of the invention provide according in node application condition need node to be changed and input
DDL and DML statement carry out the flow chart analyzed of syntax and semantics.
Specific embodiment
In order to more fully understand technology contents of the invention, combined with specific embodiments below to technical solution of the present invention into
One step introduction and explanation, but not limited to this.
The specific embodiment as shown in Fig. 3~8, it is provided in this embodiment that database O&M efficiency is promoted based on DevOps
System can be used in and open database service and automatically access monitoring and carry out all daily changes and subsequent operation operation
During, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
DevOps (Development Operations) is the general designation of one group of process, method and system, is opened for promoting
Communication, cooperation and integration between hair, technology operation and quality assurance (QA) department.Under traditional On-Premise mode, respectively
The working condition of departmental staff is often interrupted to support the work to cross, this is unfavorable for the work for being absorbed in oneself, if by this
A little regularized operations are simultaneously encapsulated, and provide unified operation interface to Dev, QA, autonomous solve the problems, such as may be implemented completely.
As shown in figure 3, present embodiments providing the system for promoting database O&M efficiency based on DevOps, the system packet
It includes:
Node application unit 1, for for tenant's application node example；
Monitoring nodes unit 2 is written for acquiring the monitoring data of the monitored node in the node application unit 1
To MySQL, monitoring data and graphical representation are read from MySQL using jqPlot chart；
DDL audits unit 3, for according to chosen in node application unit 1 DDL for needing node to be changed and input and
DML statement carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback language to all operations
Sentence.
Specifically, the above-mentioned system for promoting database O&M efficiency based on DevOps is specifically to utilize ansible+
Python+php mixes the Open-Source Tools such as inception, sqladvisor, percona toolkit, jqPlot, accomplishes to be based on
The creation of MySQL resource pool and the procedure of monitoring are handled, and self-service operation largely saves process spending.
According to the condition for tenant's application node example that node application unit 1 is got, the calling for carrying out node forms reality
Example, and monitoring nodes unit 2 is added, if enumerating change node in condition, is then analyzed, returned by DDL audit unit 3
Return change as a result, and will operation generate flashback sentence, be used for rollback；And monitoring nodes unit 2 is then responsible for carrying out node
Monitoring data is obtained, and is presented to tenant, improves research and development, database administrator efficiency, ensures that database is lasting, efficient, steady
Fixed output service.
In certain embodiments, above-mentioned node application unit 1 includes node details input module 11, example generation module
12 and installation module 13.
Node details input module 11, for inputting applied all node instance information for tenant.
Example generation module 12, the resource of information for being inputted according to the node details input module 11 from MySQL
Qualified example is read in the table of pond, change example state is deployable state.
Module 13 is installed, for being installed to the example that tenant applies, and is added to the monitoring nodes unit 2,
Example state is changed as use state.
For above-mentioned node details input module 11, essential information, configuration information and availability information have been specifically included,
Wherein essential information includes nodename, place region, intranet and extranet ip address port and creation time；Configuration information includes version
The hardware informations such as sheet, CPU/ memory/resources usage amount；Availability information includes copy mode and deployment side；Section can be passed through
Whether the point input of details input module 11 changes node or selection monitoring node etc., autonomous to select, and system is according to the choosing of tenant
It selects and carries out the generation of corresponding node example, and feed back accordingly as a result, realizing and Dev, QA, Ops are put into same system exploitation, note
The self-service completion of weight department's meet work, provides a kind of quickly and effectively mode, avoids unnecessary inter-sectional process,
Shortening business on-line time, and ensured the market competitiveness of business, it is more suitable for internet+epoch.
For above-mentioned example generation module 12 and installation module 13, it is mainly used for the process of automatic application node example
In, the generation of corresponding node example is carried out to the node application condition (i.e. node instance information) of tenant's input, on condition that DB is needed
Good installable MySQL node is planned inside MySQL resource pool table in advance, in order to according to the application condition of tenant at any time
It calls node and installs, form node instance, reach self-service operation, largely save the effect of process spending.
For above-mentioned monitoring nodes unit 2, specifically from health monitoring, monitoring resource, the monitoring of key caching, Innodb prison
Many-sided monitoring for carrying out node instance such as control, duplication monitoring and table space analysis, wherein health monitoring, which is mainly to look at, works as
Preceding connection process, number of concurrent, network flow, tps, qps；Monitoring resource is to monitor connection resource, file resource, table resource to make
With rate and maximum permissible value；Current cache utilization rate and hit rate are specifically checked in key caching monitoring；Innodb monitoring is specifically to look into
It sees that buffer pool, io Thread Count size and the additions and deletions of innodb change and looks into magnitude；It is multiple that duplication monitoring specifically obtains current principal and subordinate
The monitor state of system；Table space analysis is specifically analytical database table space size.
In addition, auditing unit 3 for above-mentioned DDL, specifically databases comparison unifies entrance, and tenant, which chooses, to need to change
Node, DDL and DML statement are submitted in input frame, system can be according to rule be custom-configured, and interior band parser can be to its grammer
It is analyzed with semanteme, is not met the node instance of auditing standards, directly return to corresponding error information；For meeting audit mark
Quasi- node instance can be executed directly in the inline system chosen, and the change can audit and can trace to the source；Meanwhile to all operations
Flashback sentence is generated, rollback is facilitated, accomplishes that change is controllable.Link is changed in audit, by repairing thus according to MySQL source code
Change, corresponding error message can all be returned to careful to the syntactic analysis of all sentences of submission, if grammer is problematic
Core person, also offer semantic analysis, reports an error when the information such as table, library, a column are incorrect or do not meet auditing standards, or
Person reports an error etc. when having used an object being not present.Additionally provide many functions for the constraint of SQL normalization, database
Administrator can be configured by system parameter, provided only audit and do not executed function, tenant no longer needs to do inline system
Be mirrored to test environment, when executing DML obtain flashback sentence also according to Binlog, provide and generate rollback sentence
Function, accuracy rate are protected, while all operations Automatic dispatching all operationss after line examination & approval can audit, can trace to the source.
In certain embodiments, above-mentioned system further includes that slow log collection unit is gone forward side by side for analyzing slow SQL statement
Row inverted sequence shows, and has automatically generated optimum indexing suggestion for slow SQL, tenant can directly debug SQL with reference to the suggestion.It will count
According to library manager from the permission modification of log database server, audit execution, data change, the passive situations solution such as slow query optimization
It has put and has come out, had more time studies study, further to the more intelligent direction of automation operation platform go to develop, improve
Database administrator productivity.
In certain embodiments, above-mentioned system further includes account management unit, the connection created for managing tenant
The application account of MySQL.Account only allows additions and deletions to change to look into, and connects database for program, if any change request, then need from
DDL audit unit 3 changes.
In certain embodiments, above-mentioned system further includes backup management unit, all interior for being created to tenant
Appearance is backed up, including all information in account, plays the role of safety management.
In addition, in certain embodiments, above-mentioned system further includes error log download unit, it to be used for node error log
Download link facilitates tenant to pay close attention to the output of oneself node mistake.
The above-mentioned system for promoting database O&M efficiency based on DevOps is provided based on the MySQL resource pool planned
Basic, normal, high three kinds of hardware grades configuration has asynchronous, half synchronization, synchronizes a variety of copy modes selections by force, provides Shen for tenant
Please, audit is handled, and ultimately produces the visualized operation of MySQL database node, meanwhile, tenant can select according to business demand
Single node or cluster distributed cloud database.After node is created, database monitoring, slow log Macro or mass analysis, backup plan
It is slightly included in database administrator management platform automatically, subsequent 99% database manipulation tenant can pass through the self-service solution of the system
Certainly.
The above-mentioned system for promoting database O&M efficiency based on DevOps is supervised by setting node application unit 1, node
It controls unit 2 and DDL and audits unit 3, to the automatic application of node instance, monitoring nodes and change the audit of node from data
Library manager frees, and forms the automated system for mixing multiple Open-Source Tools, improves database administrator productivity, data
Library service is more efficient, reliable and stable, is also beneficial to batch operation and upgrading, and more efficient inter-department cooperation promotes association, team
Make efficiency, without frittering away energy on flow system or e-mail communications, is easily accomplished the intersection of Ops and Dev, QA department
Work, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
As shown in fig. 6, the present embodiment additionally provides the method for promoting database O&M efficiency based on DevOps, this method packet
It includes:
S1, good installable node is planned in the resource pool table of MySQL；
S2, input node application condition；
S3, node is transferred from the resource pool table of MySQL according to application condition, qualified example is formed, by the reality
Example state is set as in deployable state；
S4, the example is installed；
S5, the example is added in monitoring nodes, and sets use state for example state；
S6, monitoring nodes are carried out to example；
S7, according to the DDL and DML statement of the need node to be changed in node application condition and input, carry out grammer
It is analyzed with semanteme, is fed back based on the analysis results；
S8, flashback sentence is generated to all operations.
For above-mentioned S1 step, specifically there is database administrator first to plan in the resource pool table of MySQL database
The MySQL node that can be installed well, in order to be called at any time when subsequent tenant's application node example.
For above-mentioned S2 step, node application condition includes essential information, configuration information and availability information, wherein
Essential information includes nodename, place region, intranet and extranet ip address port and creation time；Configuration information include version,
The hardware informations such as CPU/ memory/resources usage amount；Availability information includes copy mode and deployment side；It can be detailed by node
Whether the input of feelings input module 11 changes node or selection monitoring node etc., autonomous to select, system according to the selection of tenant into
Row corresponding node example generates, and feeds back accordingly as a result, realizing and Dev, QA, Ops are put into same system exploitation, emphasis portion
The self-service completion of door meet work, provides a kind of quickly and effectively mode, avoids unnecessary inter-sectional process, shorten
Business on-line time, and ensured the market competitiveness of business, it is more suitable for internet+epoch.
For above-mentioned S6 step, the step of monitoring nodes are carried out to example, comprising the following specific steps
S61, the monitoring data for obtaining monitored node in example；
S62, the monitoring data is written in MySQL；
S63, monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
Above-mentioned S61 step specifically carries out monitoring nodes by monitoring nodes unit 2 to S63 step；From health monitoring,
Many-sided prisons for carrying out node instance such as monitoring resource, the monitoring of key caching, Innodb monitoring, duplication monitoring and table space analysis
Control, wherein health monitoring is mainly to look at current connection process, number of concurrent, network flow, tps, qps；Monitoring resource is monitoring
Connection resource, file resource, the utilization rate of table resource and maximum permissible value；Current cache utilization rate is specifically checked in key caching monitoring
And hit rate；Buffer pool, io Thread Count size and the additions and deletions that Innodb monitoring is specifically to look at innodb change the amount of looking into
Grade；Duplication monitoring specifically obtains the monitor state of current leader follower replication；Table space analysis is specifically that analytical database table space is big
It is small.
For above-mentioned S7 step, according to the need node to be changed in node application condition and the DDL and DML of input
Sentence carries out the step of syntax and semantics are analyzed, fed back based on the analysis results, comprising the following specific steps
S71, example domestic demand node, DDL and DML statement to be changed are obtained；
S72, customized configuration rule is obtained, syntactic analysis can be carried out to DDL and DML statement using interior band syntax analyzer
And semantic analysis；
Whether S73, discriminatory analysis result are to meet auditing standards；
If so, S74, selection corresponding node, execute corresponding change, and enter S8 step；
S75, if it is not, then returning to error information.
Above-mentioned S71 step is completed, tenant, which chooses, to be needed to S75 step and S8 step especially by DDL audit unit 3
Node to be changed submits DDL and DML statement in input frame, and system can be according to rule be custom-configured, and interior band parser can be right
Its syntax and semantics is analyzed, and the node instance of auditing standards is not met, and directly returns to corresponding error information；For meeting
The node instance of auditing standards can be executed directly in the inline system chosen, and the change can audit and can trace to the source；Meanwhile to institute
There is operation to generate flashback sentence, facilitate rollback, accomplishes that change is controllable.Link is changed in audit, by thus according to MySQL
Codes Revised all can return corresponding error message to the syntactic analysis of all sentences of submission, if grammer is problematic
Back to auditor, also offer semantic analysis, when the information such as table, library, a column are incorrect or do not meet auditing standards
Report an error when reporting an error, or having used an object being not present etc..Additionally provide many function for the constraint of SQL normalization
Can, database administrator can all be configured by system parameter, provided only audit and do not executed function, tenant no longer needs to do
Inline system is mirrored to test environment, obtains flashback sentence also according to Binlog when executing DML, provides and generate back
The function of sentence is rolled, accuracy rate is protected, while all operations Automatic dispatching all operationss after line examination & approval can audit, can trace back
Source.
The above-mentioned method that database O&M efficiency is promoted based on DevOps, by based on the MySQL resource pool planned,
Basic, normal, high three kinds of hardware grades configuration is provided, has asynchronous, half synchronization, synchronize a variety of copy modes selections by force, mentioned for tenant
For applying, audit processing, ultimately produce the visualized operation of MySQL database node, meanwhile, tenant can according to business demand,
Select single node or cluster distributed cloud database.It is database monitoring, slow log Macro or mass analysis, standby after node is created
Part strategy is included in database administrator management platform automatically, and subsequent 99% database manipulation tenant can be self-service by the system
It solves.
The above-mentioned method for promoting database O&M efficiency based on DevOps, improves database administrator productivity, data
Library service is more efficient, reliable and stable, is also beneficial to batch operation and upgrading, and more efficient inter-department cooperation promotes association, team
Make efficiency, without frittering away energy on flow system or e-mail communications, is easily accomplished the intersection of Ops and Dev, QA department
Work, realizing reduces development difficulty and O&M difficulty, ensures lasting, efficient, the stable output service of database.
It is above-mentioned that technology contents of the invention are only further illustrated with embodiment, in order to which reader is easier to understand, but not
It represents embodiments of the present invention and is only limitted to this, any technology done according to the present invention extends or recreation, by of the invention
Protection.Protection scope of the present invention is subject to claims.
Claims (8)
1. the system for promoting database O&M efficiency based on DevOps, which is characterized in that the system comprises:
Node application unit, for for tenant's application node example；
Monitoring nodes unit is written to for acquiring the monitoring data of the monitored node in the node application unit
MySQL reads monitoring data and graphical representation from MySQL using jqPlot chart；
DDL audits unit, for according to DDL the and DML language for choosing need node to be changed and input in node application unit
Sentence, carries out the analysis of syntax and semantics, is fed back based on the analysis results, and generates flashback sentence to all operations.
2. the system according to claim 1 for promoting database O&M efficiency based on DevOps, which is characterized in that the section
Point application unit includes node details input module, example generation module and installation module；
The node details input module, for inputting applied all node instance information for tenant；
The example generation module, the resource pool table of information for being inputted according to the node details input module from MySQL
Middle to read qualified example, change example state is deployable state；
The installation module for installing to the example that tenant applies, and is added to the monitoring nodes unit, change
Example state is use state.
3. the system according to claim 1 or 2 for promoting database O&M efficiency based on DevOps, which is characterized in that institute
The system of stating further includes slow log collection unit, for analyzing slow SQL statement, and carries out inverted sequence and shows.
4. the system according to claim 3 for promoting database O&M efficiency based on DevOps, which is characterized in that the system
System further includes account management unit, for managing the application account for the connection MySQL that tenant is created.
5. the system according to claim 4 for promoting database O&M efficiency based on DevOps, which is characterized in that the system
System further includes SQL optimiged index unit, for optimizing for SQL index used in tenant.
6. the method for promoting database O&M efficiency based on DevOps, which is characterized in that the described method includes:
Good installable node is planned in the resource pool table of MySQL；
Input node application condition；
Node is transferred from the resource pool table of MySQL according to application condition, qualified example is formed, the example state is set
It is set in deployable state；
The example is installed；
The example is added in monitoring nodes, and sets use state for example state；
Monitoring nodes are carried out to example；
According to the need node to be changed in node application condition and the DDL and DML statement of input, carry out syntax and semantics into
Row analysis, is fed back based on the analysis results；
Flashback sentence is generated to all operations.
7. the method according to claim 6 for promoting database O&M efficiency based on DevOps, which is characterized in that example
The step of carrying out monitoring nodes, comprising the following specific steps
Obtain the monitoring data of monitored node in example；
The monitoring data is written in MySQL；
Monitoring data is read from MySQL using jqPlot chart and carries out graphical representation.
8. the method according to claim 7 for promoting database O&M efficiency based on DevOps, which is characterized in that according to section
The DDL and DML statement of need node to be changed and input in point application condition, carry out syntax and semantics and are analyzed, root
The step of being fed back according to analysis result, comprising the following specific steps
Obtain example domestic demand node, DDL and DML statement to be changed；
Customized configuration rule is obtained, syntactic analysis and semanteme can be carried out to DDL and DML statement using interior band syntax analyzer
Analysis；
Whether discriminatory analysis result is to meet auditing standards；
If so, selection corresponding node, executes corresponding change, and enter the step of generating flashback sentence to all operations；
If it is not, then returning to error information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710901776.4A CN110019118A (en) | 2017-09-28 | 2017-09-28 | The system and method for database O&M efficiency are promoted based on DevOps |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710901776.4A CN110019118A (en) | 2017-09-28 | 2017-09-28 | The system and method for database O&M efficiency are promoted based on DevOps |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019118A true CN110019118A (en) | 2019-07-16 |
Family
ID=67186417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710901776.4A Pending CN110019118A (en) | 2017-09-28 | 2017-09-28 | The system and method for database O&M efficiency are promoted based on DevOps |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019118A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795420A (en) * | 2019-10-30 | 2020-02-14 | 浪潮云信息技术有限公司 | Automatic MySQL database backup method based on infrastructure |
| CN110795450A (en) * | 2019-09-12 | 2020-02-14 | 连连银通电子支付有限公司 | SQL processing method and device |
| CN111651431A (en) * | 2020-06-09 | 2020-09-11 | 中国雄安集团数字城市科技有限公司 | Database service oriented management flow standardization method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1464434A (en) * | 2002-06-26 | 2003-12-31 | 联想(北京)有限公司 | Method for realizing modular query language interpreter in the flush type data base system |
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN103631924A (en) * | 2013-12-03 | 2014-03-12 | Tcl集团股份有限公司 | Application method and system for distributive database platform |
| CN103678532A (en) * | 2013-12-02 | 2014-03-26 | 中国移动（深圳）有限公司 | Alternation statement reverse analysis method, database alternating and backspacing method and database alternating and backspacing system |
| CN104333512A (en) * | 2014-10-30 | 2015-02-04 | 北京思特奇信息技术股份有限公司 | Distributed memory database access system and method |
| CN104391995A (en) * | 2014-12-15 | 2015-03-04 | 北京趣拿软件科技有限公司 | SQL (Structured Query Language) statement auditing method, and database operation and maintenance method and system |
| CN104951524A (en) * | 2015-06-11 | 2015-09-30 | 上海新炬网络信息技术有限公司 | Mobile platform for database operation and maintenance and using method of mobile platform |
| CN105302697A (en) * | 2015-11-13 | 2016-02-03 | 中国建设银行股份有限公司 | Running state monitoring method and system of density data model database |
| CN105631602A (en) * | 2016-01-11 | 2016-06-01 | 中国移动通信集团广东有限公司 | Business grade operation platform based on DevOps |
| US20170169059A1 (en) * | 2015-12-15 | 2017-06-15 | Mongodb, Inc. | Systems and methods for automating management of distributed databases |
-
2017
- 2017-09-28 CN CN201710901776.4A patent/CN110019118A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1464434A (en) * | 2002-06-26 | 2003-12-31 | 联想(北京)有限公司 | Method for realizing modular query language interpreter in the flush type data base system |
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN103678532A (en) * | 2013-12-02 | 2014-03-26 | 中国移动（深圳）有限公司 | Alternation statement reverse analysis method, database alternating and backspacing method and database alternating and backspacing system |
| CN103631924A (en) * | 2013-12-03 | 2014-03-12 | Tcl集团股份有限公司 | Application method and system for distributive database platform |
| CN104333512A (en) * | 2014-10-30 | 2015-02-04 | 北京思特奇信息技术股份有限公司 | Distributed memory database access system and method |
| CN104391995A (en) * | 2014-12-15 | 2015-03-04 | 北京趣拿软件科技有限公司 | SQL (Structured Query Language) statement auditing method, and database operation and maintenance method and system |
| CN104951524A (en) * | 2015-06-11 | 2015-09-30 | 上海新炬网络信息技术有限公司 | Mobile platform for database operation and maintenance and using method of mobile platform |
| CN105302697A (en) * | 2015-11-13 | 2016-02-03 | 中国建设银行股份有限公司 | Running state monitoring method and system of density data model database |
| US20170169059A1 (en) * | 2015-12-15 | 2017-06-15 | Mongodb, Inc. | Systems and methods for automating management of distributed databases |
| CN105631602A (en) * | 2016-01-11 | 2016-06-01 | 中国移动通信集团广东有限公司 | Business grade operation platform based on DevOps |
Non-Patent Citations (2)
| Title |
|---|
| 邹筱菁等: ""基于DevOps的软件开发管理"", 《数字技术与应用》 * |
| 陈咏秋等: ""面向云服务的DevOps知识获取与应用"", 《计算机系统应用》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795450A (en) * | 2019-09-12 | 2020-02-14 | 连连银通电子支付有限公司 | SQL processing method and device |
| CN110795420A (en) * | 2019-10-30 | 2020-02-14 | 浪潮云信息技术有限公司 | Automatic MySQL database backup method based on infrastructure |
| CN111651431A (en) * | 2020-06-09 | 2020-09-11 | 中国雄安集团数字城市科技有限公司 | Database service oriented management flow standardization method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220253298A1 (en) | Systems and methods for transformation of reporting schema | |
| Mens et al. | Evolving software architecture descriptions of critical systems | |
| US7580946B2 (en) | Smart integration engine and metadata-oriented architecture for automatic EII and business integration | |
| CN105119757A (en) | Method and system for operation and maintenance automation of enterprise servers | |
| US9411575B2 (en) | Systems and methods for quality assurance automation | |
| CN102375731A (en) | Coding-free integrated application platform system | |
| CN111651431A (en) | Database service oriented management flow standardization method | |
| CN112527774A (en) | Data center building method and system and storage medium | |
| CN110942155A (en) | Research method of machine learning engine | |
| CN110019118A (en) | The system and method for database O&M efficiency are promoted based on DevOps | |
| CN112395317B (en) | Method for collaborative design and achievement management of data model | |
| CN103412745A (en) | Development and application platform | |
| CN108876283A (en) | A kind of automatic auditing system and method | |
| CN115169011A (en) | Editing system and application system of airplane assembly outline | |
| Rabbi et al. | YAWL2DVE: An automated translator for workflow verification | |
| CN111984826B (en) | XML-based data automatic warehousing method, system, device and storage medium | |
| CN108287897A (en) | A kind of visual distributed data fix tool and its method | |
| CN112130823A (en) | Platform development method and device, computer equipment and storage medium | |
| CN114911773A (en) | Universal meta-model design method | |
| CN112130849B (en) | Code automatic generation method and device | |
| ElGamal et al. | An Architecture-Oriented Data Warehouse Testing Approach. | |
| CN112308508A (en) | Information unified management system | |
| CN108647018A (en) | Can live preview code generating system | |
| CN102841842A (en) | Automation controller for next generation testing system | |
| Ma et al. | Microservice Migration Using Strangler Fig Pattern and Domain-Driven Design. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |