CN116719872A

CN116719872A - Database deployment method and database management platform

Info

Publication number: CN116719872A
Application number: CN202310914218.7A
Authority: CN
Inventors: 向峙
Original assignee: Shenzhen Jiuniuyimao Intelligent Internet Of Things Technology Co ltd
Current assignee: Shenzhen Jiuniuyimao Intelligent Internet Of Things Technology Co ltd
Priority date: 2023-07-24
Filing date: 2023-07-24
Publication date: 2023-09-08

Abstract

The application provides a database deployment method and a database management platform, wherein the deployment method comprises the following steps: responding to a database application submitted by a user, and acquiring data characteristics of data to be stored by a database management platform; the database management platform determines a database deployment strategy corresponding to the database application according to the data characteristics; the database management platform deploys the database corresponding to the database application according to the database deployment strategy. The method and the device can reduce the time for creating the database and improve the deployment efficiency of the database.

Description

Database deployment method and database management platform

Technical Field

The present application relates to the field of computer technologies, and in particular, to a database deployment method and a database management platform.

Background

Databases are system software that organizes, stores, and manages data according to a data structure, which is a collection of large amounts of data stored in a computer for a long period of time, organized, sharable, and uniformly managed. A database is a system software that uses specific data structures (indexes), data storage formats (block stores), and data access interfaces to achieve orderly management of data. The data itself may be managed and stored in other ways, such as by storing the data in files, which may be understood as an unordered way of storing the data.

The database software mainly provides data service for other application programs, and the application program end needs a matched architecture scheme and an operation and maintenance scheme for meeting the final requirement of a user. The database management platform is application software for providing the operation and maintenance scheme and the architecture scheme for the database so as to achieve the automatic deployment and management of the operation and maintenance scheme and the architecture scheme of the database.

The existing database management platform can not effectively provide one-stop data service, can only independently provide certain database service, can support incomplete types of databases and can provide functions, if multiple database services are needed, users are required to use the databases in a combined mode, and the cooperative functions of the databases are required in the combined mode. The above reasons cause that the deployment method of the database applied to the existing database management platform is not intelligent enough, the degree of automation is not enough, excessive intervention of users is needed, and the database deployment efficiency is lower.

Disclosure of Invention

In view of the above, the present application provides a method for deploying a database and a database management platform, so as to improve the deployment efficiency of the database.

In a first aspect, a method for deploying a database is provided, including: responding to a database application submitted by a user, and acquiring data characteristics of data to be stored by a database management platform; the database management platform determines a database deployment strategy corresponding to the database application according to the data characteristics; the database management platform deploys the database corresponding to the database application according to the database deployment strategy.

In one possible implementation manner, the database management platform determines a database deployment policy corresponding to the database application according to the data feature, including: and the database management platform determines the types and the quantity of the databases to be deployed from a plurality of preset databases according to the data characteristics.

In one possible implementation manner, the database management platform determines a database deployment policy corresponding to the database application according to the data feature, and specifically further includes: and the database management platform determines corresponding database cooperative services according to the data characteristics.

In one possible implementation manner, the database management platform deploys a database corresponding to the database application according to the database deployment policy, including: the database management platform allocates machine resources according to the database deployment strategy; and the database management platform installs the database corresponding to the database application on the machine resource.

In one possible implementation manner, the database management platform installs the database corresponding to the database application on the machine resource, including: the database management platform deploys a database plug-in management program on the machine resource; and the database management platform installs the database corresponding to the database application according to the database plug-in management program.

In one possible implementation, the data characteristics include one or more of a number of data concurrency, a data day increment, online data aging, online data projected storage capacity, historical data aging, historical data projected storage capacity.

In one possible implementation, the database collaboration service includes at least one of a data service, a database routing connection program, and a distributed database middleware.

In a second aspect, a database management platform is provided, including: the acquisition module is used for acquiring data characteristics of data to be stored according to a database application submitted by a user; the determining module is used for determining a database deployment scheme according to the data characteristics; the deployment module is used for deploying the database corresponding to the database application according to the database deployment strategy.

In one possible implementation manner, the acquiring module is specifically configured to: the received database applications can be processed sequentially according to the application time.

In one possible implementation manner, the acquiring module is specifically configured to: and processing the received database application to obtain the data characteristics of the data to be stored in the database application.

In one possible implementation, the determining module is specifically configured to: and determining the types and the quantity of the databases to be deployed from a plurality of preset databases according to the data characteristics.

In one possible implementation, the determining module is specifically configured to: and determining corresponding database cooperative service according to the data characteristics.

In one possible implementation, the deployment module is specifically configured to: and distributing machine resources according to the database deployment strategy.

In one possible implementation, the deployment module is specifically configured to: and deploying a database plug-in management program on the machine resource.

In one possible implementation, the deployment module is specifically configured to: and installing the database corresponding to the database application according to the database plug-in management program.

In a third aspect, a database management platform is provided, including at least one processor, where the at least one processor is configured to couple to a memory, and read and execute instructions in the memory, to implement a method provided by any implementation manner of the first aspect.

In a fourth aspect, a computer readable medium is provided, the computer readable medium storing program code which, when run on a computer, causes the computer to perform the method provided by any one of the implementations of the first aspect.

In a fifth aspect, a chip system is provided, comprising a processor for calling and running a computer program from a memory, so that an electronic device on which the chip system is installed performs the method provided by any of the implementations of the first aspect.

In a sixth aspect, there is provided a computer program product comprising: computer program code which, when run on a computer, causes the computer to perform the method provided by any of the implementations of the first aspect described above.

It should be noted that, the above computer program code may be stored in whole or in part on a first storage medium, where the first storage medium may be packaged together with the processor or may be packaged separately from the processor, which is not specifically limited by the present application.

According to the method for deploying the database, the data characteristics of the data to be stored, which are input by the user, are obtained through responding to the database application submitted by the user, the database deployment strategy is determined based on the data characteristics, and the database deployment is completed based on the determined database deployment strategy. The embodiment of the application can realize automatic deployment and management of the database, does not need manual selection of the database by a manager or excessive interference of users, effectively avoids the problem of mismatching of a database deployment strategy and an application caused by missing database deployment information in the process of manually deploying the database, and further avoids the modification process of the deployed database, thereby shortening the database deployment period, improving the use experience of users and improving the efficiency of database deployment; further, the determined database deployment strategy can deploy the database under the condition of meeting the deployment strategy reference information, one-stop service of database deployment is completed, the utilization rate of deployment resources is effectively reduced, the consumption of deployment resources is reduced, the deployment efficiency is improved, and the labor cost is reduced.

Drawings

FIG. 1 is a general flow chart of a database management platform provided by an embodiment of the present application.

Fig. 2 is a flowchart of a method for deploying a database according to an embodiment of the present application.

FIG. 3 is a diagram of a single type of database service architecture provided by an embodiment of the present application.

Fig. 4 is a diagram of a collaboration service architecture of multiple databases of the same type provided in an embodiment of the present application.

Fig. 5 is a diagram of a collaboration service architecture of multiple types of databases provided in an embodiment of the present application.

Fig. 6 is a flowchart of a method for deploying a database management platform according to an embodiment of the present application.

FIG. 7 is a schematic block diagram of a database management platform provided by an embodiment of the present application.

Fig. 8 is a block diagram of a database management platform according to an embodiment of the present application.

Detailed Description

The technical scheme of the application will be described below with reference to the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The term "comprises/comprising" when used herein is taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the embodiments of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

In order to facilitate understanding of the technical solution of the present application, related terms and concepts will be first described in brief.

Mysql: is a relational database management system that keeps data in different tables rather than placing all the data in one large warehouse, which increases speed and increases flexibility. It is the primary open-source relational database, and the market share is the first in the world.

Redis: the key-value database, i.e. the key-value database, is a memory-based key-value pair storage system which is open in source, is mainly used as a high-performance cache server, and is the most popular cache solution at present.

TiDB: the Newsql database is a natural distributed database, which is a database of a Spanner system, is a fusion distributed database supporting online transaction processing and online analysis processing at the same time, and has the important characteristic of compatibility with mysql protocol.

ES: the search engine database is simple in elastic search, the kernel is built based on Lucene, the distributed multi-user full-text search engine is provided, and the real-time search, stability, reliability, rapidness and convenience in installation and use can be achieved.

MongoDB: the early version was a file storage database, and the current version has evolved into a distributed database of the spandex hierarchy. MongoDB is a product that is interposed between a relational database and a non-relational database, most functional among which is most like a relational database. The biggest characteristic of Mongo is that the query language supported by Mongo is very powerful, the grammar is somewhat similar to the object-oriented query language, almost most functions similar to the query of a relational database list can be realized, and the indexing of data is also supported.

TDEngine: the time sequence database is a time sequence database with open source, high performance and cloud primordia, and besides the function of the time sequence database, the TDengine also provides other functions such as caching, data subscription, stream computing and the like so as to reduce the complexity of the system and the cost of research, development and operation.

OLTP: the On-Line Transaction Processing is called transaction-oriented processing, and is basically characterized in that user data received by a foreground can be immediately transmitted to a computing center for processing, and a processing result is given in a short time, so that the On-line transaction processing is one of the modes of quick response to user operation.

qps: queries Per Second is a common measure of search traffic received by an information retrieval system (e.g., a search engine or database) within one second. It is widely used in any request-response system, more correctly called requests per second (RPS: request Per Second).

dts: data Transfer Service, a data transmission service supports data interaction among data sources such as a Relational Database (RDBMS), a non-relational database (NoSQL), data multidimensional analysis (OLAP), and the like, integrates data synchronization, migration, subscription, integration and processing, and is a safe, extensible and high-availability data architecture.

das: data Archive Service, a data archiving service, the data archiving function is to archive some historical data or temporarily unused data migration into inexpensive storage.

ddm: distributed Database Middleware, the distributed database middleware is used for solving the problem of distributed expansion of the database, breaking through the capacity and performance bottleneck of the traditional database and realizing high concurrent access of mass data.

Dbagnet: the self-developed database plug-in management program is generally deployed on the same server as the database service program, and the current technology is generally realized by using grpc service and written in go language.

Dbproxy: is a middleware located between the front-end application and the back-end Mysql database, a dynamic routing database connection routing program.

grpc: a high-performance, multi-language, universal open source RPC framework developed by google is designed primarily for mobile application development and based on the HTTP/2 protocol standard, while supporting most popular programming languages.

RPC: remote Procedure Call, a protocol for remote procedure calls between services, the main functional goal of RPC is to make it easier to build distributed computing (applications).

The embodiment of the application firstly provides a database deployment method, which can be applied to a database management platform 500, and the database management platform 500 is described below.

Fig. 1 is a diagram of information structure of a database management platform 500 according to an embodiment of the present application. As shown in fig. 1, the database management platform 500 is relatively complete in terms of the types of databases that it can support and the management functions that it covers, and can provide a deployment solution for databases for a variety of utility scenarios. The types of databases that can be managed by the database management platform 500 include, but are not limited to, traditional relational databases, in-memory databases, columnar databases, wide-table databases, graph databases, temporal databases, spatial databases, span-based distributed databases, search engine databases, and the like.

For example, a traditional relational database may include Mysql, postgres, oracle, sqlServer, etc.; the in-memory database may include reids, memcache, etc.; the columnar database may include Clickhouse, hbase, etc.; the table database may include Cassandra, scyllaDB, etc.; the graph database may include TuGraph, hugeGraph, neo j, etc.; the timing database may include TDEngine, timescaleDB, openTSDB, etc.; the spatial database may include postgis and the like; the spandex architecture distributed database may include MongoDB, tiDB, cockroachDB, etc.; the search engine database may include an elastiscearch, or the like.

In addition, the core functions of the database management platform 500 include, but are not limited to: database resource application, database installation, backup restoration, data synchronization, database high availability, database distributed architecture construction, database change auditing, database cluster architecture, database monitoring and the like.

Fig. 2 is a schematic flow chart of a database deployment method 200 provided by an embodiment of the present application. The method may be applied to the aforementioned database management platform 500, for example, may be performed by the platform or a chip or a controller in the platform, hereinafter, a deployment method 200 provided in an embodiment of the present application is described with reference to fig. 2, and the deployment method may be applied to a service architecture diagram of the database management platform 500 in fig. 3-5, where the method 200 includes:

In step S210, in response to the database application submitted by the user, the database management platform 500 obtains the data characteristics of the data to be stored.

Optionally, the user may input the data features through an input box of the Web management end database application interface, and after the Web management end detects the input determining operation of the user, the database application carrying the data features may be generated and sent to the database management platform 500, so that the database management platform 500 may extract the data features in the database application.

The Web management end of the database management platform is mainly an operation interface for users to manage various databases and various data services, and the primary function of the Web management end is resources such as a resource nano tube, a nano tube physical machine, a virtual machine, a container and the like. In some cases, the Web manager and the database management platform 500 may be integrated together, for example, on a terminal or a server, that is, they may together form a device.

Alternatively, in other implementations, the data feature may be sent separately, rather than being carried in a database application, as the application is not limited in this regard.

In practical use, the database management platform 500 may receive the database applications uploaded by multiple Web management ends, so that the database management platform 500 may store the received multiple database applications in the message queue, process the multiple database applications according to the time sequence of receiving the application, and acquire the next database application from the message queue for database deployment after completing the database deployment task corresponding to one database application.

Alternatively, the database management platform 500 may be a mobile phone, a tablet computer, a desktop computer, a central control computer, a cloud server, a cluster server, or a management application server.

Optionally, the data characteristics herein can be used to characterize the data to be stored, and may include, for example, one or more of data concurrency, data day increment, online data aging, online data projected storage capacity, historical data aging, historical data projected storage capacity. For example, the data concurrency number is a data feature that is a data performance capacity analysis, and other features are all data features of the data performance capacity analysis.

In step S220, the database management platform 500 determines a database deployment policy corresponding to the database application according to the data characteristics.

Specifically, the database management platform 500 may determine, according to the data characteristics, a database deployment policy corresponding to the database application. For example, the database deployment policy may be a single category of database deployment policy, a plurality of categories of database collaborative deployment policies (e.g., via ddm), or a plurality of different categories of database collaborative deployment policies, etc., but is not limited thereto.

For example, a deployment policy for a single category database may serve the Mysql database alone.

For another example, a deployment policy that is coordinated by multiple homogeneous databases may provide coordinated services for multiple Mysql's through ddm.

For another example, deployment policies for collaboration of multiple heterogeneous databases may be relational databases Mysql, key value databases dis, and monglondb, and provide collaboration services through das or dts.

In one possible design, the database management platform may determine the type and number of databases to be deployed from a plurality of databases preset according to the data characteristics.

Optionally, the preset database includes a plurality of databases that can be provided to the user service, including, but not limited to, a relational database Mysql, an open source relational database Postgres, a relational database Oracle, a structured query language database SqlServer, a key value database dis, a distributed memory object cache database Memcache, a column database Clickhouse, a distributed database Hbase, a distributed memory database Cassandra, noSQL database scyladb, a high-altitude map database TuGraph, hugeGraph, neo j, a native distributed database TiDB, a search engine database elastsearch, a file memory database MongoDB, a time-series database TDEngine, a time-series database TimescaleDB, a development time-series database OpenTSDB, an object relational database postgis, a distributed database cockroach db, and the like. The database management platform 500 may determine one, two or three or more types of databases to be deployed from the preset plurality of databases according to the data characteristics, and the determined number of databases of each type.

Alternatively, in an embodiment of the present application, the database management platform 500 may have a capacity analysis model built therein, where the capacity analysis model may analyze the performance capacity (for example, the concurrency number) and the storage capacity (for example, the online data expected storage capacity) of the data at the same time according to the data characteristics, so as to determine the types and the numbers of the databases to be deployed from the preset multiple databases, and the principle of selecting the corresponding databases is according to the built-in function dimension labels.

Specifically, the functional dimension labels are edited and entered in the background by a database management platform administrator. Different databases have different functional dimension labels that are used to identify the functional types of the databases. For example, redis may be used as a cache database to be tagged with a cache, mysql may be tagged with oltp, tidb is a distributed database to be tagged with a distributed cluster, and the database management platform may determine the type and number of databases to be deployed in the deployment policy by selecting a database with a proper function tag according to the data characteristics.

In step S230, the database management platform 500 deploys the database corresponding to the database application according to the database deployment policy.

Specifically, the database management platform 500 deploys databases corresponding to the database applications according to a database deployment policy, where the databases may be a single database, multiple homogeneous databases, or multiple heterogeneous databases, where the databases may be deployed on separate servers, and each database may work separately on a different server. Alternatively, multiple databases may be deployed on top of the same server.

As one possible implementation manner, the database management platform 500 deploys a database corresponding to the database application according to the database deployment policy, including: the database management platform 500 allocates machine resources according to a database deployment policy; the database management platform 500 installs the database corresponding to the database application on the machine resource.

Specifically, the database management platform 500 allocates machine resources according to a deployment policy, where the determined deployment policy includes a database to be deployed, and there is a single database, multiple databases of the same kind, or multiple databases of different kinds, where the different databases need to allocate different machine resources, that is, allocate different servers, and each database works on a different server separately; after the machine resources of the databases are allocated, the database management platform deploys the external hanging management program of the corresponding database, such as dbagant, on the machine resources of each database, and the database management platform issues an instruction to the external hanging management program to complete the deployment and installation of the databases.

Specifically, the dbagant is an independent plug-in program and corresponding database are deployed on the same server, and dbagant needs to be custom developed for each different database. The general development mode of dbagant is to use go language to package the method for managing the database into functions which can be called by grpc protocol based on grpc protocol, and the Web terminal management terminal of the database management platform calls the functions by accessing the port monitored by dbagant through grpc protocol so as to install and manage the corresponding database.

In one possible design, the database management platform 500 determines a database deployment policy corresponding to the database application according to the data characteristics, and specifically further includes: the database management platform 500 determines the corresponding database collaboration service based on the data characteristics.

Optionally, the database collaboration service includes at least one of a data service d×s, a database routing connection program, and a distributed database middleware.

For example, data services d x s include, but are not limited to, data migration services dts, data archiving services das, database routing connectivity procedures may be, for example, dbproxy, and distributed database middleware may be, for example, ddm.

Specifically, the database collaboration service needs to allocate machine resources individually to deploy, and the database management platform 500 determines a database deployment policy corresponding to the database application according to the data characteristics. The deployment policy may be a deployment policy of a single kind of database, a deployment policy of a plurality of kinds of databases in coordination (for example, through ddm), or a deployment policy of a plurality of different kinds of databases in coordination, and if the deployment policy is a single kind of database, no coordination service is required; if multiple heterogeneous databases cooperate, multiple heterogeneous databases and distributed database middleware ddm are required to cooperatively provide services (e.g., multiple Mysql require ddm to provide cooperative services), the database management platform also allocates independent machine resources to ddm, and deploys corresponding ddm on the independent machine resources; if the databases are coordinated, one or more of the databases of different types, d (such as das and dts), and Dbproxy are needed to cooperatively provide services, the database management platform also allocates independent machine resources to d, and deploys corresponding d.

According to the method 200 for deploying the database, provided by the embodiment of the application, the data characteristics of the data to be stored, which are input by the user, are obtained by responding to the database application submitted by the user, the database deployment strategy is determined based on the data characteristics, and the deployment of the database is completed based on the determined database deployment strategy. The embodiment of the application can realize automatic deployment and management of the database, does not need manual selection of the database by a manager or excessive interference of users, effectively avoids the problem of mismatching of a database deployment strategy and an application caused by missing database deployment information in the process of manually deploying the database, and further avoids the modification process of the deployed database, thereby shortening the database deployment period, improving the use experience of users and improving the efficiency of database deployment; further, the determined database deployment strategy can deploy the database under the condition of meeting the deployment strategy reference information, one-stop service of database deployment is completed, the utilization rate of deployment resources is effectively reduced, the consumption of deployment resources is reduced, the deployment efficiency is improved, and the labor cost is reduced.

To facilitate understanding of the present application, the following description will be given by way of several examples, which illustrate how the database management platform 500 determines a database deployment policy corresponding to a database application according to the data characteristics.

Example 1:

the data features input by the user include the data concurrency number and the on-line data predicted storage capacity, the capacity analysis model built in the database management platform 500 analyzes the performance capacity and the storage capacity of the data according to the acquired data features, and determines a suitable database service according to the functional dimension labels, and the service architecture is shown in fig. 3.

For example, the number of data concurrency input by the user is less than 1 ten thousand, the expected storage capacity of the online data is less than 10 hundred million, and no history data exists, namely the data characteristics comprise the number of data concurrency and the expected storage capacity of the online data. The database management platform 500 may determine that Mysql provides services by function dimension labels based on the number of data concurrency and the expected storage capacity of online data entered by the user.

Example 2:

For example, the number of concurrent data input by the user is 5 ten thousand, the expected storage capacity of the online data is less than 100 hundred million, and no historical data exists, namely, the data characteristics comprise the number of concurrent data and the expected storage capacity of the online data. The database management platform 500 may determine the TiDB to provide services by function dimension tags based on the number of data concurrency and the expected storage capacity of online data entered by the user.

Example 3:

the data features input by the user include the data concurrency number and the on-line data predicted storage capacity, the capacity analysis model built in the database management platform 500 analyzes the performance capacity and the storage capacity of the data according to the acquired data features, and determines a suitable database service according to the functional dimension labels, and the service architecture is shown in fig. 4.

For example, the user sets the number of data concurrency to 10 ten thousand, and the expected storage capacity of the online data is 1000 hundred million lines of data, that is, the data features include the number of data concurrency and the expected storage capacity of the online data at this time, and the database management platform 500 may select an appropriate database service according to the functional dimension tag according to the number of data concurrency and the expected storage capacity of the online data input by the user. The most common solution is to automatically select a database distributed middleware service ddm and provide services in conjunction with multiple mysqls. The database management platform 500 automatically determines how many Mysql instances the background needs to build as a storage service, and meanwhile builds how many ddm instances to provide a data routing service, and builds a load balancing service upstream of the plurality ddm to provide an external unified service interface.

Specifically, the number of the above construction is mainly determined according to ddm characteristics and the data concurrency number in the user data characteristics and the expected storage capacity of the online data, and the database management platform administrator can set ddm a maximum threshold value for processing the concurrency number according to ddm software computing power and corresponding hardware configuration. The database management platform 500 can calculate ddm the number according to the maximum threshold of the data concurrency number input by the user and the processing concurrency number of ddm, and ddm directly has an optimal calculation ratio with the Mysql node according to the software characteristics of itself.

Further, ddm software and the downstream Mysql nodes have optimal performance when the cpu proportion is 1/15, after the number of the system is calculated ddm, the number n1 of Mysql preselected nodes can be calculated according to the cpu proportion, then the number n2 of Mysql preselected nodes is calculated according to the online data capacity input by a user and the single-node data capacity of each Mysql node, and then the maximum value is selected from the n1 and the n2 as the final Mysql node number. For example, if the user sets 10 ten thousand concurrent data and the online data is expected to have 1000 hundred million lines of data, assuming that the cpu core number threshold for processing the 10 ten thousand concurrent ddm is 192c and the current environment ddm is provided with a machine of 64 cores, 3 ddm nodes are created, the Mysql node is provided with a machine of 32 cores and the online data storage capacity threshold is 10 hundred million, then the number of Mysql preselected nodes n1=192×15/32=90, the number of Mysql preselected nodes n2=1000 hundred million/10 hundred million=100, and the number of n2> n1 is 100.

Example 4:

the data features input by the user include the data concurrency number and the predicted storage capacity of the online data, the capacity analysis model built in the database management platform 500 analyzes the performance capacity and the storage capacity of the data according to the acquired data features, and determines a suitable database service according to the functional dimension labels, and the service architecture is shown in fig. 5.

For example, in the case where the user sets the number of data concurrency to 10 ten thousand and the expected storage capacity of online data is smaller than 10 hundred million lines of data, the data feature includes the number of data concurrency and the expected storage capacity of online data in this case. The database management platform 500 selects a proper database according to the data concurrency number and the online data expected storage capacity input by a user and the function dimension labels, the database management platform 500 intelligently judges that the online data storage capacity meets the Mysql single-instance specification processing capability on line, but the concurrency number exceeds the Mysql single-instance specification processing capability, the database management platform 500 selects Redis+Mysql as an optimal deployment scheme, the Redis provides cache service, and the Mysql is used as back-end data hardening. The database management platform 500 constructs a data synchronization mechanism between the cache service Redis and the relational database Mysql by itself, meanwhile, the database management platform 500 also constructs a network proxy service to provide services to the outside as a whole, so that users do not need to pay attention to the internal implementation, and the database management platform 500 constructs a unified Dbproxy to the outside service at the same time, thus the whole is an oltp database service with a cache.

Example 5:

For example, a user may input data characteristics less than 1 million concurrence, an on-line data projected storage capacity less than 10 billion, and a historical data projected storage capacity greater than 1000 billion, i.e., where the data characteristics include the number of data concurrence and the on-line data projected storage capacity. The database management platform 500 selects a required database according to the data concurrency number and the online data expected storage capacity input by a user and the function dimension labels, the database management platform 500 judges that the data concurrency number and the online data expected storage capacity are both within the processing capacity of a single Mysql instance, if the historical data processing capacity is exceeded, tiDB (time-dependent database) is selected to store historical data, machine construction das service is automatically allocated, and das service constructs Mysql to TiDB data archiving rules according to the timeliness of the online data input by the user.

Fig. 6 is a deployment method 600 provided in an embodiment of the present application, where the deployment method 600 may be applied to the service architecture of the database management platform in fig. 3 to 5, for example, the deployment method 600 may be performed by the database management platform in fig. 3 to 5, and the deployment method 600 specifically includes the following steps:

in step S601, the user logs into the database management platform 500 and inputs data characteristics of the data to be stored, wherein the data characteristics are characteristics for characterizing the data to be stored, including, but not limited to, one or more of data concurrency, data day increment, online data aging, online data expected storage capacity, historical data aging, and historical data expected storage capacity.

In step S602, the database management platform 500 determines a database deployment policy corresponding to the database application according to the data feature.

In particular, the database deployment policy may be a single kind of deployment policy of a database, a deployment policy of a plurality of kinds of databases in cooperation (for example, through ddm), or a deployment policy of a plurality of different kinds of databases in cooperation, etc., but is not limited thereto.

Alternatively, the capacity analysis model built in the database management platform 500 may determine the types and the numbers of the databases to be deployed from a plurality of preset databases according to the characteristics of the data and simultaneously analyze the performance capacity and the storage capacity of the data, and the types and the numbers of the databases may be one, two, three or more.

In step S603, the database management platform 500 deploys the database machine resources corresponding to the database application according to the database deployment policy.

Specifically, the database management platform 500 allocates machine resources according to a deployment policy, where the determined deployment policy includes a database to be deployed, and there is a single database, multiple databases of the same kind, or multiple databases of different kinds, where the different databases need to be allocated with different machine resources, that is, different servers, and each database works on a different server separately.

In step S604, the database management platform 500 deploys Dagent corresponding to the data type on the allocated database machine resources, and then issues an instruction to the Dagent to install the corresponding database.

Specifically, after the machine resources of the databases are allocated, the database management platform 500 deploys the plug-in manager dbagant of the corresponding database on the machine resources of each database, and the database management platform issues an instruction to the dbagant to complete the deployment and installation of the databases.

Alternatively, if the database management platform 500 determines that a single database (corresponding to the first type a of the flowchart) is required based on the data characteristics entered by the user, the database management platform allocates the database machine resources and deploys the corresponding dbagants based on the user's selection, as in example 1 and example 2 in the above examples.

Optionally, if the database management platform 500 determines that multiple databases or multiple sets of databases of the same type are required according to the data features input by the user, the database management platform allocates the main service database resources and deploys corresponding dbagants according to the user selection, automatically allocates the resources of the required cooperative database types and deploys corresponding dbagants according to the data features input by the user, and creates matched data cooperative services according to the corresponding scenes.

In step S605, if the capacity analysis model of the database management platform 500 selects the same type of multiple databases to work cooperatively (corresponding to the first type B in the flowchart) according to the performance capacity and the storage capacity of the data feature analysis data input by the user, the database management platform will allocate the machine resources required by the distributed database middleware ddm individually, and deploy the corresponding ddm, as in example 3 in the above example.

In step S606, if the capacity analysis model of the database management platform 500 selects multiple types of databases to cooperate to determine the deployment scenario (corresponding to the second type in the flowchart) according to the performance capacity and the storage capacity of the data feature analysis data input by the user, the database management platform 500 will allocate the machine resources individually to deploy the data services d×s, where d×s is operated on the independent machine resources.

In step S607, after the corresponding d×s is deployed on the separate machine resource, the database management platform 500 constructs the dbprox of the unified external service, as in example 4 and example 5 in the above examples.

In the deployment method 600, the automation management capability of the database management platform 500 is implemented by the Web management end and the Dbagent server end of the corresponding database, and in some cases, the Web management end and the database management platform 500 may be integrated together, for example, integrated on a terminal or a server, that is, the two may together form a device. Specifically, the Web management end mainly manages dbagant and various d-s services based on grpc protocol, and the Web management end does not limit development language based on grpc cross-platform characteristics. When Dbagent is deployed and d is constructed, interaction is performed between the deployed machine and the ssh protocol of the deployed machine, after deployment is successful, the Web management end of the database management platform 500 interacts with the Dbagent and d by the grpc protocol, and at this time, the Web management end of the database management platform 500 works as a client end of grpc, and Dbagent and d are used as a server end of grpc. d s and dbagant also interact through grpc protocol, where d s operates as a client of grpc and dbagant operates as a server of grpc.

The method for deploying a database provided by the embodiment of the present application is described in detail above with reference to fig. 2 to fig. 6, and the database management platform 500 of the embodiment of the present application is described in detail below with reference to fig. 7 and fig. 8. It should be appreciated that the database management platform shown in fig. 7, 8 is capable of implementing one or more of the steps of the method flows shown in fig. 2 or 6. To avoid repetition, details are not repeated here.

The embodiment of the present application further provides a database management platform 500, referring to fig. 7, where the database management platform 500 includes:

an obtaining module 501, configured to obtain a data characteristic of data to be stored sent by a client in response to a database application sent by the client;

the determining module 502 is configured to determine a database deployment policy corresponding to the database application according to the data characteristics;

the deployment module 503 is configured to deploy the database corresponding to the database application according to the database deployment policy.

In one possible implementation, the obtaining module 501 is specifically configured to: the received database applications can be processed sequentially according to the application time.

In one possible implementation, the obtaining module 501 is specifically configured to: and processing the received database application to obtain the data characteristics of the data to be stored in the database application.

In one possible implementation, the determining module 502 is specifically configured to: and determining the types and the quantity of the databases to be deployed from a plurality of preset databases according to the data characteristics.

In one possible implementation, the determining module 502 is specifically configured to: and determining corresponding database cooperative service according to the data characteristics.

In one possible implementation, the deployment module 503 is specifically configured to: and distributing machine resources according to the database deployment strategy.

In one possible implementation, the deployment module 503 is specifically configured to: and deploying a database plug-in management program on the machine resource.

In one possible implementation, the deployment module 503 is specifically configured to: and installing the database corresponding to the database application according to the database plug-in management program.

In particular, the database management 500 may correspond to a database management platform in the deployment method 200 and the deployment method 600 according to an embodiment of the present application, or a chip configured in the platform. The database management platform 500 may include various modules for performing the methods performed by the database management platform 500 of fig. 2. In addition, in order to implement the respective flows of the deployment method 200 and the deployment method 600, the specific processes of executing the respective steps by the units in each module and the other operations and/or functions in the database management platform are described in detail in the deployment method 200 and the deployment method 600, and are not described herein for brevity.

The embodiment of the application also provides a database management platform 500, and fig. 8 is a structural block diagram of the database management platform 500 provided by the embodiment of the application. As shown in fig. 8, the database management platform 500 includes a processor 510 and a memory 520, each of which may be connected by one or more buses 530.

The database management platform 500 also includes a computer program 521, the computer program 521 being stored in the memory 520, which when executed by the processor 510 causes the database management platform 500 to perform the deployment method 200 shown in fig. 2 or the deployment method 600 shown in fig. 6 described above. All relevant contents of each step related to the above method embodiment may be cited to the functional description of the corresponding entity device, which is not described herein.

Embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon, which when executed on an electronic device, causes the electronic device to perform the method 200 shown in fig. 2 or the method 600 shown in fig. 6 described above.

The embodiment of the application also provides a chip, which comprises: and a processor for calling and running a computer program from the memory, so that the electronic device on which the chip is mounted performs the method 200 shown in fig. 2 or the method 600 shown in fig. 6 described above.

The embodiment of the application also provides a computer program product, which comprises: computer program code which, when run on an electronic device, causes the electronic device to perform the method 200 shown in fig. 2 or the method 600 shown in fig. 6 described above.

From the foregoing description of the embodiments, those skilled in the art will appreciate that, for convenience and brevity of description, only the above-described division of functional modules is illustrated. In practical applications, the above-mentioned functions may be distributed by different functional modules according to the need, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

It should be understood that the apparatus and processes disclosed in the several embodiments provided by the present application may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical functional division. There may be additional divisions of actual implementations, for example, multiple units or components may be combined or may be integrated into another apparatus. In addition, some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate. The component shown as a unit may be one physical unit or a plurality of physical units. I.e. can be located in one place or can be distributed over a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of this solution.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit; or may exist physically alone; it is also possible that part of the units are integrated in one unit, part of the units being physically present separately. The integrated units may be implemented in hardware or in software functional units.

If implemented as a software functional unit and sold or used as a stand-alone product, the integrated unit may be stored in a readable storage medium. With such understanding, all or part of the technical solution of the embodiments of the present application may be embodied in the form of a software product. The software product is stored in a storage medium. The software product comprises instructions for causing a device (which may be a single-chip microcomputer, chip or the like) or processor (processor) to perform all or part of the steps of the methods of the various embodiments of the application. The aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

It should be noted that all or part of the above embodiments (e.g., part or all of any feature) provided by the present application may be arbitrarily, combined or combined with each other.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of deploying a database, the method comprising:

responding to a database application submitted by a user, and acquiring data characteristics of data to be stored by a database management platform;

the database management platform determines a database deployment strategy corresponding to the database application according to the data characteristics;

the database management platform deploys the database corresponding to the database application according to the database deployment strategy.

2. The method of claim 1, wherein the database management platform determining a database deployment policy corresponding to the database application according to the data characteristics, comprises:

and the database management platform determines the types and the quantity of the databases to be deployed from a plurality of preset databases according to the data characteristics.

3. The method according to claim 2, wherein the database to be deployed includes a plurality of databases, and the database management platform determines a database deployment policy corresponding to the database application according to the data characteristics, and specifically further includes:

and the database management platform determines corresponding database cooperative services according to the data characteristics.

4. The method of claim 1, wherein the database management platform deploys the database corresponding to the database application according to the database deployment policy, comprising:

The database management platform allocates machine resources according to the database deployment strategy;

and the database management platform installs the database corresponding to the database application on the machine resource.

5. The method of claim 4, wherein the database management platform installs the database corresponding to the database application on the machine resource, comprising:

the database management platform deploys a database plug-in management program on the machine resource;

and the database management platform installs the database corresponding to the database application according to the database plug-in management program.

6. The method of any of claims 1-5, wherein the data characteristics include one or more of a number of data concurrency, a data day increment, online data aging, online data projected storage capacity, historical data aging, historical data projected storage capacity.

7. The method of claim 3, wherein the database collaboration service comprises at least one of a data service, a database routing connection program, and a distributed database middleware.

8. A database management platform, comprising:

The acquisition module is used for responding to a database application sent by a client and acquiring the data characteristics of the data to be stored sent by the client;

the determining module is used for determining a database deployment scheme according to the data characteristics;

the deployment module is used for deploying the database corresponding to the database application according to the database deployment strategy.

9. An electronic device comprising at least one processor for coupling with a memory, reading and executing instructions in the memory to implement the method of any of claims 1-7.

10. A computer readable storage medium storing computer executable instructions which, when invoked and executed by a processor, cause the processor to implement the method of any one of claims 1 to 7.