CN113268355A - Database connection method and device for distributed cluster - Google Patents
Database connection method and device for distributed cluster Download PDFInfo
- Publication number
- CN113268355A CN113268355A CN202110711273.7A CN202110711273A CN113268355A CN 113268355 A CN113268355 A CN 113268355A CN 202110711273 A CN202110711273 A CN 202110711273A CN 113268355 A CN113268355 A CN 113268355A
- Authority
- CN
- China
- Prior art keywords
- connection
- database
- mysql
- program
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013468 resource allocation Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000007726 management method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database connection method and a database connection device for a distributed cluster, wherein the method comprises the following steps: according to the submitted program, at least one server resource required by the program operation is distributed in the distributed cluster; establishing connection with the mysql database on all server nodes required by program operation; and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection with the mysql database corresponding to the created server node. The connection with the mysql database is established on all server nodes required by program operation in advance, and the connection is directly called during operation. The invention is applied to the field of distributed technology.
Description
Technical Field
The present invention relates to the field of distributed technologies, and in particular, to a database connection method and apparatus for a distributed cluster.
Background
The database is accessed in the distributed cluster, when the task is distributed to any node in the distributed cluster, a database connection is always required to be created again on the node server, and after the task is executed each time, the connection is closed, the task is redistributed to a random node next time, a new database connection is created, and the connection is frequent. Each node of each task needs to establish connection, and the number of the connections is increased; and the connection is disconnected after one task is finished, the database still needs to be repeatedly connected when the database is distributed to the server next time, and the program execution is probably connected for thousands of times, so that the repeated connection is executed by the program, the working efficiency is reduced, and a large burden is caused on the database access.
Disclosure of Invention
The embodiment of the invention provides a database connection method of a distributed cluster, which is used for improving the working efficiency and reducing the pressure of a database, and comprises the following steps:
according to the submitted program, at least one server resource required by the program operation is distributed in the distributed cluster;
establishing connection with the mysql database on all server nodes required by program operation;
and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection which corresponds to the created server node and is connected with the mysql database.
The database connection method of the distributed cluster in the specific embodiment further includes:
after the program runs and before the server resources are released, all the created connections with the mysql database are closed.
In another embodiment, the method for connecting a database of a distributed cluster further includes:
and configuring configuration parameters connected with the mysql database in the distributed cluster.
In specific implementation, when each task of program operation executes mysql operation on a corresponding server node, the connection between the created server node and the mysql database is called, and the connection comprises the following steps:
dividing program execution into a plurality of tasks;
according to each task, determining a server IP for executing each task;
and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
The embodiment of the present invention further provides a database connection device for a distributed cluster, so as to improve the working efficiency and reduce the pressure of the database, where the device includes:
the resource allocation module is used for allocating at least one server resource required by program operation in the distributed cluster according to the submitted program;
the connection establishing module is used for establishing connection with the mysql database on all server nodes required by program operation;
and the connection calling module is used for calling the connection between each created server node and the mysql database when each task operated by the program executes the mysql operation on the corresponding server node.
The database connection apparatus of a distributed cluster in the specific embodiment further includes:
and the connection closing module is used for closing all the established connections with the mysql database after the program operation is finished and before the server resources are released.
In another specific embodiment, the apparatus for connecting a database of a distributed cluster further includes:
and the configuration module is used for configuring configuration parameters connected with the mysql database in the distributed cluster.
Specifically, the connection invoking module is specifically configured to:
dividing program execution into a plurality of tasks;
according to each task, determining a server IP for executing each task;
and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the database connection method of the distributed cluster when executing the computer program.
An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing the database connection method of the distributed cluster.
In the embodiment of the invention, at least one server resource required by program operation is distributed in the distributed cluster according to the submitted program; establishing connection with the mysql database on all server nodes required by program operation; and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection with the mysql database corresponding to the created server node. The connection with the mysql database is established on all server nodes required by program operation in advance, and the connection is directly called during operation.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a database connection method of a distributed cluster in an embodiment of the present invention.
Fig. 2 is a schematic diagram of a method for implementing step 103 in the embodiment of the present invention.
Fig. 3 is a schematic diagram of a database connection method of a distributed cluster in an embodiment of the present invention.
Fig. 4 is a schematic diagram of a database connection method of a distributed cluster in another embodiment of the present invention.
Fig. 5 is a schematic diagram of a database connection device of a distributed cluster in an embodiment of the present invention.
Fig. 6 is a schematic diagram of a database connection device of a distributed cluster according to an embodiment of the present invention.
Fig. 7 is a schematic diagram of a database connection device of a distributed cluster according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For a better understanding of the embodiments of the present invention, the terms of art to which the embodiments of the present invention relate will first be explained:
mysql: mysql is an open source relational database management system (RDBMS) that uses the most common database management language, Structured Query Language (SQL), for database management. mysql is open source code and therefore can be downloaded by anyone with permission from General Public License and modified as needed for personalization. mysql is of great interest because of its speed, reliability and adaptability. Most people believe mysql is the best choice to manage content without requiring transactional processing.
Database connection: the database Connection is responsible for the communication between databases, and the SQL execution and the transaction processing are all performed in a certain Connection environment. A Statement to execute SQL may be generated.
Spark: apache Spark is a fast, general-purpose computing engine designed specifically for large-scale data processing. Spark is a universal parallel framework similar to Hadoop MapReduce and derived from UC Berkeley AMP lab (AMP labs of Berkeley university, California), Spark has the advantages of Hadoop MapReduce; but different from MapReduce, Job intermediate output results can be stored in a memory, so that HDFS reading and writing are not needed, and Spark can be better suitable for MapReduce algorithms which need iteration, such as data mining, machine learning and the like.
Distributed clustering: the basic principle of the distributed platform is to utilize multiple computers to cooperatively solve the problem that cannot be solved by a single computer, and the problem is often caused by the fact that the data size exceeds the storage capacity of a single computer system or the calculation amount exceeds the calculation capacity of the single computer system. The processing problem of large-scale data can be well solved by utilizing the resource sharing and collaborative computing capability provided by the distributed platform. In the physical structure of the distributed platform, the hosts are connected through a high-speed internal network, and a distributed management system is configured on the basis of the connection, so as to externally provide various resource sharing services such as hardware sharing, software sharing, data sharing, service sharing and the like.
And (3) node: refer to a driver, an executor the like server in the Spark cluster.
The embodiment of the invention provides a database connection method of a distributed cluster, which is used for improving the working efficiency and reducing the pressure of a database, and as shown in figure 1, the method comprises the following steps:
step 101: according to the submitted program, at least one server resource required by the program operation is distributed in the distributed cluster;
step 102: establishing connection with the mysql database on all server nodes required by program operation;
step 103: and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection with the mysql database corresponding to the created server node.
As can be known from the flow shown in fig. 1, in the embodiment of the present invention, at least one server resource required for program operation is allocated in a distributed cluster according to a submitted program; establishing connection with the mysql database on all server nodes required by program operation; and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection with the mysql database corresponding to the created server node. The connection with the mysql database is established on all server nodes required by program operation in advance, and the connection is directly called during operation.
In specific implementation, after a program is submitted, at least one server resource required for program operation is allocated in the distributed cluster according to the submitted program, for example, servers A, B and E in the distributed cluster (including 11 servers a to K in total) are allocated to the currently submitted program for program execution.
After the server resources are allocated, all server nodes required by program operation are determined, and connection with the mysql database is established on all the server nodes required by the program operation and is stored on the corresponding server nodes.
After all the server nodes are created, when each task operated by the program executes mysql operation on the corresponding server node, the connection between the created server node and the mysql database is called. In a specific embodiment, as shown in fig. 2, the method includes:
step 201: dividing program execution into a plurality of tasks;
step 202: according to each task, determining a server IP for executing each task;
step 203: and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
The execution of a program is necessarily divided into a plurality of tasks to be completed, the execution of the program is divided into a plurality of tasks, each task corresponds to a server node, and a corresponding server IP is determined. When each task is allocated to a task and each task executes mysql database related operation, the created database connection is directly called, the connection is not repeatedly created, and the database connection number and resources are not wasted.
In an embodiment, in order to further save resources and avoid waste, in an embodiment, the method for connecting databases of distributed clusters, as shown in fig. 3, on the basis of fig. 1, further includes:
step 301: after the program runs and before the server resources are released, all the created connections with the mysql database are closed.
In order to establish a connection with the mysql database more quickly and better, in another embodiment, as shown in fig. 4, on the basis of fig. 1, the method further includes:
step 401: and configuring configuration parameters connected with the mysql database in the distributed cluster.
In a specific implementation process, the configuration parameter may be related parameter information such as an address, an IP port, a user name, and a password.
A specific example is given below to illustrate how embodiments of the present invention perform database connections for distributed clusters. The method is applied to database connection under a distributed system, mainly applied to distributed systems such as big data clusters and the like, and particularly applied to database connection under Spark distributed clusters.
After the Spark program is submitted and the resource allocation is completed, Spark stress is created, and meanwhile, the connection between the currently allocated driver and executor nodes and the mysql database is created and stored on the respective nodes.
When the specific task executes mysql operation at the corresponding executor, the connection which is already established at the initial stage is called, the connection is not repeatedly established, and the connection number and the resources of the database are not wasted.
The method is characterized in that the method is generated in advance at each executive node and driver node, stored on each node, and used for directly calling the existing connection without creating and closing each time, so that the processing is troublesome, the time is wasted, and the resources are wasted.
The specific process comprises the following steps:
creating a database connection: after the resource manager allocates the resources, the resources allocated for the current task are determined, and then connection with the mysql database is established on the executer and driver nodes.
Managing database connection: when the Spark jobis executed, specific tasks are distributed to the tasks, and when each task executes mysql database related operation, the established database connection is directly called.
Closing the database connection: at the end of the Spark program execution, before Spark segment closes, the established connection to the mysql database is closed.
Through the above process, the mysql database in the Spark big data cluster is not explicitly connected with the database by the code. The method avoids repeated connection in the big data cluster, and solves the problems of reduced calculation performance of big data, excessive pressure on database forgiveness and the like. And the connection problem of mysql is solved from the cluster level.
In order to realize the process, the construction of the related system comprises the following steps:
a configuration module: and configuring related configuration information of mysql connection in the cluster, and providing related parameter information of mysql connection.
A creation module: for creating a connection to the mysql database for the resource server that is currently allocated.
A management module: the method is used for uniformly managing and scheduling the connection which is established by the establishing module and is connected with the mysql database, allocating the connection which is required by the task allocation of the program and is connected with the mysql database, and allocating the connection in real time according to the current server.
By adding the connection which is uniformly managed by the cluster and is connected with the mysql database, the mysql connection does not need to be considered in the program compiling process, and the code complexity and the redundancy are reduced. The mysql connection is established at the initial stage of resource allocation, the mysql connection is closed before the program finishes releasing resources, the mysql connection is established before the program is executed and is closed after the program is finished, the connection is guaranteed to be established for all servers designed by tasks, and the connection is not wasted for servers unrelated to the tasks. And a management module of mysql connection is added to be responsible for management and use allocation of the mysql connection.
And the mysql database of the Spark big data distributed cluster is accessed more simply and conveniently. The code development difficulty when mysql is used is reduced. The performance of executing a big data task when mysql is used is improved. The pressure of mysql database connection is reduced when mysql is used by large data clusters.
The implementation of the above specific application is only an example, and the rest of the embodiments are not described in detail.
Based on the same inventive concept, embodiments of the present invention further provide a database connection apparatus for a distributed cluster, where the principle of the problem solved by the database connection apparatus for a distributed cluster is similar to that of the database connection method for a distributed cluster, so that the implementation of the database connection apparatus for a distributed cluster can refer to the implementation of the database connection method for a distributed cluster, repeated details are not described, and a specific structure is shown in fig. 5:
a resource allocation module 501, configured to allocate, according to the submitted program, at least one server resource required by the program to run in the distributed cluster;
a connection creating module 502, configured to create connections with the mysql database on all server nodes required for program operation;
and a connection calling module 503, configured to call, when each task of the program running executes the mysql operation on the corresponding server node, a connection between the created server node and the mysql database.
In a specific embodiment, the database connection apparatus of a distributed cluster shown in fig. 6 further includes, on the basis of fig. 5:
and a connection closing module 601, configured to close all created connections with the mysql database after the program runs and before the server resources are released.
In another specific embodiment shown in fig. 7, the database connection apparatus for distributed clusters further includes, based on fig. 5:
a configuration module 701, configured to configure configuration parameters connected to the mysql database in the distributed cluster.
In a specific implementation process, the connection calling module 503 is specifically configured to:
dividing program execution into a plurality of tasks;
according to each task, determining a server IP for executing each task;
and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the database connection method of the distributed cluster when executing the computer program.
An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing the database connection method of the distributed cluster.
In summary, the database connection method and apparatus for a distributed cluster provided in the embodiments of the present invention have the following advantages:
allocating at least one server resource required for program operation in the distributed cluster by the submitted program; establishing connection with the mysql database on all server nodes required by program operation; and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection with the mysql database corresponding to the created server node. The connection with the mysql database is established on all server nodes required by program operation in advance, and the connection is directly called during operation.
Although the present invention provides method steps as described in the examples or flowcharts, more or fewer steps may be included based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or client product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, apparatus (system) or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "upper", "lower", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the referred devices or elements must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Unless expressly stated or limited otherwise, the terms "mounted," "connected," and "connected" are intended to be inclusive and mean, for example, that they may be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations. It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention is not limited to any single aspect, nor is it limited to any single embodiment, nor is it limited to any combination and/or permutation of these aspects and/or embodiments. Moreover, each aspect and/or embodiment of the present invention may be utilized alone or in combination with one or more other aspects and/or embodiments thereof.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (10)
1. A database connection method of a distributed cluster is characterized by comprising the following steps:
according to the submitted program, at least one server resource required by the program operation is distributed in the distributed cluster;
establishing connection with the mysql database on all server nodes required by program operation;
and when each task operated by the program executes mysql operation on the corresponding server node, calling the connection which corresponds to the created server node and is connected with the mysql database.
2. The database connection method of a distributed cluster according to claim 1, further comprising:
after the program runs and before the server resources are released, all the created connections with the mysql database are closed.
3. The database connection method of a distributed cluster according to claim 1, further comprising:
and configuring configuration parameters connected with the mysql database in the distributed cluster.
4. The database connection method of the distributed cluster according to claim 1, wherein each task executed by the program calls a connection with the mysql database corresponding to the server node that has been created when mysql operation is executed on the corresponding server node, and the method comprises:
dividing program execution into a plurality of tasks;
according to each task, determining a server IP for executing each task;
and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
5. A database connection apparatus for a distributed cluster, comprising:
the resource allocation module is used for allocating at least one server resource required by program operation in the distributed cluster according to the submitted program;
the connection establishing module is used for establishing connection with the mysql database on all server nodes required by program operation;
and the connection calling module is used for calling the connection between each created server node and the mysql database when each task operated by the program executes the mysql operation on the corresponding server node.
6. The distributed clustered database connectivity apparatus of claim 5, further comprising:
and the connection closing module is used for closing all the established connections with the mysql database after the program operation is finished and before the server resources are released.
7. The distributed clustered database connectivity apparatus of claim 5, further comprising:
and the configuration module is used for configuring configuration parameters connected with the mysql database in the distributed cluster.
8. The apparatus for connecting databases of distributed clusters according to claim 5, wherein the connection invoking module is specifically configured to:
dividing program execution into a plurality of tasks;
according to each task, determining a server IP for executing each task;
and when each task executes mysql operation on the corresponding server node, determining the established connection with the mysql database corresponding to the server node according to the server IP of each task, and calling the connection with the mysql database.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 4 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110711273.7A CN113268355A (en) | 2021-06-25 | 2021-06-25 | Database connection method and device for distributed cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110711273.7A CN113268355A (en) | 2021-06-25 | 2021-06-25 | Database connection method and device for distributed cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113268355A true CN113268355A (en) | 2021-08-17 |
Family
ID=77235859
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110711273.7A Pending CN113268355A (en) | 2021-06-25 | 2021-06-25 | Database connection method and device for distributed cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113268355A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036031A (en) * | 2014-06-27 | 2014-09-10 | 北京航空航天大学 | Large-scale CFD parallel computing method based on distributed Mysql cluster storage |
| CN111031123A (en) * | 2019-12-10 | 2020-04-17 | 中盈优创资讯科技有限公司 | Spark task submission method, system, client and server |
| CN112231384A (en) * | 2020-10-30 | 2021-01-15 | 中国建设银行股份有限公司 | Distributed application localization transformation method and device adaptive to multiple data sources |
-
2021
- 2021-06-25 CN CN202110711273.7A patent/CN113268355A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036031A (en) * | 2014-06-27 | 2014-09-10 | 北京航空航天大学 | Large-scale CFD parallel computing method based on distributed Mysql cluster storage |
| CN111031123A (en) * | 2019-12-10 | 2020-04-17 | 中盈优创资讯科技有限公司 | Spark task submission method, system, client and server |
| CN112231384A (en) * | 2020-10-30 | 2021-01-15 | 中国建设银行股份有限公司 | Distributed application localization transformation method and device adaptive to multiple data sources |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10824474B1 (en) | Dynamically allocating resources for interdependent portions of distributed data processing programs | |
| US10310908B2 (en) | Dynamic usage balance of central processing units and accelerators | |
| US9846589B2 (en) | Virtual machine placement optimization with generalized organizational scenarios | |
| US9401835B2 (en) | Data integration on retargetable engines in a networked environment | |
| JP2021057072A (en) | Processing data from multiple sources | |
| US8984516B2 (en) | System and method for shared execution of mixed data flows | |
| US11902251B2 (en) | Socket transferring for HPC networks using kernel tracing | |
| US9256460B2 (en) | Selective checkpointing of links in a data flow based on a set of predefined criteria | |
| CN110121704B (en) | Execution method, device and system for stored procedure | |
| US20170171026A1 (en) | Configuring a cloud from aggregate declarative configuration data | |
| CN105765578B (en) | Parallel access of data in a distributed file system | |
| US20200065208A1 (en) | Model-based key performance indicator service for data analytics processing platforms | |
| US11216454B1 (en) | User defined functions for database query languages based on call-back functions | |
| US20140201371A1 (en) | Balancing the allocation of virtual machines in cloud systems | |
| Wang et al. | Actcap: Accelerating mapreduce on heterogeneous clusters with capability-aware data placement | |
| US10621389B1 (en) | Selecting platform-supported services | |
| US20220391749A1 (en) | Method and system for discovery of inference servers in a machine learning serving infrastructure | |
| Lin et al. | A configurable and executable model of Spark Streaming on Apache YARN | |
| Wu et al. | Towards big data analytics across multiple clusters | |
| Asan Baker Kanbar | Modern load balancing techniques and their effects on cloud computing | |
| Gu et al. | Characterizing job-task dependency in cloud workloads using graph learning | |
| CN113268355A (en) | Database connection method and device for distributed cluster | |
| Galante et al. | Extending parallel programming patterns with adaptability features | |
| Higashino et al. | Attributed graph rewriting for complex event processing self-management | |
| Goncalves et al. | Flexible MapReduce Workflows for Cloud Data Analytics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |