CN111897887B - Parameter configuration method, device, system, electronic equipment and storage medium - Google Patents
Parameter configuration method, device, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111897887B CN111897887B CN202010784376.1A CN202010784376A CN111897887B CN 111897887 B CN111897887 B CN 111897887B CN 202010784376 A CN202010784376 A CN 202010784376A CN 111897887 B CN111897887 B CN 111897887B
- Authority
- CN
- China
- Prior art keywords
- client
- configuration
- dimension
- configuration information
- mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 15
- 238000012423 maintenance Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000005457 optimization Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Data Mining & Analysis (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The disclosure relates to a parameter configuration method, a parameter configuration device, a parameter configuration system, an electronic device and a computer readable storage medium, and relates to the technical field of computers. The parameter configuration method comprises the following steps: responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client; if the client is in the target configuration mode, judging whether the candidate dimension comprises configuration information; and if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client. The configuration information of the Hive client can be uniformly managed, so that the operation of acquiring the Hive configuration parameters is decoupled from the Hive client.
Description
Technical Field
The present disclosure relates to the field of computer technology, and in particular, to a parameter configuration method, a parameter configuration device, a parameter configuration system, an electronic device, and a computer readable storage medium.
Background
APACHE HIVE is a Hadoop-based data warehouse tool that can map structured data files into a database table and provide structured query language (Structured Query Language, SQL) like query functions.
Currently, after APACHE HIVE is built, there are two ways to set the parameter configuration required by Hive operation. The first way is that parameters can be configured in the configuration file hive-site. Alternatively, after the client starts Hive, the parameter is set by means of a set command, where the parameter priority is higher than the parameter value in the configuration file, and if there is a conflict, the default value in the configuration file is covered.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The disclosure aims to provide a parameter configuration method, a parameter configuration device, a parameter configuration system, an electronic device and a computer readable storage medium, so as to overcome the problems of unsafe, unreliable and inflexible existing in the conventional Hive parameter configuration method, and the parameter configuration needs to be performed by relying on a Hive client machine at least to a certain extent.
Other features and advantages of the present disclosure will be apparent from the following detailed description, or may be learned in part by the practice of the invention.
According to a first aspect of the present disclosure, there is provided a parameter configuration method, including: the method is applied to the parameter configuration server and comprises the following steps: responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client; if the client is in the target configuration mode, judging whether the candidate dimension comprises configuration information; and if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client.
Optionally, determining whether the client is in the target configuration mode includes: obtaining a mode parameter value of a client; the mode parameter value is used for representing the configuration mode of the client; judging whether the mode parameter value is equal to the target mode value; if the mode parameter value is equal to the target mode value, it is determined that the client is in the target configuration mode.
Optionally, the method further comprises: if the mode parameter value is not equal to the target mode value, the mode parameter value is configured as the target mode value to switch the client to the target configuration mode.
Optionally, the candidate dimension includes a scheduling dimension and a cluster dimension, and acquiring configuration information according to the candidate dimension includes: judging whether the scheduling dimension comprises configuration information or not; if the scheduling dimension comprises configuration information, acquiring the configuration information according to the scheduling dimension; if the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information; and if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
Optionally, the method further comprises: if the client is not in the target configuration mode, configuration information is obtained from the client.
Optionally, the parameter configuration server includes a graphical user interface, and the method further includes: receiving a user operation for a graphical user interface; wherein the user operations include configuration operations for the data warehouse client; and changing the configuration information according to the user operation.
According to a second aspect of the present disclosure, there is provided a parameter configuration apparatus comprising: the mode judging module is used for responding to the parameter configuration request of the client and judging whether the client is in a target configuration mode or not; wherein the client comprises a data warehouse client; the information judging module is used for judging whether the candidate dimension comprises configuration information or not if the client is in the target configuration mode; and the first information acquisition module is used for acquiring the configuration information according to the candidate dimension if the candidate dimension comprises the configuration information so as to perform parameter configuration operation on the data warehouse client.
Optionally, the mode judging module includes a mode judging unit, configured to obtain a mode parameter value of the client; the mode parameter value is used for representing the configuration mode of the client; judging whether the mode parameter value is equal to the target mode value; if the mode parameter value is equal to the target mode value, it is determined that the client is in the target configuration mode.
Optionally, the parameter configuration device further includes a mode switching module configured to configure the mode parameter value to a target mode value if the mode parameter value is not equal to the target mode value, so as to switch the client to the target configuration mode.
Optionally, the first information acquisition module includes an information acquisition unit, configured to determine whether the scheduling dimension includes configuration information; if the scheduling dimension comprises configuration information, acquiring the configuration information according to the scheduling dimension; if the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information; and if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
Optionally, the parameter configuration device further includes a second information obtaining module, configured to obtain configuration information from the client if the client is not in the target configuration mode.
Optionally, the parameter configuration device further includes a configuration information modification module for receiving a user operation for the graphical user interface; wherein the user operations include configuration operations for the data warehouse client; and changing the configuration information according to the user operation.
According to a third aspect of the present disclosure, there is provided a parameter configuration system comprising: the parameter configuration server is used for receiving a parameter configuration request sent by the client and judging whether the client is in a target configuration mode or not; if the client is in the target configuration mode, acquiring configuration information according to the candidate dimension so as to perform parameter configuration operation on the client; the at least one client is used for sending a parameter configuration request to the parameter configuration server and receiving configuration information returned by the parameter configuration server; wherein the client comprises a data warehouse client; and the database is used for storing the configuration information of the client.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory having stored thereon computer readable instructions which when executed by the processor implement a parameter configuration method according to any one of the above.
According to a fifth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a parameter configuration method according to any one of the above.
The technical scheme provided by the disclosure can comprise the following beneficial effects:
The parameter configuration method in the exemplary embodiment of the present disclosure is applied to a parameter configuration server, and includes: responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client; if the client is in the target configuration mode, judging whether the candidate dimension comprises configuration information; and if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client. According to the parameter configuration method, on one hand, the configuration information of the data warehouse client is uniformly managed through the parameter configuration server, and the maintenance scheme of parameter configuration is standardized, so that the configuration information of the client is safer and more reliable. On the other hand, the parameter configuration process of the data warehouse client can be performed at the parameter configuration server, and the parameter configuration process is decoupled from the data warehouse client, so that parameter configuration operation does not need to be performed through the data warehouse client.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:
FIG. 1 schematically illustrates an overall architecture diagram for parameter configuration of Hive clients using the prior art;
FIG. 2 schematically illustrates a flow chart of a parameter configuration method according to an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a system architecture diagram for parameter configuration with a target configuration mode according to an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow chart of a client obtaining configuration parameters according to an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates class diagrams corresponding to a selected configuration information acquisition mode according to an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates class diagrams corresponding to dynamic management configuration information according to an exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a visual interface diagram corresponding to a target configuration mode according to an exemplary embodiment of the present disclosure;
FIG. 8 schematically illustrates a block diagram of a parameter configuration apparatus according to an exemplary embodiment of the present disclosure;
FIG. 9 schematically illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure;
Fig. 10 schematically illustrates a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the disclosed aspects may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
Referring to fig. 1, fig. 1 schematically shows an overall architecture diagram for parameter configuration of Hive clients using the prior art. The user 110 may directly perform parameter configuration operations on the Hive client, and through the Hive client 120 that completes parameter configuration, the HDFS data 130 in the distributed file system (Hadoop Distributed FILE SYSTEM, HDFS) may be obtained. By adopting the whole architecture, there are two ways to set the parameter configuration required by Hive operation at present: firstly, configuring parameters in hive-site.xml; and secondly, after the client starts the Hive, setting parameters in a set command mode, wherein the parameter priority is higher than the parameter value in the configuration file, and if conflict exists, the default value in the configuration file is covered.
However, the parameter configuration performed in the above manner has the following drawbacks: (1) unsafe and unreliable. Hive optimization parameters are obtained through a local configuration file, so that very important optimization strategies are easy to leak, and great potential safety hazards exist. In addition, the parameter is set in a set command mode, and after the client exits, the parameter setting is invalid and the client has unreliability. (2) inflexibility. Because Hive client machines are more and different clusters exist, the configuration information of Hive clients of service lines is different, so that on one hand, huge operation and maintenance cost is brought, and on the other hand, hive parameter tuning is good, and whether the test, gray scale on-line or formal on-line is carried out, the method can face huge challenges. (3) rely on Hive client machines to do so. At present, two ways of obtaining Hive configuration parameters can be completed only by logging in a Hive client machine, and the two ways are tightly coupled with the Hive client machine.
Based on this, in the present exemplary embodiment, a parameter configuration method is provided first, where the parameter configuration method of the present disclosure may be implemented by using a server, and the method described in the present disclosure may also be implemented by using a terminal device, where a terminal described in the present disclosure may include a mobile terminal and a fixed terminal. Fig. 2 schematically illustrates a schematic diagram of a parameter configuration method flow in accordance with some embodiments of the present disclosure. Referring to fig. 2, the parameter configuration method may include the steps of:
Step S210, responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client.
Step S220, if the client is in the target configuration mode, judging whether the candidate dimension comprises configuration information.
And step S230, if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client.
According to the parameter configuration method in the embodiment, on one hand, the configuration information of the data warehouse client is uniformly managed through the parameter configuration server, and the maintenance scheme of parameter configuration is standardized, so that the configuration information of the client is safer and more reliable. On the other hand, the parameter configuration process of the data warehouse client can be performed at the parameter configuration server, and the parameter configuration process is decoupled from the data warehouse client, so that parameter configuration operation does not need to be performed through the data warehouse client.
Next, a parameter configuration method in the present exemplary embodiment will be further described.
In step S210, in response to the parameter configuration request of the client, determining whether the client is in the target configuration mode; wherein the client comprises a data warehouse client.
In some example embodiments of the present disclosure, the client may be a data warehouse client, e.g., the client may be a Hive client. When APACHE HIVE is configured, parameters required by the operation of the Hive client can be configured. The parameter configuration request may be a request sent by the client to the parameter configuration server for performing configuration of the operating parameter. The target configuration mode, also known as a hole-ming optimization mode, may be a specific configuration mode for parameter configuration for the data warehouse clients. The relevant configuration of the Hive client can be uniformly managed by adopting the target configuration mode.
When the parameter configuration client receives the parameter configuration request of the client, the client can be judged whether to be in a target configuration mode or not in response to the parameter configuration request, so that corresponding parameter configuration operation is performed according to the configuration mode corresponding to the client.
The parameter configuration method can be carried out by the parameter configuration server, the data warehouse client can be configured into a target configuration mode, and the parameter configuration operation is carried out on the data warehouse server through the parameter configuration server. Referring to fig. 3, fig. 3 schematically illustrates a system architecture diagram for parameter configuration using a target configuration mode according to an exemplary embodiment of the present disclosure. After the Hive client 320 is built, if the user 310 wants to perform the parameter configuration operation on the Hive client 320, a parameter configuration request may be sent to the parameter configuration server 340 by the Hive client 320, so that the parameter configuration server 340 performs the next operation according to the configuration mode of the Hive client 320, and after the configuration of the operation parameters is completed, the Hive client 320 may obtain the HDFS data 330.
According to some exemplary embodiments of the present disclosure, a mode parameter value of a client is obtained; the mode parameter value is used for representing the configuration mode of the client; judging whether the mode parameter value is equal to the target mode value; if the mode parameter value is equal to the target mode value, it is determined that the client is in the target configuration mode. The mode parameter value may be a specific value for representing a configuration mode parameter to which the client corresponds. The target pattern value may be a value corresponding to a target pattern of the client. In the present disclosure, when the target mode value may be set to true, it indicates that the client is in the target configuration mode.
Referring to fig. 4, fig. 4 schematically illustrates a flow chart of a client obtaining configuration parameters according to an exemplary embodiment of the present disclosure. In step S410, in response to the parameter configuration request of the client, a configuration initialization operation of the Hive client may be performed. In step S420, the corresponding mode of the client may be determined first, and when the mode state of the client is determined, the mode parameter value of the client may be obtained, and the parameter indicating the corresponding configuration mode of the client may be hive_ KONGMING _ OPTIMIZE _enable. If the mode parameter value is true, the client is in a target configuration mode, and parameter configuration can be carried out on the Hive client through the parameter configuration server.
According to some exemplary embodiments of the present disclosure, if the mode parameter value is not equal to the target mode value, the mode parameter value is configured as the target mode value to switch the client to the target configuration mode. If the obtained mode parameter value is not equal to the target mode value, the mode parameter value may be configured as the target mode value, for example, when the Hive client is configured, a configuration item hive_conf_dir/Hive-site.xml file may be added, which may be a mode parameter value configured, and default value is false. Referring to FIG. 3, if the Hive client is not in the target configuration mode, configuration information is obtained locally from the Hive client 320. Only when the configuration item is true, the Hive client can be switched to the target configuration mode, and the configuration item can be configured in the following configuration mode:
<property>
<name>hive.kongming.optimize.enable</name>
<value>true</value>
</property>
Referring to fig. 3, when the configuration item configuration is completed, the configuration completed configuration item may be synchronized into database 350.
In step S220, if the client is in the target configuration mode, it is determined whether the candidate dimension includes configuration information.
In some exemplary embodiments of the present disclosure, the candidate dimension may be a dimension employed in acquiring data warehouse client configuration information. The configuration information may be information about configuration items of the data warehouse client. If the client is in the target configuration mode, it may be determined whether the candidate dimension includes configuration information, i.e., whether the configuration information of the data warehouse client may be obtained according to the candidate dimension.
In step S230, if the candidate dimension includes configuration information, the configuration information is acquired according to the candidate dimension, so as to perform parameter configuration operation on the data warehouse client.
In some example embodiments of the present disclosure, the parameter configuration operation may be an initialization parameter configuration operation performed for the data warehouse client.
If the candidate dimension comprises configuration information, corresponding configuration information can be obtained according to the candidate dimension, and initializing configuration operation is carried out on the data warehouse client according to the obtained configuration information.
According to some exemplary embodiments of the present disclosure, determining whether a scheduling dimension includes configuration information; if the scheduling dimension comprises configuration information, acquiring the configuration information according to the scheduling dimension; if the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information; and if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension. The scheduling dimension may be an acquisition dimension that determines configuration information from a service to which the client belongs. The cluster dimension may be an acquisition dimension that determines configuration information from the cluster in which the client is located.
Referring to fig. 4, in step S440, when the data warehouse client is in the target configuration mode, the configuration information of the client may be acquired by calling the target configuration mode service interface. In step S450, a judgment of a configuration information acquisition dimension is performed; in step S460, configuration information is acquired by a scheduling dimension, typically by default; in step S470, it is first determined whether the configuration information in the scheduling dimension is empty; if the configuration information in the scheduling dimension is not null, the configuration information is acquired according to the scheduling dimension, and after the configuration information is acquired through the scheduling dimension, the initialization configuration process is ended. If the configuration information in the scheduling dimension is empty, in step S480, continuing to acquire the configuration information by querying the cluster dimension, in step S490, judging whether the cluster dimension configuration information is empty, if the cluster dimension configuration information is not empty, acquiring the configuration information according to the cluster dimension, and after acquiring the corresponding configuration information, ending the initialization configuration process; if the configuration information cannot be obtained from the cluster dimension, the initialization exception is thrown.
Specifically, the process of Hive routing configuration mode may be performed by the following classes, and referring to fig. 5, fig. 5 schematically illustrates a class diagram corresponding to a selection configuration information acquisition mode according to an exemplary embodiment of the present disclosure. During Hive initialization, the Hive driver will call optimizeHive (conf) methods in KongmingOptimizeHiveUtils classes by default and determine whether the Hive client is in target configuration mode by the hive_ KONGMING _ OPTIMIZE _enable parameter. If the Hive client is in the target configuration mode, a sendPostByHttpURLConnection (kongmingOptimizeUrl, baos, timeout, retry) method of HttpUtil can be called, the acquired default configuration message is sent to the parameter configuration server in a String type manner, and the Hive parameters configured by the parameter configuration server are returned; wherein baos may represent a configuration message, timeout may represent a timeout delay, retry may represent a retry number. After the Hive client acquires the corresponding configuration items, the Hive client traverses the configuration information and sets the configuration information into the conf object in sequence, so that the Hive client can be started safely.
Referring to fig. 6, fig. 6 schematically illustrates class diagrams corresponding to dynamic management configuration information according to an exemplary embodiment of the present disclosure. After the parameter configuration server receives the request configuration signal sent by the Hive client, the optimization method optimizeHiveConf (hiveConfigXmlStr) of HiveConfController class is called. At this time, the message information sent from the client may be sent to addProperties (hiveConfigXmlStr, hiveConf) method in HiveConfService class and getOptimizerConfMap (paramMap) method in OptimizerConf class. The implementation class of getOptimizerConfMap is MySqlOptimizerConf, mySqlOptimizerConf, by parsing the task dimension in the parameter, first reads the configuration information from the cache, if the corresponding configuration information does not exist in the cache, then the corresponding configuration information will be continuously read from the database, for example, referring to fig. 3, the corresponding configuration information may be read from the Mysql database 350, and the cache is updated. Storing Hive client related configuration information in database 350 ensures that its configuration information is persistent and consistent.
According to some exemplary embodiments of the present disclosure, configuration information is obtained from a client if the client is not in a target configuration mode. Referring to fig. 4, in step S430, if the Hive client is not in the target configuration mode, the profile information may be acquired locally from the Hive client.
According to some exemplary embodiments of the present disclosure, a user operation for a graphical user interface is received; wherein the user operations include configuration operations for the data warehouse client; and changing the configuration information according to the user operation. The graphical user interface may be an interface provided to a user for relevant operation of the configuration item. The user can perform operations such as inquiring, editing, deleting and the like through the graphical user interface. Referring to fig. 7, fig. 7 schematically illustrates a visual interface diagram corresponding to a target configuration mode according to an exemplary embodiment of the present disclosure. The graphic user interface can comprise configuration items such as a technology stack, a cluster name, a scheduling number and the like, and a user can inquire optional parameters of the configuration items through the visual interface and acquire configuration information or add the configuration information according to the selected parameters. The user can perform configuration operations on each listed configuration item, for example, the user can edit and delete related configuration items through a visual interface, so as to change the configuration information of the Hive client.
In summary, the parameter configuration method disclosed by the disclosure is applied to a parameter configuration server, and includes: responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client; if the client is in the target configuration mode, judging whether the candidate dimension comprises configuration information; and if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client. According to the parameter configuration method, on one hand, the configuration information of the data warehouse client is uniformly managed through the parameter configuration server, and the maintenance scheme of parameter configuration is standardized, so that the configuration information of the client is safer and more reliable. On the other hand, the parameter configuration process of the data warehouse client can be performed at the parameter configuration server, and the parameter configuration process is decoupled from the data warehouse client, so that parameter configuration operation does not need to be performed through the data warehouse client. On the other hand, through the visual graphical user interface, the parameter configuration process can be more flexible, whether testing, gray scale online, formal online and the like can be dynamically configured through the visual interface, the operation flexibility can be increased, the operation and maintenance cost can be reduced, and the Hive development efficiency can be improved.
It should be noted that although the steps of the method of the present invention are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in that particular order or that all of the illustrated steps be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
In addition, in the present exemplary embodiment, a parameter configuration apparatus is also provided. Referring to fig. 8, the parameter configuration apparatus 800 may include: a mode determination module 810, an information determination module 820, and a first information acquisition module 830.
Specifically, the mode determining module 810 is configured to determine, in response to a parameter configuration request of the client, whether the client is in a target configuration mode; wherein the client comprises a data warehouse client; the information judging module 820 is configured to judge whether the candidate dimension includes configuration information if the client is in the target configuration mode; the first information obtaining module 830 is configured to obtain configuration information according to the candidate dimension if the candidate dimension includes the configuration information, so as to perform parameter configuration operation on the data warehouse client.
The parameter configuration device 800 can perform unified management on the configuration information of the data warehouse client through the parameter configuration server, and standardizes the maintenance scheme of parameter configuration, so that the configuration information of the client is safer and more reliable; and the parameter configuration process of the data warehouse client is decoupled from the data warehouse client, so that the parameter configuration operation does not need to be performed through the data warehouse client.
In one exemplary embodiment of the present disclosure, the mode determining module includes a mode determining unit for obtaining a mode parameter value of the client; the mode parameter value is used for representing the configuration mode of the client; judging whether the mode parameter value is equal to the target mode value; if the mode parameter value is equal to the target mode value, it is determined that the client is in the target configuration mode.
In one exemplary embodiment of the present disclosure, the parameter configuration apparatus further includes a mode switching module for configuring the mode parameter value to a target mode value to switch the client to the target configuration mode if the mode parameter value is not equal to the target mode value.
In an exemplary embodiment of the present disclosure, the first information acquisition module includes an information acquisition unit for determining whether the scheduling dimension includes configuration information; if the scheduling dimension comprises configuration information, acquiring the configuration information according to the scheduling dimension; if the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information; and if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
In an exemplary embodiment of the present disclosure, the parameter configuration apparatus further includes a second information acquisition module for acquiring configuration information from the client if the client is not in the target configuration mode.
In an exemplary embodiment of the present disclosure, the parameter configuration apparatus further includes a configuration information modification module for receiving a user operation for the graphical user interface; wherein the user operations include configuration operations for the data warehouse client; and changing the configuration information according to the user operation.
Further, there is also provided a parameter configuration system, referring to fig. 3, the parameter configuration system 300 may include: at least one of the clients 320, the parameter configuration server 340, and the database 350.
The parameter configuration server 340 is configured to receive a parameter configuration request sent by a client, and determine whether the client is in a target configuration mode; if the client is in the target configuration mode, acquiring configuration information according to the candidate dimension so as to perform parameter configuration operation on the client; the at least one client 320 is configured to send a parameter configuration request to the parameter configuration server and receive configuration information returned by the parameter configuration server; wherein the client comprises a data warehouse client; database 350 is used to store configuration information for clients.
The parameter configuration system 300 can perform unified management on configuration information of the data warehouse client through the parameter configuration server, and the maintenance scheme of parameter configuration is standardized, so that the parameter configuration of the client can be safer and more reliable; and, the configuration process of the data warehouse client may be performed at the parameter configuration server, and the parameter configuration operation may be decoupled from the data warehouse client.
The specific details of the virtual module of each parameter configuration apparatus are described in detail in the corresponding parameter configuration method, so that they will not be described in detail herein.
It should be noted that although in the above detailed description several modules or units of the parameter configuration apparatus are mentioned, this division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 900 according to such an embodiment of the invention is described below with reference to fig. 9. The electronic device 900 shown in fig. 9 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 9, the electronic device 900 is embodied in the form of a general purpose computing device. Components of electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one storage unit 920, a bus 930 connecting the different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.
Wherein the storage unit stores program code that is executable by the processing unit 910 such that the processing unit 910 performs steps according to various exemplary embodiments of the present invention described in the above-described "exemplary methods" section of the present specification.
The storage unit 920 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 921 and/or cache memory 922, and may further include Read Only Memory (ROM) 923.
The storage unit 920 may include a program/utility 924 having a set (at least one) of program modules 925, such program modules 925 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 930 may represent one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 900 may also communicate with one or more external devices 970 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 900, and/or any device (e.g., router, modem, etc.) that enables the electronic device 900 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 950. Also, electronic device 900 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 over the bus 930. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 900, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the "exemplary methods" section of this specification, when said program product is run on the terminal device.
Referring to fig. 10, a program product 1000 for implementing the above-described method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
Furthermore, the above-described drawings are only schematic illustrations of processes included in the method according to the exemplary embodiment of the present invention, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (9)
1. The parameter configuration method is characterized by being applied to a parameter configuration server and comprising the following steps:
Responding to a parameter configuration request of a client, and judging whether the client is in a target configuration mode; wherein the client comprises a data warehouse client;
If the client is in a target configuration mode, judging whether a candidate dimension comprises configuration information, wherein the candidate dimension comprises a scheduling dimension and a cluster dimension, the scheduling dimension is an acquisition dimension for determining the configuration information according to a service to which the client belongs, and the cluster dimension is an acquisition dimension for determining the configuration information according to a cluster in which the client is located;
if the candidate dimension comprises configuration information, acquiring the configuration information according to the candidate dimension so as to perform parameter configuration operation on the data warehouse client;
the obtaining the configuration information according to the candidate dimension includes:
Judging whether the scheduling dimension comprises the configuration information or not;
If the scheduling dimension comprises the configuration information, acquiring the configuration information according to the scheduling dimension;
If the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information;
And if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
2. The method of claim 1, wherein said determining whether the client is in a target configuration mode comprises:
Obtaining a mode parameter value of the client; wherein the mode parameter value is used for representing a configuration mode of the client;
Judging whether the mode parameter value is equal to a target mode value;
and if the mode parameter value is equal to a target mode value, determining that the client is in the target configuration mode.
3. The method according to claim 2, wherein the method further comprises:
and if the mode parameter value is not equal to the target mode value, configuring the mode parameter value as the target mode value so as to switch the client to a target configuration mode.
4. The method according to claim 1, wherein the method further comprises:
and if the client is not in the target configuration mode, acquiring the configuration information from the client.
5. The method according to any one of claims 1 to 4, wherein the parameter configuration server comprises a graphical user interface, the method further comprising:
Receiving a user operation for the graphical user interface; wherein the user operation comprises a configuration operation for the data warehouse client;
and changing the configuration information according to the user operation.
6. A parameter configuration apparatus, comprising:
The mode judging module is used for responding to the parameter configuration request of the client and judging whether the client is in a target configuration mode or not; wherein the client comprises a data warehouse client;
The information judging module is used for judging whether the candidate dimension comprises configuration information or not if the client is in a target configuration mode, wherein the candidate dimension comprises a scheduling dimension and a cluster dimension, the scheduling dimension is an acquisition dimension for determining the configuration information according to the service to which the client belongs, and the cluster dimension is an acquisition dimension for determining the configuration information according to the cluster in which the client is located;
the first information acquisition module is used for acquiring configuration information according to the candidate dimension if the candidate dimension comprises the configuration information so as to perform parameter configuration operation on the data warehouse client;
the first information acquisition module is further configured to determine whether the scheduling dimension includes the configuration information;
If the scheduling dimension comprises the configuration information, acquiring the configuration information according to the scheduling dimension;
If the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information;
And if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
7. A parameter configuration system, comprising:
the parameter configuration server is used for receiving a parameter configuration request sent by the client and judging whether the client is in a target configuration mode or not; if the client is in a target configuration mode, configuration information is acquired according to candidate dimensions, so that parameter configuration operation is carried out on the client, wherein the candidate dimensions comprise a scheduling dimension and a cluster dimension, the scheduling dimension is the acquisition dimension for determining the configuration information according to the service to which the client belongs, and the cluster dimension is the acquisition dimension for determining the configuration information according to the cluster in which the client is located;
At least one client for sending the parameter configuration request to the parameter configuration server and receiving configuration information returned by the parameter configuration server; wherein the client comprises a data warehouse client;
The database is used for storing the configuration information of the client;
The parameter configuration server is further configured to determine whether the scheduling dimension includes the configuration information;
If the scheduling dimension comprises the configuration information, acquiring the configuration information according to the scheduling dimension;
If the scheduling dimension does not include the configuration information, judging whether the cluster dimension includes the configuration information;
And if the cluster dimension comprises the configuration information, acquiring the configuration information according to the cluster dimension.
8. An electronic device, comprising:
A processor; and
A memory having stored thereon computer readable instructions which, when executed by the processor, implement the parameter configuration method according to any one of claims 1 to 5.
9. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the parameter configuration method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784376.1A CN111897887B (en) | 2020-08-06 | 2020-08-06 | Parameter configuration method, device, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784376.1A CN111897887B (en) | 2020-08-06 | 2020-08-06 | Parameter configuration method, device, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111897887A CN111897887A (en) | 2020-11-06 |
| CN111897887B true CN111897887B (en) | 2024-09-20 |
Family
ID=73246000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010784376.1A Active CN111897887B (en) | 2020-08-06 | 2020-08-06 | Parameter configuration method, device, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111897887B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434462A (en) * | 2021-06-21 | 2021-09-24 | 北京金山云网络技术有限公司 | HDFS data transfer method and device, storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094472A (en) * | 2007-07-26 | 2007-12-26 | 北京立通无限科技有限公司 | Method for configuring parameters of software in mobile terminal, mobile terminal, and server |
| CN101442469A (en) * | 2007-11-22 | 2009-05-27 | 中国移动通信集团公司 | Method, system and apparatus for down distributing configuration data |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10581686B2 (en) * | 2017-03-02 | 2020-03-03 | Schweitzer Engineering Laboratories, Inc. | Automatic configuration in a distributed network |
| CN108282363B (en) * | 2017-12-30 | 2021-03-12 | 京信通信系统（中国）有限公司 | Remote configuration method, device and system for CPE and ACS |
| CN110825425B (en) * | 2018-08-09 | 2024-09-20 | 北京京东尚科信息技术有限公司 | Configuration data management method and device, electronic equipment and storage medium |
-
2020
- 2020-08-06 CN CN202010784376.1A patent/CN111897887B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094472A (en) * | 2007-07-26 | 2007-12-26 | 北京立通无限科技有限公司 | Method for configuring parameters of software in mobile terminal, mobile terminal, and server |
| CN101442469A (en) * | 2007-11-22 | 2009-05-27 | 中国移动通信集团公司 | Method, system and apparatus for down distributing configuration data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111897887A (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11582321B2 (en) | Service request processing method and related apparatus | |
| CN109905431B (en) | Message processing method and system, storage medium and electronic equipment | |
| CN111626602B (en) | Service processing method, service processing device, storage medium and electronic equipment | |
| CN109669787B (en) | Data transmission method and device, storage medium and electronic equipment | |
| CN113760638A (en) | Log service method and device based on kubernets cluster | |
| CN111897887B (en) | Parameter configuration method, device, system, electronic equipment and storage medium | |
| CN114422436B (en) | Gateway, gateway control method, gateway control device, electronic equipment and storage medium | |
| CN116668402A (en) | Intelligent cloud box access method, device, equipment and storage medium | |
| CN109873861B (en) | Cross-block chain node interaction method and device, storage medium and electronic equipment | |
| CN114338820A (en) | Network access method, device, electronic equipment and storage medium | |
| US20220370912A1 (en) | Game Development Method And Apparatus, Game Running Method And Apparatus, And Electronic Device | |
| CN114301789B (en) | Data transmission method and device, storage medium and electronic equipment | |
| CN113886353B (en) | Data configuration recommendation method and device for hierarchical storage management software and storage medium | |
| CN115080113A (en) | Item code detection method and device, readable storage medium and electronic equipment | |
| CN113722007B (en) | Configuration method, device and system of VPN branch equipment | |
| CN114090514A (en) | Log retrieval method and device for distributed system | |
| CN111158645B (en) | System and method for providing an integrated development environment | |
| CN110716915A (en) | Method and device for operating database, electronic equipment and storage medium | |
| CN111324374A (en) | Application program registration method and device based on application performance management system | |
| CN112187842A (en) | Local area network data processing system and local area network data processing method | |
| CN113014409B (en) | Networking equipment configuration method, device, equipment and storage medium | |
| CN111314457B (en) | Method and device for setting virtual private cloud | |
| CN116360710B (en) | Data storage method applied to server cluster, electronic device and readable medium | |
| CN112765056B (en) | Method, system, equipment and medium for reserving storage cluster LUN | |
| CN114281350B (en) | Scene adaptation method based on embedded terminal, embedded terminal and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |