CN112749055A - Resource consumption metering method and device, electronic equipment and storage medium - Google Patents

Resource consumption metering method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112749055A
CN112749055A CN202011585082.2A CN202011585082A CN112749055A CN 112749055 A CN112749055 A CN 112749055A CN 202011585082 A CN202011585082 A CN 202011585082A CN 112749055 A CN112749055 A CN 112749055A
Authority
CN
China
Prior art keywords
resource
task
yarn
management system
time period
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011585082.2A
Other languages
Chinese (zh)
Inventor
不公告发明人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lakala Payment Co ltd
Original Assignee
Lakala Payment Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lakala Payment Co ltd filed Critical Lakala Payment Co ltd
Priority to CN202011585082.2A priority Critical patent/CN112749055A/en
Publication of CN112749055A publication Critical patent/CN112749055A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3452—Performance evaluation by statistical analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request

Abstract

The embodiment of the disclosure discloses a resource consumption metering method, a resource consumption metering device, electronic equipment, a storage medium and a program product, wherein the method comprises the following steps: responding to the resource consumption metering request, and determining a resource use task initiated by a resource user within a preset time period; acquiring the resource consumption condition of the resource use task from a YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task; and determining the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period. According to the technical scheme, the resource consumption condition of each resource user can be counted according to the time period, and the problems of inaccurate resource consumption counting and low counting efficiency in the prior art can be solved.

Description

Resource consumption metering method and device, electronic equipment and storage medium
Technical Field
The embodiment of the disclosure relates to the technical field of big data, in particular to a resource consumption metering method and device, electronic equipment and a storage medium.
Background
In the field of big data clusters, the use of big data clusters by downstream is mainly embodied in two forms: offline analysis query and online scheduling computation. In the prior art, the occupied cost of each department is measured only by simple numbers, such as query times, scheduling script number and scheduling frequency, but the resource consumption of the cluster is greatly different by different query and calculation scripts, so that the resource consumption of each department cluster cannot be accurately measured at all, and various branches and disputes are easy to occur in the cost calculation.
Disclosure of Invention
The embodiment of the disclosure provides a resource consumption metering method and device, electronic equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a resource consumption metering method, including:
responding to the resource consumption metering request, and determining a resource use task initiated by a resource user within a preset time period;
acquiring the resource consumption condition of the resource use task from a YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
and determining the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
Further, the method further comprises:
acquiring front-end operation of the resource user on big data cluster resources;
mapping the front-end operation to the resource usage task of the YARN resource management system.
Further, mapping the front-end operation to the resource usage task of the YARN resource management system further comprises:
and establishing an incidence relation between the front-end operation and the resource using task.
Further, acquiring the resource consumption of the resource usage task from the YARN resource management system includes:
sending a request for acquiring the resource consumption condition of the resource use task to the YARN resource management system;
and receiving the resource consumption condition returned by the YARN resource management system.
Further, in response to a resource consumption metering request, determining the resource usage task initiated by a resource user within a preset time period includes:
determining the front-end operation generated by the resource user in a preset time period;
and determining the resource using task initiated by the resource using party within a preset time period according to the front-end operation and the incidence relation.
Further, determining the total consumption of the big data cluster resources by the resource user in the preset time period according to the resource consumption corresponding to the resource usage task in the preset time period, including:
determining the front-end operation generated by the resource user in the preset time period and the type of the front-end operation;
and determining the total consumption condition of the resource using task corresponding to the front-end operation on the big data cluster resources aiming at different types.
Further, the types of the front-end operations include an offline query operation and an online scheduling operation.
In a second aspect, an embodiment of the present disclosure provides a resource consumption metering method, including:
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task;
responding to a resource consumption metering request, a resource analysis server determines a resource use task initiated by a resource user within a preset time period, and acquires the resource consumption condition of the resource use task from a YARN resource management system;
and the resource analysis server determines the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
Further, the method further comprises:
the resource user equipment receives front-end operation of a user and sends the front-end operation to a resource analysis server;
the resource analysis server sends a resource request to a YARN resource management system based on the front-end operation;
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task, and the method comprises the following steps:
after receiving the resource request, the YARN resource management system creates the resource usage task corresponding to the front-end operation and allocates big data cluster resources to the resource usage task.
Further, the method further comprises:
and the resource analysis server establishes an incidence relation between the front-end operation and the resource use task.
Further, acquiring the resource consumption of the resource usage task from the YARN resource management system includes:
the resource analysis server sends a request for acquiring the resource consumption condition of the resource use task to the YARN resource management system;
and the YARN resource management system returns the resource consumption condition of the resource use task to the resource analysis server.
Further, in response to the resource consumption metering request, the resource analysis server determines the resource usage task initiated by the resource using party within a preset time period, including:
and the resource analysis server determines the front-end operation generated by the resource user in a preset time period, and determines the resource use task initiated by the resource user in the preset time period according to the front-end operation and the incidence relation.
Further, the resource analysis server determines, according to the resource consumption condition corresponding to the resource usage task in the preset time period, a total consumption condition of the resource usage party on the big data cluster resource in the preset time period, including:
and the resource analysis server determines the front-end operation and the type of the front-end operation generated by the resource user in the preset time period, and determines the total consumption condition of the resource use task corresponding to the front-end operation on the big data cluster resources aiming at different types.
Further, the types of the front-end operations include an offline query operation and an online scheduling operation.
In a third aspect, an embodiment of the present disclosure provides a resource consumption metering device, including:
the response module is configured to respond to the resource consumption metering request and determine a resource use task initiated by a resource using party within a preset time period;
the acquisition module is configured to acquire the resource consumption condition of the resource use task from the YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
and the determining module is configured to determine the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource usage task in the preset time period.
The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.
In one possible design, the apparatus includes a memory configured to store one or more computer instructions that enable the apparatus to perform the corresponding method, and a processor configured to execute the computer instructions stored in the memory. The apparatus may also include a communication interface for the apparatus to communicate with other devices or a communication network.
In a fourth aspect, an embodiment of the present disclosure provides a resource consumption metering system, including: a YARN resource management system and a resource analysis server;
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task;
responding to a resource consumption metering request, the resource analysis server determines the resource use task initiated by a resource user within a preset time period, and acquires the resource consumption condition of the resource use task from a YARN resource management system;
and the resource analysis server determines the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
In a fifth aspect, the disclosed embodiment provides an electronic device, including a memory for storing one or more computer instructions for supporting any of the above apparatuses to perform the corresponding methods described above, and a processor configured to execute the computer instructions stored in the memory. Any of the above may also include a communication interface for communicating with other devices or a communication network.
In a sixth aspect, the disclosed embodiments provide a computer-readable storage medium for storing computer instructions for use by any of the above-mentioned apparatuses, including computer instructions for performing any of the above-mentioned methods.
In a seventh aspect, the disclosed embodiments provide a computer program product comprising computer instructions for implementing the steps of the method according to any one of the above aspects when executed by a processor.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
the technical scheme provided by the embodiment of the disclosure manages the large data cluster resources based on the YARN resource management system, allocates resources for the resource use task initiated by front-end operation, and records the resource consumption condition of the resource use task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the disclosure.
Drawings
Other features, objects, and advantages of embodiments of the disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 illustrates a flow diagram of a resource consumption metering method according to an embodiment of the present disclosure;
FIG. 2 illustrates a flow diagram of a resource consumption metering method according to another embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating an application scenario of a resource consumption metering method according to an embodiment of the present disclosure;
FIG. 4 illustrates an overall flow diagram of a resource consumption metering method according to an embodiment of the present disclosure;
FIG. 5 is a block diagram showing the construction of a resource consumption metering device according to an embodiment of the present disclosure;
FIG. 6 shows a block diagram of a resource consumption metering system according to an embodiment of the present disclosure;
FIG. 7 is a schematic block diagram of a computer system suitable for implementing a resource consumption metering method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the disclosed embodiments will be described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the disclosed embodiments, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, behaviors, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or combinations thereof may be present or added.
It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
The technical scheme provided by the embodiment of the disclosure manages the large data cluster resources based on the YARN resource management system, allocates resources for the resource use task initiated by front-end operation, and records the resource consumption condition of the resource use task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
Fig. 1 shows a flowchart of a resource consumption metering method according to an embodiment of the present disclosure, and as shown in fig. 1, the resource consumption metering method includes the following steps S101 to S103:
in step S101, in response to a resource consumption metering request, determining a resource usage task initiated by a resource user within a preset time period;
in step S102, acquiring the resource consumption of the resource usage task from the YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
in step S103, according to a resource consumption condition of the resource usage task within the preset time period, determining a total consumption condition of the resource usage party on the big data cluster resource within the preset time period.
As mentioned above, in the field of big data clusters, the use of big data clusters downstream is mainly embodied in two forms: offline analysis query and online scheduling computation. In the prior art, the occupied cost of each department is measured only by simple numbers, such as query times, scheduling script number and scheduling frequency, but the resource consumption of the cluster is greatly different by different query and calculation scripts, so that the resource consumption of each department cluster cannot be accurately measured at all, and various branches and disputes are easy to occur in the cost calculation.
In view of the above problem, in this embodiment, a resource consumption metering method is proposed, which is used for resource allocation and coordinated management of resource usage tasks initiated by various resource users of the YARN resource management system according to the usage of large data cluster resources by different resource users. The embodiment of the disclosure manages the big data cluster resources based on the YARN resource management system, allocates resources for the resource usage task initiated by the front-end operation, and records the resource consumption condition of the resource usage task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
In an embodiment of the present disclosure, the data processing method may be applied to measure resource usage of each resource user on the resource analysis server.
In an embodiment of the present disclosure, a YARN resource management system may be established for big data cluster resources, which may include, but are not limited to, cluster resources under a Hadoop system.
The Apache Hadoop YARN (hereinafter referred to as YARN) is a general resource management system, and can provide uniform resource management and scheduling service for applications. YARN includes a global Resource Manager (RM) and several Application Masters (AM) for tasks. For each task using big data cluster resources, YARN creates an Application Master (AM), and the Application Master (AM) is responsible for coordinating resources from Resource Manager, and monitors the execution of container and Resource usage (Resource allocation of CPU, memory, etc.) by Node Manager. The Node Manager manages each Node in a YARN cluster. The Node Manager provides services for each Node in the cluster, from overseeing lifetime management of a container to monitoring resources and tracking Node health. The NodeManager manages the abstract containers that represent the resources for each node that are available for a particular task.
To use the YARN cluster resources, the task initiator first initiates a Resource use request to the YARN Resource management system, the Resource Manager negotiates the necessary resources of a container, and starts an ApplicationMaster to manage the task receiving the request. Using a resource request protocol, the ApplicationMaster negotiates the resource containers on each node for use by the tasks. While executing a task, the ApplicationMaster monitors the resource container until the task execution is complete. When the task is completed, the ApplicationMaster logs out the resource container occupied by the ApplicationMaster from the ResourceManager, and the execution cycle is ended.
The resource consumers may be, for example, people involved in various departments of the enterprise. The related personnel can use the big data cluster resources through front-end operation. The resource analysis server can realize the analysis and management of the use condition of each department on the large data cluster resources by mapping the front-end operation of the related personnel to the task in the YARN resource management system.
In an implementation manner of the present disclosure, when the resource analysis server counts the usage of the big data cluster resource for each resource user, for example, each department, the resource analysis server may first determine a resource usage task initiated by the resource user within a preset time period, and obtain the resource consumption of the resource usage task from the YARN resource management system. The YARN resource management system is used for coordinating and managing the resource using process of the resource using task and distributing cluster resources in the executing process of the resource using task until the resource using task is executed completely.
In an embodiment of the present disclosure, the resource usage task may be initiated by a resource user through a front-end operation such as data query, statistical analysis calculation, and the like using a big data cluster resource. The resource user may be a person or an organization having a usage right for the large data cluster resource. The resource analysis server is used for managing the use of the big data cluster resources by the resource users, periodically counting the consumption condition of the big data cluster resources by the resource users, and further giving a report of the resource consumption condition of each resource user to the big data resources in each preset time period.
In an embodiment of the present disclosure, the preset time period may be day, week, month, year, etc.
In an embodiment of the present disclosure, big data cluster resources may include, but are not limited to, new resource types such as memory, CPU, similar graphics processing unit, or dedicated processing device. Resource consumption may include, but is not limited to, occupied memory size, CPU occupancy, memory and CPU usage duration, and the like.
In an embodiment of the present disclosure, the resource consumption metering request may be triggered by a statistical person or periodically.
The technical scheme provided by the embodiment of the disclosure manages the large data cluster resources based on the YARN resource management system, allocates resources for the resource use task initiated by front-end operation, and records the resource consumption condition of the resource use task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
In an embodiment of the present disclosure, the method further includes:
acquiring front-end operation of the resource user on big data cluster resources;
mapping the front-end operation to the resource usage task of the YARN resource management system.
In this embodiment, a user or other resource user may use the big data cluster resource through a front-end operation. During the use process, a user or other resource users can still operate in the modes of executing scripts, writing data access statements and the like through the front-end interface. After detecting the front-end operation, the resource analysis server may map the front-end operation to the YARN resource management system in the background, that is, create a corresponding resource usage task based on the front-end operation, and send the created resource usage task to the YARN resource management system to request resource allocation and execute corresponding task content. After the YARN resource management system receives the request, an AM may be created for the resource usage task, and the AM may perform resource coordination and allocation on the resource usage task.
In an embodiment of the present disclosure, after the step of mapping the front-end operation to the resource usage task of the YARN resource management system, the pacifying further may include the steps of:
and establishing an incidence relation between the front-end operation and the resource using task.
In this embodiment, the resource analysis server maps the front-end operation to the resource usage task of the YARN resource management system by capturing the front-end operation initiated by the resource using party at the front-end. After the YARN resource management system creates the resource usage task for the operation of using the big data cluster resource by the resource user, the ID of the resource usage task is returned, and the resource analysis server may establish an association relationship between the task ID and the front-end operation initiated by the resource user, so that the resource consumption condition may be subsequently acquired from the YARN resource management system based on the task ID associated with the front-end operation initiated by the resource user.
In an embodiment of the present disclosure, the step S102, namely the step of acquiring the resource consumption condition of the resource usage task from the YARN resource management system, may further include the following steps:
sending a request for acquiring the resource consumption condition of the resource use task to the YARN resource management system;
and receiving the resource consumption condition returned by the YARN resource management system.
In this embodiment, for a resource user who needs to perform resource consumption statistics, the resource analysis server first determines a resource usage task initiated by the resource user within a preset time period, and then requests the YARN resource management system to obtain a resource consumption condition of the resource usage task. In the actual implementation process, the YARN command supported by the YARN resource management system may be used to obtain the resource consumption of the resource usage task, count the resource usage since the resource usage task was started, and count the current real-time resource usage of the resource usage task. The following commands may be used, for example, to obtain resource usage since the resource usage task started:
yann application-status $ application Id; \\ applicationId is the task identification of the resource usage task in the YARN resource management system.
And the current real-time resource occupation condition of the resource use task can be obtained through the used resources parameter returned by the yarnScheduler interface in the YARN resource management system.
In an embodiment of the present disclosure, the step S101, that is, the step of determining the resource usage task initiated by the resource user within a preset time period, may further include the following steps:
determining the front-end operation generated by the resource user in a preset time period;
and determining the resource using task initiated by the resource using party within a preset time period according to the front-end operation and the incidence relation.
In this embodiment, when performing the resource consumption statistics for a certain or some resource users, the front-end operations executed by the resource users within a preset time period may be counted first, and then the resource usage tasks associated with the front-end operations are determined according to the association relationship. The resource consumption of the resource using tasks in the preset time period is the resource consumption of the resource using party in the preset time period.
In an embodiment of the present disclosure, in step S101, that is, the step of determining the total consumption condition of the resource consumer for the big data cluster resource in the preset time period according to the resource consumption condition corresponding to the resource usage task in the preset time period, may further include the following steps:
determining the front-end operation generated by the resource user in the preset time period and the type of the front-end operation;
and determining the total consumption condition of the resource using task corresponding to the front-end operation on the big data cluster resources aiming at different types.
In order to be able to intuitively summarize resource consumption of each resource user within a preset time period, in this embodiment, classification statistics may be performed based on different types of front-end operations initiated by the resource users. That is, for the front-end operations of the same type, the resource consumption conditions of the resource usage tasks corresponding to the front-end operations within the preset time period are counted. And for different types of front-end operations, the resource consumption conditions corresponding to the different types of front-end operations can be obtained through statistics.
In an embodiment of the present disclosure, the types of front-end operations include an offline query operation and an online scheduling operation.
It can be understood that the online scheduling of big data cluster resources and the offline query analysis using big data cluster resources are two different resource usage modes, and the resource usage costs to be spent by the two different usage modes are different. Offline query analysis can generally be used when big data cluster resources are not busy, and may not be in real time, so the resource use cost is low, while online scheduling of resources requires real-time results, so the use cost is high. Therefore, the embodiment of the present disclosure may count the resource consumption of each resource user in a preset time period for the two operation types.
Fig. 2 shows a flowchart of a resource consumption metering method according to an embodiment of the present disclosure, and as shown in fig. 2, the resource consumption metering method includes the following steps S201 to S203:
in step S201, the YARN resource management system creates a resource usage task and allocates big data cluster resources to the resource usage task;
in step S202, in response to the resource consumption metering request, the resource analysis server determines the resource usage task initiated by the resource user within a preset time period, and obtains the resource consumption condition of the resource usage task from the YARN resource management system;
in step S203, the resource analysis server determines a total consumption condition of the resource consumer for the big data cluster resource in the preset time period according to a resource consumption condition corresponding to the resource usage task in the preset time period.
As mentioned above, in the field of big data clusters, the use of big data clusters downstream is mainly embodied in two forms: offline analysis query and online scheduling computation. In the prior art, the occupied cost of each department is measured only by simple numbers, such as query times, scheduling script number and scheduling frequency, but the resource consumption of the cluster is greatly different by different query and calculation scripts, so that the resource consumption of each department cluster cannot be accurately measured at all, and various branches and disputes are easy to occur in the cost calculation.
In view of the above problem, in this embodiment, a resource consumption metering method is proposed, which is used for resource allocation and coordinated management of resource usage tasks initiated by various resource users of the YARN resource management system according to the usage of large data cluster resources by different resource users. The embodiment of the disclosure manages the big data cluster resources based on the YARN resource management system, allocates resources for the resource usage task initiated by the front-end operation, and records the resource consumption condition of the resource usage task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
In an embodiment of the present disclosure, the resource consumption metering method may be applied to perform statistical metering on the resource usage by using a system including a YARN resource management system and a resource analysis server.
In an embodiment of the present disclosure, the YARN resource management system is configured to manage resource usage of each cluster node in the big data cluster resource, and schedule and allocate resources for tasks using the resources. The YARN resource management system establishes a resource use task for the request of the resource user and allocates corresponding resources to the resource use task, in order to uniformly manage and schedule the big data cluster resources for the use of each resource user, in order to ensure that the big data cluster resources are used by the resource user.
The resource analysis server can periodically acquire the resource consumption condition of each resource use task from the YARN resource management system, and count the total consumption condition of one or more resource users to the big data cluster resources within a preset time period according to the acquired resource consumption request.
In an embodiment of the present disclosure, the method further includes:
the resource user equipment receives front-end operation of a user and sends the front-end operation to a resource analysis server;
the resource analysis server sends a resource request to a YARN resource management system based on the front-end operation;
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task, and the method comprises the following steps:
after receiving the resource request, the YARN resource management system creates the resource usage task corresponding to the front-end operation and allocates big data cluster resources to the resource usage task.
In order to be able to quickly and accurately count the usage of the big data cluster resources by each resource user, in this embodiment, after receiving the front-end operation of the user, the resource user device sends the front-end operation to the resource analysis server, and the resource analysis server maps the front-end operation to the resource usage task in the YARN resource management system, so that the YARN resource management system manages and allocates the resource usage of the front-end operation.
In an embodiment of the present disclosure, the method further includes:
and the resource analysis server establishes an incidence relation between the front-end operation and the resource use task.
In an embodiment of the present disclosure, acquiring the resource consumption of the resource usage task from the YARN resource management system includes:
the resource analysis server sends a request for acquiring the resource consumption condition of the resource use task to the YARN resource management system;
and the YARN resource management system returns the resource consumption condition of the resource use task to the resource analysis server.
The resource analysis server may periodically request the YARN resource management system for the resource consumption situation since the start of the certain or some resource usage tasks, or the resource analysis server may request the YARN resource management system for the resource usage situation of the certain or some resource usage tasks at that time in real time, and record the obtained resource usage situation, so as to perform statistical analysis during the statistical process and the installation of the preset time period.
In an embodiment of the present disclosure, in response to a resource consumption metering request, a resource analysis server determines the resource usage task that is initiated by a resource user within a preset time period, including:
and the resource analysis server determines the front-end operation generated by the resource user in a preset time period, and determines the resource use task initiated by the resource user in the preset time period according to the front-end operation and the incidence relation.
In an embodiment of the present disclosure, the determining, by the resource analysis server, a total consumption condition of the resource consumer for the big data cluster resource in the preset time period according to a resource consumption condition corresponding to the resource usage task in the preset time period includes:
and the resource analysis server determines the front-end operation and the type of the front-end operation generated by the resource user in the preset time period, and determines the total consumption condition of the resource use task corresponding to the front-end operation on the big data cluster resources aiming at different types.
In an embodiment of the present disclosure, the types of front-end operations include an offline query operation and an online scheduling operation.
Technical terms and technical features related to the technical terms and technical features shown in fig. 2 and related embodiments are the same as or similar to those of the technical terms and technical features shown in fig. 1 and related embodiments, and for the explanation and description of the technical terms and technical features related to the technical terms and technical features shown in fig. 2 and related embodiments, reference may be made to the above explanation of the explanation of fig. 1 and related embodiments, and no further description is provided here.
Fig. 3 is a schematic view illustrating an application scenario of a resource consumption metering method according to an embodiment of the present disclosure. Fig. 4 illustrates an overall flowchart of a resource consumption metering method according to an embodiment of the present disclosure. As shown in fig. 3 and 4, each department user in the enterprise generates a front-end operation through a resource using device, that is, a front-end device, the front-end operation is transmitted to the resource analysis server by the front-end device, and the resource analysis server requests the YARN resource management system to use the resource based on the front-end operation. The YARN resource management system creates a resource usage task based on the request and allocates resources for the resource usage task.
The resource analysis server can count resource consumption conditions of each department in each preset time period according to one or more combinations of the preset time periods such as day, week, month, quarter, year and the like. And the resource analysis server determines a resource use task corresponding to front-end operation initiated by a department to be counted under the trigger of the resource counting and metering request. For example, the resource analysis server determines the scheduling resource task corresponding to the online resource scheduling operation generated by each resource user and the offline analysis query task corresponding to the offline query analysis operation, and then requests the YARN resource management system for the resource consumption of these resource usage tasks. The resource analysis server can count the resource consumption condition of each department in a preset time period according to the result returned by the YARN resource management system. For example, the resource consumption of the online scheduling resource task and the resource consumption of the offline analysis query task corresponding to each resource user may be counted respectively. Finally, the statistics can be summarized according to the resource consumption condition of each department in a preset time period through different types of resource using modes.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
Fig. 5 shows a block diagram of a resource consumption metering device according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 5, the resource consumption metering device includes:
a response module 501, configured to determine, in response to the resource consumption metering request, a resource usage task initiated by a resource consumer within a preset time period;
an obtaining module 502, configured to obtain resource consumption of the resource usage task from the YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
a determining module 503, configured to determine, according to a resource consumption condition of the resource usage task corresponding to the preset time period, a total consumption condition of the resource consumer on the big data cluster resource in the preset time period.
As mentioned above, in the field of big data clusters, the use of big data clusters downstream is mainly embodied in two forms: offline analysis query and online scheduling computation. In the prior art, the occupied cost of each department is measured only by simple numbers, such as query times, scheduling script number and scheduling frequency, but the resource consumption of the cluster is greatly different by different query and calculation scripts, so that the resource consumption of each department cluster cannot be accurately measured at all, and various branches and disputes are easy to occur in the cost calculation.
In view of the above problem, in this embodiment, a resource consumption metering device is proposed, which allocates and coordinates the resource of the resource usage task initiated by each resource user in the YARN resource management system according to the usage of the large data cluster resource by different resource users. The embodiment of the disclosure manages the big data cluster resources based on the YARN resource management system, allocates resources for the resource usage task initiated by the front-end operation, and records the resource consumption condition of the resource usage task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
In one embodiment of the present disclosure, the data processing apparatus may be adapted to measure resource usage of each resource user on the resource analysis server.
In an embodiment of the present disclosure, a YARN resource management system may be established for big data cluster resources, which may include, but are not limited to, cluster resources under a Hadoop system.
The Apache Hadoop YARN (hereinafter referred to as YARN) is a general resource management system, and can provide uniform resource management and scheduling service for applications. YARN includes a global Resource Manager (RM) and several Application Masters (AM) for tasks. For each task using big data cluster resources, YARN creates an Application Master (AM), and the Application Master (AM) is responsible for coordinating resources from Resource Manager, and monitors the execution of container and Resource usage (Resource allocation of CPU, memory, etc.) by Node Manager. The Node Manager manages each Node in a YARN cluster. The Node Manager provides services for each Node in the cluster, from overseeing lifetime management of a container to monitoring resources and tracking Node health. The NodeManager manages the abstract containers that represent the resources for each node that are available for a particular task.
To use the YARN cluster resources, the task initiator first initiates a Resource use request to the YARN Resource management system, the Resource Manager negotiates the necessary resources of a container, and starts an ApplicationMaster to manage the task receiving the request. Using a resource request protocol, the ApplicationMaster negotiates the resource containers on each node for use by the tasks. While executing a task, the ApplicationMaster monitors the resource container until the task execution is complete. When the task is completed, the ApplicationMaster logs out the resource container occupied by the ApplicationMaster from the ResourceManager, and the execution cycle is ended.
The resource consumers may be, for example, people involved in various departments of the enterprise. The related personnel can use the big data cluster resources through front-end operation. The resource analysis server can realize the analysis and management of the use condition of each department on the large data cluster resources by mapping the front-end operation of the related personnel to the task in the YARN resource management system.
In an implementation manner of the present disclosure, when the resource analysis server counts the usage of the big data cluster resource for each resource user, for example, each department, the resource analysis server may first determine a resource usage task initiated by the resource user within a preset time period, and obtain the resource consumption of the resource usage task from the YARN resource management system. The YARN resource management system is used for coordinating and managing the resource using process of the resource using task and distributing cluster resources in the executing process of the resource using task until the resource using task is executed completely.
In an embodiment of the present disclosure, the resource usage task may be initiated by a resource user through a front-end operation such as data query, statistical analysis calculation, and the like using a big data cluster resource. The resource user may be a person or an organization having a usage right for the large data cluster resource. The resource analysis server is used for managing the use of the big data cluster resources by the resource users, periodically counting the consumption condition of the big data cluster resources by the resource users, and further giving a report of the resource consumption condition of each resource user to the big data resources in each preset time period.
In an embodiment of the present disclosure, the preset time period may be day, week, month, year, etc.
In an embodiment of the present disclosure, big data cluster resources may include, but are not limited to, new resource types such as memory, CPU, similar graphics processing unit, or dedicated processing device. Resource consumption may include, but is not limited to, occupied memory size, CPU occupancy, memory and CPU usage duration, and the like.
In an embodiment of the present disclosure, the resource consumption metering request may be triggered by a statistical person or periodically.
The technical scheme provided by the embodiment of the disclosure manages the large data cluster resources based on the YARN resource management system, allocates resources for the resource use task initiated by front-end operation, and records the resource consumption condition of the resource use task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system, the resource consumption statistics can be carried out by taking the resource use party initiating the resource use task as a dimension during the resource consumption statistics, the resource consumption condition of each resource use party can be counted according to a time period, and the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
The technical features related to the above device embodiments and the corresponding explanations and descriptions thereof are the same as, corresponding to or similar to the technical features related to the above method embodiments and the corresponding explanations and descriptions thereof, and for the technical features related to the above device embodiments and the corresponding explanations and descriptions thereof, reference may be made to the technical features related to the above method embodiments and the corresponding explanations and descriptions thereof, and details of the disclosure are not repeated herein.
Fig. 6 shows a block diagram of a resource consumption metering system according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 6, the resource consumption metering system includes: a YARN resource management system 601 and a resource analysis server 602;
the YARN resource management system 601 creates a resource usage task and allocates big data cluster resources to the resource usage task;
in response to the resource consumption metering request, the resource analysis server 602 determines the resource usage task initiated by the resource user within a preset time period, and obtains the resource consumption condition of the resource usage task from the YARN resource management system 601;
the resource analysis server 602 determines the total consumption of the big data cluster resource by the resource user in the preset time period according to the resource consumption corresponding to the resource usage task in the preset time period.
As mentioned above, in the field of big data clusters, the use of big data clusters downstream is mainly embodied in two forms: offline analysis query and online scheduling computation. In the prior art, the occupied cost of each department is measured only by simple numbers, such as query times, scheduling script number and scheduling frequency, but the resource consumption of the cluster is greatly different by different query and calculation scripts, so that the resource consumption of each department cluster cannot be accurately measured at all, and various branches and disputes are easy to occur in the cost calculation.
In view of the above problem, in this embodiment, a resource consumption metering system is proposed, which is used for resource allocation and coordinated management of resource usage tasks initiated by various resource users by the YARN resource management system 601 for usage of large data cluster resources by different resource users. The embodiment of the present disclosure manages large data cluster resources based on the YARN resource management system 601, allocates resources to a resource usage task initiated by front-end operation, and records the resource consumption condition of the resource usage task in the execution process. The resource consumption condition of each resource use task is obtained from the YARN resource management system 601 in the embodiment of the disclosure, and when the resource consumption statistics is performed, the resource use party initiating the resource use task can be taken as the dimension to perform the resource consumption statistics, and the resource consumption condition of each resource use party can be also counted according to the time period, so that the problems of inaccurate resource consumption statistics and low statistical efficiency in the prior art can be solved.
In an embodiment of the present disclosure, the resource consumption metering system may be adapted to perform statistical metering on the resource usage by using a system including the YARN resource management system 601 and the resource analysis server 602.
In an embodiment of the present disclosure, the YARN resource management system 601 is configured to manage resource usage of each cluster node in the big data cluster resource, and schedule and allocate resources for tasks using the resources. The YARN resource management system 601, upon receiving a request for using the big data cluster resource from a resource user, for example, the resource user may use the device to perform data query at the front end by executing a script, etc., and in the data query process, it inevitably consumes various resources in the big data cluster resource, and in order to uniformly manage and schedule the big data cluster resource for each resource user to use, the YARN resource management system 601 establishes a resource usage task for the request of the resource user, and allocates corresponding resources to the resource usage task.
The resource analysis server 602 may periodically obtain the resource consumption of each resource usage task from the YARN resource management system 601, and count the total consumption of the big data cluster resources by one or more resource users within a preset time period according to the obtained resource consumption request.
In an embodiment of the present disclosure, the data processing system may be applied to a system for data access through an index table established on an ES, where the system includes a production system for generating data, one or more data storage nodes for storing the data, and an ES search engine for externally providing mass data in a current application scenario.
The technical features and corresponding explanations and explanations related to the above system embodiments are the same as, corresponding to or similar to those related to the above method embodiments and corresponding explanations and explanations, and for the technical features and corresponding explanations and explanations related to the above system embodiments, reference may be made to the technical features and corresponding explanations and explanations related to the above method embodiments, and details of the disclosure are not repeated herein.
The embodiment of the present disclosure also discloses an electronic device, which includes a memory and a processor; wherein the content of the first and second substances,
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to perform any of the method steps described above.
FIG. 7 is a schematic block diagram of a computer system suitable for implementing a resource consumption metering method according to an embodiment of the present disclosure.
As shown in fig. 7, the computer system 700 includes a processing unit 701 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the computer system 700 are also stored. The processing unit 701, the ROM702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary. The processing unit 701 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.
In particular, the above described methods may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the data transmission method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711.
A computer program product is also disclosed in embodiments of the present disclosure, the computer program product comprising computer programs/instructions which, when executed by a processor, implement any of the above method steps.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the disclosed embodiment also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims (10)

1. A resource consumption metering method, comprising:
responding to the resource consumption metering request, and determining a resource use task initiated by a resource user within a preset time period;
acquiring the resource consumption condition of the resource use task from a YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
and determining the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
2. The method of claim 1, wherein the method further comprises:
acquiring front-end operation of the resource user on big data cluster resources;
mapping the front-end operation to the resource usage task of the YARN resource management system.
3. The method of claim 2 wherein mapping the front-end operation to the resource usage task of the YARN resource management system further comprises:
and establishing an incidence relation between the front-end operation and the resource using task.
4. The method of claim 2 or 3, wherein obtaining the resource consumption of the resource usage task from a YARN resource management system comprises:
sending a request for acquiring the resource consumption condition of the resource use task to the YARN resource management system;
and receiving the resource consumption condition returned by the YARN resource management system.
5. A resource consumption metering method, comprising:
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task;
responding to a resource consumption metering request, a resource analysis server determines a resource use task initiated by a resource user within a preset time period, and acquires the resource consumption condition of the resource use task from a YARN resource management system;
and the resource analysis server determines the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
6. A resource consumption metering device comprising:
the response module is configured to respond to the resource consumption metering request and determine a resource use task initiated by a resource using party within a preset time period;
the acquisition module is configured to acquire the resource consumption condition of the resource use task from the YARN resource management system; the YARN resource management system is used for allocating resources to the resource usage task;
and the determining module is configured to determine the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource usage task in the preset time period.
7. A resource consumption metering system comprising: a YARN resource management system and a resource analysis server;
the YARN resource management system creates a resource use task and distributes big data cluster resources for the resource use task;
responding to a resource consumption metering request, the resource analysis server determines the resource use task initiated by a resource user within a preset time period, and acquires the resource consumption condition of the resource use task from a YARN resource management system;
and the resource analysis server determines the total consumption condition of the resource user on the big data cluster resources in the preset time period according to the corresponding resource consumption condition of the resource use task in the preset time period.
8. An electronic device comprising a memory and a processor; wherein the content of the first and second substances,
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method of any one of claims 1-5.
9. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the steps of the method of any one of claims 1-5.
10. A computer program product comprising computer programs/instructions which, when executed by a processor, carry out the steps of the method of any one of claims 1 to 5.
CN202011585082.2A 2020-12-29 2020-12-29 Resource consumption metering method and device, electronic equipment and storage medium Pending CN112749055A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011585082.2A CN112749055A (en) 2020-12-29 2020-12-29 Resource consumption metering method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011585082.2A CN112749055A (en) 2020-12-29 2020-12-29 Resource consumption metering method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112749055A true CN112749055A (en) 2021-05-04

Family

ID=75646504

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011585082.2A Pending CN112749055A (en) 2020-12-29 2020-12-29 Resource consumption metering method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112749055A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391925A (en) * 2021-06-25 2021-09-14 北京字节跳动网络技术有限公司 Cloud resource management method, system, medium, and computer device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868070A (en) * 2015-12-25 2016-08-17 乐视网信息技术（北京）股份有限公司 Method and apparatus for determining resources consumed by tasks
CN108021450A (en) * 2017-12-04 2018-05-11 北京小度信息科技有限公司 Job analysis method and apparatus based on YARN
CN109815008A (en) * 2018-12-21 2019-05-28 航天信息股份有限公司 Hadoop cluster user resource monitoring method and system
CN110597621A (en) * 2019-08-09 2019-12-20 苏宁金融科技(南京)有限公司 Method and system for scheduling cluster resources

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868070A (en) * 2015-12-25 2016-08-17 乐视网信息技术（北京）股份有限公司 Method and apparatus for determining resources consumed by tasks
CN108021450A (en) * 2017-12-04 2018-05-11 北京小度信息科技有限公司 Job analysis method and apparatus based on YARN
CN109815008A (en) * 2018-12-21 2019-05-28 航天信息股份有限公司 Hadoop cluster user resource monitoring method and system
CN110597621A (en) * 2019-08-09 2019-12-20 苏宁金融科技(南京)有限公司 Method and system for scheduling cluster resources

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391925A (en) * 2021-06-25 2021-09-14 北京字节跳动网络技术有限公司 Cloud resource management method, system, medium, and computer device

Similar Documents

Publication Publication Date Title
US8402086B2 (en) Brokered cloud computing architecture
WO2018103590A1 (en) Method, device, and system for assigning pickup tasks
US9032406B2 (en) Cooperative batch scheduling in multitenancy system based on estimated execution time and generating a load distribution chart
JP2020173778A (en) Method, apparatus, electronic facility, computer readable medium, and computer program for allocating resource
CN109213597A (en) Resource allocation methods, device, computer equipment and computer readable storage medium
CN106959894B (en) Resource allocation method and device
US8910128B2 (en) Methods and apparatus for application performance and capacity analysis
CN108733464B (en) Method and device for determining scheduling scheme of computing task
CN109697637B (en) Object type determination method and device, electronic equipment and computer storage medium
US20170155596A1 (en) Method And Electronic Device For Bandwidth Allocation
US20190095245A1 (en) System and Method for Apportioning Shared Computer Resources
US20150317081A1 (en) Adaptive system provisioning
CN103810045A (en) Resource allocation method, resource manager, resource server and system
CN112579692A (en) Data synchronization method, device, system, equipment and storage medium
US10789307B2 (en) Cloud-based discovery and inventory
CN112749055A (en) Resource consumption metering method and device, electronic equipment and storage medium
CN114489985A (en) Data processing method, device and storage medium
CN107193749B (en) Test method, device and equipment
CN107092556B (en) Test method, device and equipment
US10893015B2 (en) Priority topic messaging
CN113360481B (en) Data processing method, device, equipment and computer readable storage medium
US11758021B2 (en) System for processing coherent data
CN106130757B (en) Information acquisition method and device
CN114531361A (en) Service topology analysis method and device of distributed system and storage medium
CN111538575B (en) Resource scheduling system, method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination