CN111435316A - Resource capacity expansion method and device - Google Patents
Resource capacity expansion method and device Download PDFInfo
- Publication number
- CN111435316A CN111435316A CN201910032119.XA CN201910032119A CN111435316A CN 111435316 A CN111435316 A CN 111435316A CN 201910032119 A CN201910032119 A CN 201910032119A CN 111435316 A CN111435316 A CN 111435316A
- Authority
- CN
- China
- Prior art keywords
- memory
- original process
- communication channel
- target node
- source node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The application discloses a resource capacity expansion method and a device thereof, wherein the method comprises the following steps: receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process; and generating a process copy corresponding to the original process by utilizing the runtime information. By the application, the capacity expansion of the application program can be rapidly realized.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a resource expansion method and apparatus.
Background
A Serverless (Serverless) architecture is a recently emerging architecture style, and the Serverless architecture enables a developer to allocate computing resources and ensure the execution of an application as required by a platform in the process of constructing the application program without concerning the server running state of the application program in the local or cloud.
On-demand allocation means that the number of copies of an application can be dynamically determined according to the traffic access flow during the time when the application consumes computing resources and provides application services, a small number of copies of the application can be provided when the traffic access flow is low, and a large number of copies of the application can be provided when the traffic access flow is high, that is, an elastic operation can be performed on the application according to the traffic access flow.
How to quickly perform flexible operations (especially, the extension operations) on the application programs has been the focus of research by those skilled in the art. In the existing technical solution, an elastic method based on snapshot is usually adopted, and in brief, a "photographing" operation is performed on an application program running right now, the application program running right now is stored as a snapshot and then distributed to other nodes, and an application copy is generated on other nodes by using the snapshot. The disadvantage of this approach is that the storage space of the snapshot is large and a large amount of elastic time is consumed to generate the application copy.
The elastic time of the application program is an important index for executing elastic operation, and if the elastic time of the application program is too long, the sudden flow peak value cannot be dealt with in time during capacity expansion.
Therefore, there is a need in the art for a solution that can quickly implement flexible operation of an application. Further, those skilled in the art will note that the above information is presented merely as background information to aid in understanding the present disclosure. No determination has been made, nor has a statement been made, as to whether any of the above information is applicable as prior art against the present disclosure.
Disclosure of Invention
One of the main objectives of the present application is to provide a resource expansion method and a device thereof, aiming to solve the above mentioned technical problem of quickly implementing flexible operation on an application.
An exemplary embodiment of the present application provides a resource capacity expansion method, including: receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process; and generating a process copy corresponding to the original process by utilizing the runtime information.
Another exemplary embodiment of the present application provides a computer-readable storage medium having stored thereon computer instructions, wherein the instructions, when executed, implement the above-described method.
Another exemplary embodiment of the present application provides a resource expansion apparatus, including a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process; and generating a process copy corresponding to the original process by utilizing the runtime information.
Another exemplary embodiment of the present application provides a resource capacity expansion method, including: establishing a communication channel with a target node by using address information of the target node executing the elastic operation; and sending the stored runtime information corresponding to the original process to the target node by using the communication channel.
Another exemplary embodiment of the present application provides a computer-readable storage medium having stored thereon computer instructions, wherein the instructions, when executed, implement the above-described method.
Another exemplary embodiment of the present application provides a resource expansion apparatus, including a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: establishing a communication channel with a target node by using address information of the target node executing the elastic operation; and sending the stored runtime information corresponding to the original process to the target node by using the communication channel.
Another exemplary embodiment of the present application provides a resource capacity expansion system, including: the source node is configured to: establishing a communication channel with a target node by using address information of the target node executing the elastic operation; and sending the stored runtime information corresponding to the original process to the target node by using the communication channel. The target node is configured to: receiving the runtime information from a source node using the communication path; and generating a process copy corresponding to the original process by utilizing the runtime information.
According to another exemplary embodiment of the present application, a resource capacity expansion method is provided, where the method includes: and sending the stored runtime information corresponding to the original process to the target node by using a communication channel established with the target node. And sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the copy-on-write strategy.
According to another exemplary embodiment of the present application, a resource capacity expansion method is provided, where the method includes: and sending the stored runtime information corresponding to the original process to the target node by using a communication channel established with the target node. And sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the strategy covered during writing.
The above-mentioned at least one technical scheme that this application example embodiment adopted can reach following beneficial effect:
in the resource capacity expansion method of the exemplary embodiment of the application, in the process of performing the elastic operation on the original process, a communication path is established between the source node and the target node, so that data can be transmitted between the source node and the target node, and further, during data transmission, the data of the original process is transmitted in batches, so that the speed of generating the process copy is increased, particularly, the runtime information of non-memory data is transmitted first, so that the process copy can be operated as soon as possible, and the urgency of users is overcome.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic diagram illustrating a resource expansion system performing a resiliency operation in accordance with an illustrative embodiment;
FIG. 2 is a scenario diagram of multiple process copies being generated by a single process in accordance with an illustrative embodiment of the present application;
fig. 3 is a flowchart of a resource expansion method performed by a source node according to an exemplary embodiment of the present application;
FIG. 4 is a flowchart of a resource expansion method performed by a target node according to an exemplary embodiment of the present application;
fig. 5 is a block diagram of a node that performs a resource expansion method according to an exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Before describing exemplary embodiments of the present application, terms referred to in the present application will be explained first to facilitate better understanding of the present application by those skilled in the art.
A node refers to a device with a unique network address, and a node may be a workstation, a client, a network user, or a personal computer, and may also be a server, a printer, and other network-connected devices. A plurality of nodes are connected according to a communication line to form a network.
The application may be deployed in different operating systems, e.g., L inux, Windows, Mac OS X, etc. the running unit of the application may be a container for Docker, Pouch, gvisor, Sandbox, VM, Unikernel, etc. for ease of description, the application will be described from the perspective of a single thread, but those skilled in the art will appreciate that the method is not limited to the number of applications.
Elasticity refers to performing an expansion or contraction process on an application, for example, changing an application from one application copy to two application copies is called expansion, and changing two application copies to one application copy is called contraction. Elasticity can be divided into vertical elasticity and horizontal elasticity, wherein vertical elasticity refers to the expansion and contraction behavior of the same node, and horizontal elasticity refers to the expansion and contraction behavior of the cross-node. The data processing method according to the exemplary embodiment of the present application is more suitable for horizontal elasticity.
The cold start refers to links of the application program starting from a computer loader code, starting the process from a null state, completely undergoing code loading, allocating running resources, initializing the program and the like. The cold start time of an application is often long, which is strongly related to the development language of the application, the performance and the scale of a deployment machine, and a complicated cold start time of the application may reach the 10-minute level. Corresponding to the application, when the capacity expansion operation is performed on the application, the cold start time of the copy of the application is an important factor for limiting the capacity expansion operation.
Copy-on-write (copy-on-write) is an optimization strategy widely used in memory management, data storage, and programming language concurrency control. The core idea is that if multiple callers request the same resource (e.g., memory or data store on disk) at the same time, they will get the same pointer to the same resource together, until a caller tries to modify the content of the resource, the system will not really copy a private copy to the caller, but the original resource seen by other callers will remain unchanged.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
FIG. 1 shows a schematic diagram of a system performing a resilient operation according to an exemplary embodiment. For convenience of description, the following will be described with respect to a capacity expansion operation among elastic operations.
In the present application, a node running a process may be referred to as a source node, for convenience of distinction, the process is referred to as an original process hereinafter, and when the processing capacity of the source node is insufficient and a capacity expansion operation needs to be performed on the process, a node running a copy of the process equivalent to the original process may be referred to as a target node.
As shown in fig. 1, in the Serverless system, an original process runs on a source node 100, and when a control node in the system determines that a capacity expansion operation needs to be performed on the original process, the original process may be stopped from running (also referred to as "freezing"), so that information such as resources in the original process may be prevented from changing when an elastic operation is performed on the original process.
In an optional embodiment, the user may determine in advance that the number of the original processes is insufficient and the original processes need to be expanded. For example, when a shopping website executes a promotion activity, a capacity expansion operation may be performed in advance, and for example, when a news website explodes hot search-level news, a capacity expansion operation may be performed in advance.
The control node in the system determines a target node according to the condition of each node and directly or indirectly sends node information of the target node to a source node, wherein the node information comprises communication information of the target node, so that the source node can conveniently establish a communication channel with the target node by using the communication information.
For example, the system may rank the nodes according to information such as load conditions or resource utilization rates of the nodes, and determine the node with the lowest load condition or resource utilization rate as the target node. The target node may then prepare context information for generating a copy of the process, where the context information includes container information for running the copy. Thus, the target node is ready to generate and run a process copy based on the context information. For example, the target node may create a run container that runs a copy of the process.
Subsequently, the source node may establish a communication channel with the target node according to the received node information. For example, the source node may establish a connection with the target node based on the received IP address of the target node. In this way, the source node may send runtime information related to the process to the target node through the communication channel, where the runtime information may include process tree information and process state information of the original process.
After receiving the runtime information, the target node may establish a process tree of the process copy according to the process tree information in the runtime information, for example, in an L inux system, the process tree of the process copy may be generated by Fork or Clone, and a running state of the process copy is simulated according to the process state information.
As can be seen from the above, in the present application, the process copy created by the target node has the same runtime state as the original process, but does not have the data stored in the memory by the original process. That is, the process copy may be self-running but unable to perform a read operation, and in order for the process copy to be able to perform a read operation, the manner in which the process copy obtains data will be described in detail below.
The target node can forward the data request to the source node when the process copy receives the data request, particularly the data request related to accessing the memory data, and send the memory data to the target node after the source node determines the memory data in the data request.
In addition, an error triggering mechanism can be set for the process copy of the target node, wherein the error triggering mechanism is a mechanism which can trigger an error when the process copy executes the access operation and acquire corresponding data from the memory of the original process according to the error. Specifically, when the process copy executes the access operation, an error mechanism is triggered, the mechanism can generate corresponding error information according to the access operation, the error information includes address information of memory data to be accessed and stored, then, the process copy acquires the memory data from the original process according to the address information, and finally, the memory data is sent to the process copy through the communication channel.
Optionally, when the process copy performs a write operation on specific data, if the specific relates to a whole page, the whole page may be directly written into the memory of the process copy without an error triggering mechanism. After a plurality of times of access operation, the process copy can store a large amount of memory data, so that when the access operation is executed on the process copy again, the data can be directly obtained from the memory of the process copy.
In addition, after the source node sends the runtime information, the data storage mode in the original process may be marked as a copy-on-write (COW) mode, that is, when the memory data of the original process is changed, the corresponding memory data is copied and sent to the target node. In this way, the state of the original process and the state of the process copy in the target node can be guaranteed to be synchronous.
According to the exemplary embodiment of the present application, after the data storage mode in the original process is marked as copy-on-write (COW) mode, the original process is "unfrozen", that is, the original process is made to continue to run.
It has been described above that for a capacity expansion operation in the elastic operation, a capacity reduction operation may be performed on the system when the traffic access traffic is below a predetermined threshold, that is, the number of copies of the process copy is reduced. This may be done for all processes based on user-set priorities, e.g., process copies for certain nodes with too scattered address locations or poor performance may be de-registered based on the geographical location or performance of the node running the process copy.
FIG. 2 illustrates a scenario in which multiple process copies are generated by a single process according to an exemplary embodiment of the present application.
As shown in fig. 2, an original process is running on the source node 100, and upon determining that a capacity expansion operation needs to be performed on the original process running on the source node 100, a copy of the process may be generated on the target nodes 201-206. Although the number of process copies shown in FIG. 2 is 6, those skilled in the art will appreciate that more or fewer process copies may be included as desired.
The target nodes 201-206 may be prepared for generating the process copy upon determining to perform a capacity expansion operation for the target nodes 201-206, e.g., a container may be established in the target nodes on which the process copy may run. Subsequently, the address information of the target nodes 201 to 206 is transmitted to the source node, and the source node can establish communication channels with these target nodes as shown in fig. 2, respectively.
The target nodes 201 to 206 may create a runtime environment of the process copy using the runtime information received from the source node, and obtain memory data through the communication channel when performing the memory access operation.
In order to more clearly describe the exemplary embodiments of the present application, a flowchart of a resource expansion method will be described below in conjunction with fig. 3 and 4 from the source host and the target host, respectively.
Fig. 3 is a flowchart of a resource expansion method performed by a source node according to an exemplary embodiment of the present application.
In step S310, a communication channel is established with the target node using address information of the target node performing the resilient operation. Specifically, a target node performing the resilient operation may be determined, which may be determined by a control node in a resource expansion system (e.g., a distributed system) according to a state of a source node and/or a number of processes requesting a service, and then address information of the target node may be transmitted to the source node. Alternatively, the step of determining the target node may be performed by the source node, that is, the source node may be used as a control node to determine whether the resilient operation needs to be performed for the source node and the target node performing the resilient operation while performing the original process. In addition, the target node can actively send a request for executing the elastic operation to the source node, wherein the request comprises the address information of the target node.
According to the exemplary embodiment of the present application, the source node may establish a communication channel with the target node after receiving the address information of the target node. For example, the source node may establish a wireless communication channel with the target node based on the IP address of the target node.
In addition, according to the embodiment of the present application, before establishing a communication channel with a target node by using address information of the target node performing the resilient operation, the running original process may be stopped. In the implementation process, the ptrace function or the cgroup freezer function can be used for executing the operation, and in addition, a stop signal can be sent to the original process, so that the original process can stop running when receiving the stop signal.
In step S320, the stored runtime information corresponding to the original process is sent to the target node through the communication channel, where the runtime information may include the runtime information including the process tree information of the original process and the runtime state information of the original process. In other words, the runtime information includes all information corresponding to the original process except for the stored data existing in the memory.
In addition, after the source node sends the runtime information, the data storage mode in the original process may be marked as a copy-on-write (COW) mode, that is, when the memory data of the original process is changed, the corresponding memory data is copied and sent to the target node. In this way, the state of the original process and the state of the process copy in the target node can be guaranteed to be synchronous.
According to an exemplary embodiment of the present application, information corresponding to an original process is divided into memory data stored in a memory and non-memory information (the above-mentioned runtime information) other than the memory data. And in order to improve the speed of the elastic operation, the non-memory information can be sent to the target node, so that the target node can generate a process copy after receiving the information. And the memory data with larger data volume is sent to the target node separately.
Optionally, the sending, by using the communication channel, memory data corresponding to the original process, stored in the memory of the source node, to the target node specifically includes: receiving a data request including address information from a source node; determining memory data corresponding to the address information; and sending the memory data to a target node by using the communication channel.
Further, the sending, to the target node, the memory data corresponding to the original process stored in the memory of the source node using the communication channel includes: receiving error information from a source node, wherein the error information comprises address information of the memory data; and sending the memory data corresponding to the address information in the memory area to a target node by using the communication channel.
In addition, the memory data may also be transmitted by using a write-time overlay policy, and specifically, before sending the memory data corresponding to the address information in the memory area to the target node by using the communication channel, the method includes: determining whether the memory data includes a plurality of memory pages; if the plurality of memory pages are included, the plurality of memory pages are sent to the target node at one time by using the communication channel based on a group paging policy, wherein the group paging policy takes data requests for a plurality of memory pages as a policy of one-time data requests.
According to an exemplary embodiment of the present application, after setting the memory data corresponding to the original process stored in the memory of the source node to copy on write, the method may include: and restoring the original process which is stopped to run.
The resource expansion method performed by the source node has been described above with reference to fig. 3, and the resource expansion method performed by the target node will be described in detail below with reference to fig. 4.
Fig. 4 is a flowchart illustrating a resource expansion method performed by a target node according to an exemplary embodiment of the present application. In the application, the target node can transmit the information related to the original process in an asynchronous mode, so that the service can be provided quickly.
In step S410, runtime information corresponding to the original process is received from the source node using a communication path established with the source node running the original process. Before performing step S410, according to an embodiment of the present application, a run container may be created to run the process copy. In particular, the target node may prepare context information for generating a copy of the process, wherein the context information includes container information for running the copy. Thus, the target node is ready to generate and run a process copy based on the context information. Optionally, the runtime information may include process tree information and process state information of the original process.
Specifically, after receiving the process tree in the runtime information, the target node may establish the process tree of the process copy in the target node, for example, in the L inux system, the process tree of the process copy may be generated by Fork or Clone, and the process copy is established according to the process state information, it should be noted that the runtime information does not relate to data stored in the memory of the original process, that is, the runtime information does not include any data related to the process.
The target node may then use the process state information to set the state of the process copy, e.g., open a file handle for the process copy, create a network connection for the process copy, etc.
According to an exemplary embodiment of the present application, the transmission of the memory data may be performed after receiving the data request. Specifically, in response to receiving a data request, address information included in the data request is determined; judging whether the memory of the process copy has memory data corresponding to the address information; and if so, executing data operation on the memory data according to the data request. The data request can be sent to a source node by utilizing the communication channel under the condition that the memory data is not contained in the memory; and receiving the memory data of the original process in the memory area of the source node from the source node by utilizing the communication channel.
That is, in the initial stage, the memory of the process copy does not have any memory data, so the memory data needs to be acquired from the source node after responding to the data request. After a period of time, a large amount of memory data is stored in the memory of the process copy, and then it is determined whether the memory of the process copy has the required memory data after receiving the data request.
Optionally, the memory of the process copy may be set as an error triggering mechanism, where the error triggering mechanism is triggered when data corresponding to each address in the memory is accessed for the first time. That is, the error triggering mechanism is triggered when data at an address in the memory is accessed for the first time. Specifically, after triggering an error triggering mechanism, a source node generates error information corresponding to the data request, wherein the error information includes address information of the memory data; transmitting the error information to a source node using the communication channel; and receiving the memory data of the original process in the memory area of the source node from the source node by using the communication channel. In summary, the memory data can be obtained by using the error triggering mechanism without affecting the operation of the process copy, and the complexity of obtaining the memory data by the process copy is reduced.
In addition, receiving, from the source node, the memory data of the original process in the memory area of the source node using the communication channel when the memory data includes the memory data of the plurality of memory pages includes: and receiving the memory data of the plurality of memory pages in the memory area of the source node from the source node at one time through the communication channel based on a group page switching strategy, wherein the group page switching strategy is a strategy of taking data requests aiming at a plurality of memory pages as one data request. Therefore, a large amount of data can be transmitted at one time, and multiple reading is avoided.
According to an exemplary embodiment of the present application, a resource capacity expansion method is provided, where the method includes sending stored runtime information corresponding to an original process to a target node by using a communication channel established with the target node. And sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the copy-on-write strategy.
According to an exemplary embodiment of the present application, a resource capacity expansion method is provided, where the method includes: and sending the stored runtime information corresponding to the original process to the target node by using a communication channel established with the target node. And sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the strategy covered during writing. Optionally, the write-time override policy is a policy that, when it is determined that the memory data includes a plurality of memory pages, the communication channel is used to send the plurality of memory pages to the target node at one time based on the group paging policy.
In the resource capacity expansion method of the exemplary embodiment of the application, in the process of performing the elastic operation on the original process, a communication path is established between the source node and the target node, so that process information related to the original process can be transmitted between the source node and the target node, further, during the process information transmission, the process information can be distinguished and asynchronously transmitted, and particularly, runtime information of non-memory data is transmitted first, so that a process copy can be operated as soon as possible, and the urgency of a user is solved. Furthermore, the memory of the process copy can be set as an error triggering mechanism, so that the memory data can be acquired while the process copy is not influenced to run. Furthermore, after the source node sends the runtime information, the data storage mode in the original process may be marked as a copy-on-write (COW) mode, that is, when the memory data of the original process is changed, the corresponding memory data is copied and sent to the target node. In this way, the state of the original process and the state of the process copy in the target node can be guaranteed to be synchronous. Furthermore, the memory of the process copy can be set as an error triggering mechanism, so that the memory data can be acquired by using the error triggering mechanism under the condition of not influencing the operation of the process copy, and the complexity of acquiring the memory data by using the process copy is reduced. Furthermore, when a plurality of memory pages are operated, a group page replacement strategy can be adopted, so that a large amount of data can be transmitted at one time, and multiple reading is avoided.
In order to more clearly understand the inventive concept of the exemplary embodiment of the present application, a block diagram of a resource expansion apparatus of the exemplary embodiment of the present application will be described below with reference to fig. 5. Those of ordinary skill in the art will understand that: the apparatus in fig. 5 shows only components related to the present exemplary embodiment, and common components other than those shown in fig. 5 are also included in the apparatus.
Fig. 5 is a block diagram of a node that executes a resource expansion method according to an exemplary embodiment of the present application. It should be noted that the node may be either the target node or the source node, that is, the target node and the source node are different in that processors performing different resource expansion methods are different.
In the case where fig. 5 is a target node, referring to fig. 5, the apparatus includes a processor, an internal bus, and a computer-readable storage medium, wherein the computer-readable storage medium includes a volatile memory and a non-volatile memory, at a hardware level. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.
Specifically, the processor performs the following operations: receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process; and generating a process copy corresponding to the original process by utilizing the runtime information.
Optionally, the runtime information includes process tree information of the original process and state information of the original process during runtime.
Optionally, after the step of implementing generating the process copy corresponding to the original process, the processor includes: in response to receiving a data request, determining address information included in the data request; judging whether the memory of the process copy has memory data corresponding to the address information; and if so, executing data operation on the memory data according to the data request.
Optionally, the processor, in a case that the implementing step does not have the memory data in the memory, further includes: sending the data request to a source node by using the communication channel; and receiving the memory data of the original process in the memory area of the source node from the source node by utilizing the communication channel.
Optionally, after the processor generates a process copy corresponding to the original process by using the runtime information in the implementation step, the method includes: and setting the memory of the process copy as an error triggering mechanism, wherein the error triggering mechanism is triggered when data corresponding to each address in the memory is accessed for the first time.
Optionally, the processor further includes, in a case that the memory does not have the memory data in the memory: triggering an error triggering mechanism in response to the memory not having the memory data. And acquiring the memory data from a source node by using the error triggering mechanism.
Optionally, the obtaining, by the processor in the implementing step, the memory data from the source node by using the error triggering mechanism includes: generating error information corresponding to the data request, wherein the error information comprises address information of the memory data; transmitting the error information to a source node using the communication channel; and receiving the memory data of the original process in the memory area of the source node from the source node by using the communication channel.
Optionally, the receiving, by the processor, the memory data of the original process in the memory area of the source node from the source node by using the communication channel when the memory data includes memory data of a plurality of memory pages includes: and receiving the memory data of the plurality of memory pages in the memory area of the source node from the source node at one time through the communication channel based on a group page switching strategy, wherein the group page switching strategy is a strategy of taking data requests aiming at a plurality of memory pages as one data request.
Optionally, before the receiving, by the processor, runtime information corresponding to the original process from the source node by using a communication path established with the source node running the original process in the implementation step, the method includes: a run container is created that runs a copy of the process.
Further, in the case where the node in fig. 5 is a source node, referring to fig. 5, the apparatus includes a processor, an internal bus, and a computer-readable storage medium, wherein the computer-readable storage medium includes a volatile memory and a non-volatile memory, at a hardware level. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.
Specifically, the processor performs the following operations: establishing a communication channel with a target node by using address information of the target node executing the elastic operation; and sending the stored runtime information corresponding to the original process to the target node by using the communication channel.
Optionally, the processor before implementing the step of establishing the communication channel with the target node by using the address information of the target node performing the resilient operation includes: and stopping the running original process.
Optionally, after the processor operates the communication channel to send the stored runtime information corresponding to the original process to the target node, the method further includes: and setting the memory data corresponding to the original process stored in the memory as copy-on-write.
Optionally, after the implementing step uses the communication channel to send the stored runtime information corresponding to the original process to the target node, the processor further includes: and sending the memory data which is stored in the memory of the source node and corresponds to the original process to the target node by utilizing the communication channel.
Optionally, the sending, by the processor in the implementation step, memory data corresponding to the original process, stored in the memory of the source node to the target node by using the communication channel includes: receiving a data request including address information from a source node; determining memory data corresponding to the address information; and sending the memory data to a target node by using the communication channel.
Optionally, the sending, by the processor in the implementation step, memory data corresponding to the original process, stored in the memory of the source node to the target node by using the communication channel includes: receiving error information from a source node, wherein the error information comprises address information of the memory data; and sending the memory data corresponding to the address information in the memory area to a target node by using the communication channel.
Optionally, before the implementing step uses the communication channel to send the memory data corresponding to the address information in the memory area to the target node, the processor includes: determining whether the memory data includes a plurality of memory pages; and if the memory pages are included, the memory pages are sent to the target node at one time by using the communication channel based on a group page changing strategy, wherein the group page changing strategy takes data requests aiming at the memory pages as a strategy of one-time data requests.
Optionally, the processor, after the implementing step sets memory data corresponding to the original process stored in the memory of the source node to copy when writing, includes: and restoring the original process which is stopped to run.
In the process of performing the elastic operation on the original process, the resource expansion device of the exemplary embodiment of the present application establishes the communication path between the source node and the target node, so that the process information related to the original process can be transmitted between the source node and the target node, further, during the process information transmission, the process information can be distinguished and asynchronously transmitted, and particularly, the runtime information of non-memory data is transmitted first, so that a process copy can be run as soon as possible, and the urgency of users is solved. Furthermore, the memory of the process copy can be set as an error triggering mechanism, so that the memory data can be acquired while the process copy is not influenced to run. Furthermore, after the source node sends the runtime information, the data storage mode in the original process may be marked as a copy-on-write (COW) mode, that is, when the memory data of the original process is changed, the corresponding memory data is copied and sent to the target node. In this way, the state of the original process and the state of the process copy in the target node can be guaranteed to be synchronous. Furthermore, the memory of the process copy can be set as an error triggering mechanism, so that the memory data can be acquired by using the error triggering mechanism under the condition of not influencing the operation of the process copy, and the complexity of acquiring the memory data by using the process copy is reduced. Furthermore, when a plurality of memory pages are operated, a group page replacement strategy can be adopted, so that a large amount of data can be transmitted at one time, and multiple reading is avoided.
It should be noted that the execution subjects of the steps of the method provided in embodiment 1 may be the same device, or different devices may be used as the execution subjects of the method. For example, the execution subject of steps 21 and 22 may be device 1, and the execution subject of step 23 may be device 2; for another example, the execution subject of step 21 may be device 1, and the execution subjects of steps 22 and 23 may be device 2; and so on.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (25)
1. A method for expanding resources is characterized by comprising the following steps:
receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process;
and generating a process copy corresponding to the original process by utilizing the runtime information.
2. The method of claim 1, wherein the runtime information comprises process tree information of the original process and state information of the original process at runtime.
3. The method of claim 1, wherein generating the process copy corresponding to the original process comprises:
in response to receiving a data request, determining address information included in the data request;
judging whether the memory of the process copy has memory data corresponding to the address information;
and if so, executing data operation on the memory data according to the data request.
4. The method of claim 3, wherein in the absence of the memory data in the memory, further comprising:
sending the data request to a source node by using the communication channel;
and receiving the memory data of the original process in the memory area of the source node from the source node by utilizing the communication channel.
5. The method of claim 3, wherein generating a process copy corresponding to the original process using the runtime information comprises:
and setting the memory of the process copy as an error triggering mechanism, wherein the error triggering mechanism is triggered when data corresponding to each address in the memory is accessed for the first time.
6. The method of claim 5, wherein in the absence of the memory data in the memory, further comprising:
triggering an error triggering mechanism in response to the memory not having the memory data;
and acquiring the memory data from a source node by using the error triggering mechanism.
7. The method of claim 5, wherein obtaining the memory data from a source node using the error triggering mechanism comprises:
generating error information corresponding to the data request, wherein the error information comprises address information of the memory data;
transmitting the error information to a source node using the communication channel;
and receiving the memory data of the original process in the memory area of the source node from the source node by using the communication channel.
8. The method according to claim 6 or 7, wherein receiving the memory data of the original process in the memory region of the source node from the source node using the communication channel in case the memory data comprises memory data of a plurality of memory pages comprises:
and receiving the memory data of the plurality of memory pages in the memory area of the source node from the source node at one time through the communication channel based on a group page switching strategy, wherein the group page switching strategy is a strategy of taking data requests aiming at a plurality of memory pages as one data request.
9. The method of claim 1, wherein prior to receiving runtime information corresponding to an original process from a source node using a communication path established with the source node running the original process, comprising:
a run container is created that runs a copy of the process.
10. A computer-readable storage medium having computer instructions stored thereon that, when executed, implement the method of any of claim 1.
11. A resource capacity expansion apparatus, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
receiving runtime information corresponding to an original process from a source node by using a communication channel established with the source node running the original process;
and generating a process copy corresponding to the original process by utilizing the runtime information.
12. A method for expanding resources is characterized by comprising the following steps:
establishing a communication channel with a target node by using address information of the target node executing the elastic operation;
and sending the stored runtime information corresponding to the original process to the target node by using the communication channel.
13. The method of claim 12, prior to establishing a communication channel with a target node using address information of the target node performing the resilient operation, comprising:
and stopping the running original process.
14. The method of claim 12, wherein sending the stored runtime information corresponding to the original process to the target node using the communication channel further comprises:
and setting the memory data corresponding to the original process stored in the memory as copy-on-write.
15. The method of claim 12, wherein sending the stored runtime information corresponding to the original process to the target node using the communication channel further comprises:
and sending the memory data which is stored in the memory of the source node and corresponds to the original process to the target node by utilizing the communication channel.
16. The method of claim 15, wherein sending memory data corresponding to the original process stored in the memory of the source node to the target node using the communication channel comprises:
receiving a data request including address information from a source node;
determining memory data corresponding to the address information;
and sending the memory data to a target node by using the communication channel.
17. The method of claim 16, wherein sending memory data corresponding to the original process stored in the memory of the source node to the target node using the communication channel comprises:
receiving error information from a source node, wherein the error information comprises address information of the memory data;
and sending the memory data corresponding to the address information in the memory area to a target node by using the communication channel.
18. The method of claim 17, wherein before sending memory data corresponding to the address information in a memory region to a target node using the communication channel, comprising:
determining whether the memory data includes a plurality of memory pages;
and if the memory pages are included, the memory pages are sent to the target node at one time by using the communication channel based on a group page changing strategy, wherein the group page changing strategy takes data requests aiming at the memory pages as a strategy of one-time data requests.
19. The method as claimed in claim 13, wherein the setting of the copy-on-write of the memory data corresponding to the original process stored in the memory of the source node comprises:
and restoring the original process which is stopped to run.
20. A computer readable storage medium having computer instructions stored thereon that, when executed, implement the method of any of claims 12.
21. A resource capacity expansion apparatus, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
establishing a communication channel with a target node by using address information of the target node executing the elastic operation;
and sending the stored runtime information corresponding to the original process to the target node by using the communication channel.
22. A resource capacity expansion system is characterized by comprising a source node and a control node, wherein,
the source node is configured to:
establishing a communication channel with a target node by using address information of the target node executing the elastic operation;
transmitting the stored runtime information corresponding to the original process to the target node by using the communication channel,
the target node is configured to:
receiving the runtime information from a source node using the communication path;
and generating a process copy corresponding to the original process by utilizing the runtime information.
23. A method for expanding resources is characterized by comprising the following steps:
sending the stored runtime information corresponding to the original process to the target node by using a communication channel established with the target node;
and sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the copy-on-write strategy.
24. A method for expanding resources is characterized by comprising the following steps:
sending the stored runtime information corresponding to the original process to the target node by using a communication channel established with the target node;
and sending the memory data which is stored in the memory and corresponds to the original process to the target node according to the strategy covered during writing.
25. The method of claim 24, wherein the write-time override policy is a policy to send a plurality of memory pages to a target node at one time using the communication channel based on a group paging policy if it is determined that memory data includes the plurality of memory pages.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910032119.XA CN111435316A (en) | 2019-01-14 | 2019-01-14 | Resource capacity expansion method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910032119.XA CN111435316A (en) | 2019-01-14 | 2019-01-14 | Resource capacity expansion method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111435316A true CN111435316A (en) | 2020-07-21 |
Family
ID=71580539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910032119.XA Pending CN111435316A (en) | 2019-01-14 | 2019-01-14 | Resource capacity expansion method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111435316A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101002177A (en) * | 2004-06-30 | 2007-07-18 | 国际商业机器公司 | Method for controlling a software process, method and system for redistributing or continuing operation in a multi-computer architecture |
| CN108023742A (en) * | 2016-10-31 | 2018-05-11 | 阿里巴巴集团控股有限公司 | A kind of expansion method of application, device and system |
-
2019
- 2019-01-14 CN CN201910032119.XA patent/CN111435316A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101002177A (en) * | 2004-06-30 | 2007-07-18 | 国际商业机器公司 | Method for controlling a software process, method and system for redistributing or continuing operation in a multi-computer architecture |
| CN108023742A (en) * | 2016-10-31 | 2018-05-11 | 阿里巴巴集团控股有限公司 | A kind of expansion method of application, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 郑婷婷等: "云计算环境下的虚拟机快速克隆技术" * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10445140B1 (en) | Serializing duration-limited task executions in an on demand code execution system | |
| US10725826B1 (en) | Serializing duration-limited task executions in an on demand code execution system | |
| US8402200B2 (en) | Method and apparatus for storing and restoring state of virtual machine | |
| TWI574202B (en) | Memory management model and interface for new applications | |
| US11281388B2 (en) | Method for managing a multi-system shared memory, electronic device and non-volatile computer-readable storage medium | |
| US10469405B2 (en) | Network-accessible data volume modification | |
| US10318347B1 (en) | Virtualized tasks in an on-demand network code execution system | |
| US10037298B2 (en) | Network-accessible data volume modification | |
| CN110704177B (en) | Computing task processing method and device, computer equipment and storage medium | |
| CN110750336B (en) | OpenStack virtual machine memory hot-expanding method | |
| CN113037794A (en) | Computing resource allocation scheduling method, device and system | |
| US20170153909A1 (en) | Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine | |
| CN110851285B (en) | Resource multiplexing method, device and equipment based on GPU virtualization | |
| CN110704161B (en) | Virtual machine creation method and device and computer equipment | |
| CN111694639A (en) | Method and device for updating address of process container and electronic equipment | |
| CN115086166A (en) | Computing system, container network configuration method, and storage medium | |
| CN116805946A (en) | Message request processing method and device, electronic equipment and storage medium | |
| US20220206836A1 (en) | Method and Apparatus for Processing Virtual Machine Migration, Method and Apparatus for Generating Virtual Machine Migration Strategy, Device and Storage Medium | |
| CN112631994A (en) | Data migration method and system | |
| CN108124021B (en) | Method, device and system for obtaining Internet Protocol (IP) address and accessing website | |
| CN110750221B (en) | Volume cloning method, apparatus, electronic device and machine-readable storage medium | |
| CN111831451A (en) | Cloud host memory allocation method, cloud host, cloud device and storage medium | |
| CN110968406B (en) | Method, device, storage medium and processor for processing task | |
| CN111078127B (en) | Data migration method, system and device | |
| CN110764930A (en) | Request or response processing method and device based on message mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40034024 Country of ref document: HK |