CN112416506A - Container management method, device and computer storage medium - Google Patents
Container management method, device and computer storage medium Download PDFInfo
- Publication number
- CN112416506A CN112416506A CN201910775705.3A CN201910775705A CN112416506A CN 112416506 A CN112416506 A CN 112416506A CN 201910775705 A CN201910775705 A CN 201910775705A CN 112416506 A CN112416506 A CN 112416506A
- Authority
- CN
- China
- Prior art keywords
- container
- load
- target
- program
- virtual machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a container management method, which comprises the following steps: acquiring the load of each first container with a first program deployed in the distributed system; calculating the load to obtain a target load; if the target load is larger than a first preset load, creating a second container with a second program deployed; wherein the first program and the second program are both used for processing a target service. The embodiment of the invention also discloses a container management device and a computer storage medium. The invention improves the service processing capability of the distributed system.
Description
Technical Field
The present invention relates to the field of electronics and information technologies, and in particular, to a method and an apparatus for managing a container, and a computer storage medium.
Background
In the related art, a distributed system ensures that the system can work normally when large-flow access is performed by means of redundancy of a large number of servers. The distributed system comprises a load balancing server and a server for processing service data, and when a user initiates a service request, the load balancing server of the distributed system receives the service request and forwards the service request to the server according to a shunting strategy. However, when a large volume of impact occurs in the distributed system in the related art, the system may operate slowly or down.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention desirably provide a container management method, a device, and a computer storage medium, so as to solve the problem that a distributed system in the related art is slow in operation or down when a large flow of impact occurs, and improve the capability of the distributed system in processing services.
The technical scheme of the invention is realized as follows:
a method of container management, the method comprising:
acquiring the load of each first container with a first program deployed in the distributed system;
calculating the load to obtain a target load;
if the target load is larger than a first preset load, creating a second container with a second program deployed; wherein the first program and the second program are both used for processing a target service.
Optionally, the method further comprises:
if the target load is less than or equal to a second preset load, determining a third container from the first container; wherein the second preset load is less than or equal to the first preset load;
controlling the release of resources of the third container.
Optionally, the calculating the load to obtain the target load includes:
acquiring performance parameters of each first container;
calculating a load sum based on each load and calculating a performance parameter sum based on each performance parameter;
and dividing the sum of the load by the sum of the performance parameters to obtain the target load.
Optionally, if the target load is greater than a first preset load, creating a second container in which a second program is deployed, including:
if the target load is greater than a first preset load, acquiring the resource utilization rate of a first virtual machine for operating the first container;
if the resource utilization rate is less than or equal to a first threshold value, creating the second container on the first virtual machine;
if the resource utilization rate is larger than the first threshold value, calling a second virtual machine;
creating the second container on the second virtual machine.
Optionally, if the target load is greater than a first preset load, creating a second container in which a second program is deployed, including:
if the target load is greater than the first preset load, acquiring a mirror library for storing mirror images of at least two different types of programs and the type of the first program;
determining an image of a target program matching the type of the first program from the image library;
creating the second container based on the image of the target program.
Optionally, after the creating a second container with a second program deployed therein if the target load is greater than a first preset load, the method further includes:
acquiring first attribute information of the second container;
sending the first attribute information to first electronic equipment; the first electronic device is used for adding the first attribute information to a current attribute list to obtain a first target list; the first target list is used for enabling the second electronic equipment to forward request information for processing the target service to the first container or the second container based on the first target list.
Optionally, if the target load is less than or equal to a second preset load, determining a third container from the first container includes:
if the target load is less than or equal to the second preset load, acquiring resource utilization rates of at least two first virtual machines for operating the first container;
calculating an average resource utilization of the at least two resource utilizations;
if the average resource utilization rate is smaller than or equal to a second threshold value, determining a third virtual machine from the at least two first virtual machines;
determining that the first container running on the third virtual machine is the third container.
Optionally, before the controlling releases the resource of the third container, the method further includes:
acquiring second attribute information of the third container;
sending the second attribute information to the first electronic equipment; the first electronic device is used for deleting the second attribute information in the current attribute list to obtain a second target list; the second target list is used for enabling the second electronic equipment to forward the request information for processing the target service to the first container except the third container based on the second target list.
A container management apparatus, the container management apparatus comprising: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is configured to execute a program of a container management method in a memory to implement the steps of:
acquiring the load of each first container with a first program deployed in the distributed system;
calculating the load to obtain a target load;
if the target load is larger than a first preset load, creating a second container with a second program deployed; wherein the first program and the second program are both used for processing a target service.
A computer storage medium storing one or more programs executable by one or more processors to implement the steps of the container management method described above.
The container management method, device and computer storage medium provided by the embodiments of the present invention obtain the load of each first container with a first program deployed therein in a distributed system, calculating the load to obtain a target load, if the target load is greater than a first preset load, creating a second container with a second program deployed, because the first program and the second program are both used for processing the target service, when the load of the container in the distributed system is large, the second container with the second program for processing the target service is automatically created, and the target service can be processed through both the first container and the second container, so that the load pressure of the first container is reduced, therefore, the situation that the system runs slowly or is down when the distributed system in the related technology has large-flow impact can be avoided, and the service processing capacity of the distributed system is improved.
Drawings
Fig. 1 is a schematic flow chart of a container management method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of another container management method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of another container management method according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of a container management method according to another embodiment of the present invention;
fig. 5 is a schematic structural diagram of a distributed system according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a container management apparatus according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
It should be appreciated that reference throughout this specification to "an embodiment of the present invention" or "an embodiment described previously" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in an embodiment of the present invention" or "in the foregoing embodiments" in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Unless otherwise specified, the container management apparatus may execute any step in the embodiments of the present invention, and a processor or a CPU of the container management apparatus may execute any step in the embodiments of the present invention.
An embodiment of the present invention provides a container management method, which is applied to a container management device, and as shown in fig. 1, the method includes the following steps:
step 101: the load of each first container in the distributed system, in which the first program is deployed, is obtained.
Wherein the first container is used for processing the target service based on the first program.
The container management device may be any device with data processing capabilities, such as a server, a laptop or desktop computer, etc. The container management device in the embodiment of the invention is a management server in a distributed system.
The distributed system may be a Web system. The distributed system comprises a load balancing server, an application server and a database server. The load balancing server is used for receiving request information which is sent by the user terminal equipment and used for processing the target service, and forwarding the request information to the application server in the distributed system, so that the application server can call target data corresponding to the request information from the database server based on the request information, and process the target service according to the target data and the request information. In the embodiment of the present invention, the application server may be any container mentioned in the embodiment of the present invention.
The container mentioned in the embodiment of the invention can be a Docker container which is an open-source application container engine, so that developers can package their applications and dependence packages into a portable container and then distribute the portable container to any popular Linux machine, and virtualization can also be realized. A container is a collection of interfaces between an application/component and a server platform so that the application/component can be deployed to run on a virtual or physical machine with ease.
The load balancing server may include a first tier load balancing server and a second tier load balancing server. The first-layer load balancing server is used for determining a target load balancing server from the second-layer load balancing server and forwarding an access request to the target load balancing server, and the target load balancing server is used for acquiring the performance information of each first container in the distributed system, selecting a specific container for processing a target service from the first containers and forwarding the request information to the specific container. Wherein the performance information of the first container may include: throughput, response time, CPU occupancy, memory occupancy, or the greater of the two values of CPU occupancy and memory occupancy.
At least one of the container management device, the load balancing server, the first-layer load balancing server, the second-layer load balancing server, the application server and the database server is deployed in a containerization mode. When the server is deployed in a container, the program for processing the corresponding service is deployed in the container, and the container with the program deployed can be used for processing request information or other service information.
Load is a measure of the current CPU workload of the first container and may be defined as the average number of threads in the run queue over a particular time interval. The lower the load of the first container is, the better the processing performance of the first container is, and the too high load of the first container may result in failure to process the service request or even cause downtime.
Step 102: and calculating the load to obtain the target load.
The target load is obtained by calculating the load, which may be obtained by obtaining the target load through any mathematical formula (or any functional formula) for the load. The calculation method for calculating the load includes, but is not limited to, summing, averaging, finding a median, finding a maximum or finding a minimum, and the like.
In one embodiment, calculating the load to obtain the target load may include: and summing the loads to obtain a load sum, and taking the load sum as a target load. In another embodiment, calculating the load to obtain the target load may include: and averaging the loads to obtain an average load, and taking the average load as a target load. Similarly, when the calculation method is to calculate a median, a maximum, or a minimum, to obtain the target load, reference may be made to the calculation method to obtain the target load by summing or averaging, and details thereof are not repeated herein.
Step 103: and if the target load is greater than the first preset load, creating a second container with a second program.
The first program and the second program are both used for processing the target service.
The container management apparatus may determine the first preset load based on the performance information of each of the first containers. For example, when the calculation manner is summation, based on the maximum load that each first container can handle, the obtained maximum load sum is M, and the first preset load may be α × M, where α may be any real number greater than or equal to 0.6 and less than 1, and for example, α may be 0.6, 0.8, or 0.9.
In a possible implementation, the container management device may determine the number of created second containers and/or performance information of the second containers based on a difference between the target load and the first preset load; the performance information of the second container may include throughput, response time, CPU occupancy, memory occupancy, or the greater of the two values of CPU occupancy and memory occupancy.
The container management method provided by the embodiment of the invention obtains the load of each first container with a first program deployed in a distributed system, calculates the load to obtain a target load, creates a second container with a second program deployed therein if the target load is greater than a first preset load, and because the first program and the second program are both used for processing a target service, the second container with the second program deployed therein for processing the target service can be automatically created when the load of the container in the distributed system is large, and further the target service can be processed by both the first container and the second container, so that the load pressure of the first container is reduced, thereby avoiding the occurrence of slow operation or downtime of the system when the distributed system in the related technology has large-flow impact, and improving the service processing capacity of the distributed system.
Based on the foregoing embodiments, an embodiment of the present invention provides a control method, as shown in fig. 2, including the following steps:
step 201: the container management device obtains a load of each first container in the distributed system, wherein the first container is provided with the first program.
Step 202: and the container management equipment calculates the load to obtain a target load.
In one embodiment, step 202 may be implemented by the following steps a1 to A3:
step A1: the container management apparatus acquires performance parameters of each first container.
The performance parameter of the first container may include a throughput, a response time, a CPU occupancy, a memory occupancy of the first container, or a greater of two values of the CPU occupancy and the memory occupancy.
Step A2: the container management apparatus calculates a load sum based on each load and calculates a performance parameter sum based on each performance parameter.
The load sum is a result obtained by adding each load, and the performance parameter sum is a result obtained by adding each performance parameter.
Step A3: the container management device obtains a target load based on the load sum divided by the performance parameter sum.
In this way, the target load is determined based on the performance parameters, as well as the load, so that the determined target load can reflect the operation condition of the first container in the distributed system.
Step 203: and if the target load is greater than the first preset load, the container management equipment creates a second container with a second program deployed.
The first program and the second program are both used for processing the target service.
Step 204: and if the target load is less than or equal to the second preset load, the container management equipment determines a third container from the first container.
The second preset load is less than or equal to the first preset load.
The container management apparatus may determine the second preset load based on the performance information of each of the first containers.
In one embodiment, the container management apparatus may determine the third container from the first containers based on the performance information of the first containers. For example, the container management apparatus may select a first container having a response time lower than the target time as the third container. In another embodiment, the container management apparatus may determine the third container based on a difference between the second preset load and the target load. In still another embodiment, the container management apparatus may determine the third container based on the performance information of the first container and a difference between the second preset load and the target load. It is to be understood that the third container may be at least one first container, i.e. the third container may comprise a number of containers of one or at least two.
In one possible embodiment, step 204 may be implemented by the following steps B1 through B4:
step B1: if the target load is less than or equal to the second preset load, the container management device obtains resource utilization rates of at least two first virtual machines for operating the first container.
The distributed storage system may include at least two first virtual machines, each of the first virtual machines having a first container running thereon, wherein at least one first container may run on one of the first virtual machines, and a performance parameter of the at least one first container may be the same or different.
Step B2: the container management apparatus calculates an average resource utilization of at least two resource utilizations.
In the embodiment of the present invention, the container management device obtains the resource utilization rate of each of the at least two first virtual machines, and calculates the average resource utilization rate of the at least two resource utilization rates. In other embodiments, the container management appliance may calculate a sum, a maximum, a minimum, or a median of at least two resource utilizations.
Step B3: and if the average resource utilization rate is less than or equal to a second threshold value, the container management equipment determines a third virtual machine from the at least two first virtual machines.
Wherein the second threshold may be greater than, less than, or equal to the first threshold.
In an embodiment, the container management device may determine, from the at least two first virtual machines, that the first virtual machine corresponding to the minimum resource utilization rate is a third virtual machine, where the number of the third virtual machines is one. In another embodiment, the container management device may determine, from the at least two first virtual machines, that the first virtual machine corresponding to the resource utilization rate smaller than the target resource utilization rate is a third virtual machine, where the number of the third virtual machines is one or at least two.
Step B4: the container management device determines that the first container running on the third virtual machine is a third container.
The container management device takes the first container which runs on the third virtual machine as the third container.
Step 205: the container management device controls the release of the resources of the third container.
Since the second electronic device does not forward the service request to the third container, the third container cannot receive the service request, and the container management device may control to release the resource of the third container.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
In the container management method provided in the embodiment of the present invention, the container management device may add a second container for processing the service request when the target load is greater than the first preset load, determine a third container from the first container when the target load is less than or equal to the second preset load, and control to release resources of the third container, thereby achieving the purpose of saving resources and cost.
Based on the foregoing embodiments, an embodiment of the present invention provides a container management method, as shown in fig. 3, including the following steps:
step 301: the container management device obtains a load of each first container in the distributed system, wherein the first container is provided with the first program.
Step 302: and the container management equipment calculates the load to obtain a target load.
Step 303: if the target load is greater than the first preset load, the container management device obtains the resource utilization rate of a first virtual machine for operating the first container.
The first container runs on the first virtual machines, and when the number of the first containers is at least two, the number of the first virtual machines running at least two first containers may be one or at least two.
The container management device may obtain a CPU occupancy rate and/or a memory occupancy rate of the first virtual machine, and determine a resource utilization rate of the first virtual machine based on the CPU occupancy rate and/or the memory occupancy rate. The CPU occupancy of the first virtual machine may be a ratio of a current performance parameter of the first virtual machine to a maximum performance parameter, where the current performance parameter may include a current bandwidth, a current throughput, or a combination of the current bandwidth and the current throughput. The memory occupancy rate of the first virtual machine may be a ratio of a current memory size of the first virtual machine to a maximum memory size.
In one embodiment, the container management device may obtain a larger value of the CPU occupancy rate and the memory occupancy rate of the first virtual machine, and use the larger value as the resource utilization rate of the first virtual machine. For example, the CPU occupancy rate of the first virtual machine is 50%, the memory occupancy rate of the first virtual machine is 60%, and the container management device determines that the resource utilization rate of the first virtual machine is 60%.
The container management apparatus may determine the number of created second containers and/or performance information of the second containers based on a difference between the target load and the first preset load.
Step 304: and if the resource utilization rate is less than or equal to the first threshold value, the container management equipment creates a second container on the first virtual machine.
When the number of the first virtual machines is at least two, the container management device may further obtain a resource utilization rate of each first virtual machine, determine a target virtual machine based on the resource utilization rate of each first virtual machine, and create a second container on the target virtual machine. For example, the container management device may determine, from the resource utilization rate of each first virtual machine, that the first virtual machine corresponding to the smallest resource utilization rate is a target virtual machine, where in this manner, the target virtual machine is one first virtual machine; or, the container management device may determine, from the resource utilization rate of each virtual machine, that the first virtual machine corresponding to the resource utilization rate smaller than the preset resource utilization rate is the target virtual machine, and in this manner, the target virtual machine may be one first virtual machine or at least two first virtual machines.
Step 305: and if the resource utilization rate is greater than the first threshold value, the container management equipment calls a second virtual machine.
The container management device may invoke the second virtual machine from the cloud platform. Specifically, the container management device may call, through an interface of the cloud platform, a second virtual machine having preset performance parameters from the cloud platform. The preset performance parameter may be a preset processing capability, where the cloud platform may be a cloud server/cloud virtual machine, and the cloud server/cloud virtual machine may be, for example, an ali cloud server/cloud virtual machine, an Tencent cloud server/cloud virtual machine, or a Baidu cloud server/cloud virtual machine.
Step 306: the container management device creates a second container on the second virtual machine.
The method of creating the second container on the second virtual machine by the container management device may be the same as the method of creating the second container on the first virtual machine.
Step 307: and if the target load is less than or equal to the second preset load, the container management equipment determines a third container from the first container.
The second preset load is less than or equal to the first preset load.
Step 308: the container management device controls the release of the resources of the third container.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
In the container management method provided in the embodiment of the present invention, when the resource utilization rate of a first virtual machine for running a first container is less than or equal to a first threshold, that is, when the resource utilization rate of the first virtual machine is low, a second container is directly created on the first virtual machine; when the resource utilization rate of the first virtual machine is greater than a first threshold value, namely when the resource utilization rate of the first virtual machine is large, the second virtual machine is called, and a second container is created on the second virtual machine, so that the container management equipment can determine to create the second container on the first virtual machine according to the resource utilization rate of the first virtual machine, or call the second virtual machine and create the second container on the second virtual machine, not only is the resources of the first virtual machine fully utilized, and the cost is saved, but also the situation that the second container cannot be created and the load pressure of the first container cannot be reduced when the resource utilization rate of the first virtual machine is too high is avoided because the second virtual machine can be called.
Based on the foregoing embodiments, an embodiment of the present invention provides a container management method, as shown in fig. 4, the method includes the following steps:
step 401: the container management device obtains a load of each first container in the distributed system, wherein the first container is provided with the first program.
In one embodiment, the application servers in the distributed storage system are all used for processing the target service, and the container management device may obtain the load of each first container used for processing the target service in the distributed system. In another embodiment, an application server in a distributed storage system may process at least two services including a target service, and a container management device may obtain a load of a first container for processing the target service and a load of a sixth container for processing services other than the target service in the distributed system, where the first container and the sixth container both belong to the application server, but the first container is used for processing a first type of service, such as the target service, and the sixth container is used for processing a second type of service different from the first type of service.
Step 402: and the container management equipment calculates the load to obtain a target load.
Step 403: if the target load is greater than the first preset load, the container management device acquires a mirror library for storing mirrors of at least two different types of programs and the type of the first program.
The mirror library includes at least a mirror of the target program matching the type of the first container. In particular, the mirror library may include images of at least two different types of programs, the types of containers created by the images of the different types of programs being different, the different types of programs being embodied in that the types of business data for processing by the programs are different. For example, in one embodiment, a first type of program is used to process conversational class service requests, a second type of program is used to process streaming class service requests, and a third type of program is used to process interactive class service requests. For another example, in another embodiment, the first type of program is used to process request information sent by the client device through a first application program, the second type of program is used to process request information sent by the client device through a second application program, and so on, where the first application program and the second application program are different application programs, for example, the first application program is a WeChat, and the second application program is a Payment treasure. Of course, in other embodiments, the program may have other expressions according to different types, and the embodiments of the present invention are not limited to the examples.
In one embodiment, the mirror library may be stored locally on the container management device, and the management device retrieves the mirror library locally. In another embodiment, the mirror library may be stored in a cloud device, and the management device acquires the mirror library from the cloud device, where the cloud device may be a cloud server.
Step 404: the container management apparatus determines an image of the target program matching the type of the first program from the image library.
Step 405: the container management apparatus creates a second container based on the image of the target program.
Step 406: and if the target load is less than or equal to the second preset load, the container management equipment determines a third container from the first container.
The second preset load is less than or equal to the first preset load.
Step 407: the container management device controls the release of the resources of the third container.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
In the container management method provided by the embodiment of the present invention, the container management device obtains the types of the mirror image library and the first container, searches the mirror image of the target program matching the type of the first container in the mirror image library, and creates the second container based on the mirror image of the target program, so that the target service can be processed by both the first container and the second container, thereby reducing the load pressure of the first container. In addition, since the second container is created by the image of the target program acquired from the image library, the manner of creating the container is simple.
Based on the foregoing embodiments, in other embodiments of the present invention, after the second container creation is completed, i.e., after step 203, step 304, step 306 or step 405, the container management method may further include the following steps C1 to C2:
step C1: the container management apparatus acquires first attribute information of a second container.
Wherein the first attribute information includes at least identification information of the second container. The identification information is used to identify a location of the second container in the distributed system. In other embodiments, the first attribute information may further include performance information of the second container.
Step C2: the container management device sends the first attribute information to the first electronic device.
The first electronic equipment is used for adding first attribute information to the current attribute list to obtain a first target list; the first target list is used for enabling the second electronic equipment to forward request information for processing the target service to the first container or the second container based on the first target list. Wherein the first electronic device may be a service registry in a distributed system.
The second electronic device may read the current attribute list from the first electronic device at preset intervals, so that the second electronic device may know in time whether the current attribute list is updated to the first target list. The second electronic device may be a layer two load balancing server, wherein the layer two load balancing server may also be referred to as a reverse proxy server. The second electronic device is configured to forward the request message to the first container based on the current attribute list. And when the current attribute list is determined to be updated to the first target list, forwarding the request information to the first container or the second container based on the first target list.
It should be understood that the container management device may further obtain loads of the first container and the second container in real time, and when it is determined that the loads of the first container and the second container are too large (for example, when the sum of the loads of the first container and the second container is greater than a third preset load), may further create a fourth container, so that the first electronic device updates the first target list through the fourth container, and further, the second electronic device forwards the request information to the first container, the second container, or the fourth container according to the updated first target list. Similarly, when it is determined that the loads of the first container and the second container are too small, the releasable container may be further determined from the currently running containers, so that the first electronic device updates the attribute list according to the releasable container, so that the second electronic device forwards the request information to the container corresponding to the updated attribute list, and since the attribute information of the releasable container does not exist in the updated attribute list, the releasable container no longer receives the request information, and the container management device may control to release the resource of the releasable container.
That is, the container management apparatus creates a new container when it is determined that the load is excessive by constantly acquiring the load of each container existing in the distributed system in real time. When the load is determined to be too small, the resources of the containers are released, so that the number of the containers used for processing the request information in the distributed system can be adjusted in real time according to the load of the current containers.
Based on the foregoing embodiments, in other embodiments of the present invention, after the container management apparatus determines to the third container and before the control releases the resource of the third container, i.e. after step 204 and before step 205, or after step 307 and before step 308, or after step 406 and before step 407, the container management method may further include the following steps D1 to D2:
step D1: the container management apparatus acquires second attribute information of a third container.
When the third container includes one first container, the container management apparatus may acquire the second attribute information of the one first container. When the third container includes at least two first containers, the container management apparatus may acquire the second attribute information of each of the at least two first containers.
Step D2: and the container management equipment sends the second attribute information to the first electronic equipment.
The first electronic equipment is used for deleting second attribute information in the current attribute list to obtain a second target list; the second target list is used for enabling the second electronic equipment to forward the request information for processing the target service to the first container except the third container based on the second target list.
The first electronic device may include a current attribute list, where the current attribute list at least includes second attribute information, and the first electronic device may delete the second attribute information of the current attribute list after receiving the second attribute information, to obtain a second target list.
The second electronic device may be a layer two load balancing server, and the second electronic device forwards the service request to the first container corresponding to the second target list. Wherein the first container corresponding to the second target list is the same as the first container except for the third container.
It should be understood that the container management device may further obtain the load of the first container except the third container in real time, and when it is determined that the load of the first container except the third container is too small, may further determine a fifth container from the first container except the third container, so that the first electronic device updates the second target list through the fifth container, and further causes the second electronic device to forward the request information to the first container except the third container and the fifth container according to the updated second target list. Similarly, when the container management device determines that the load of the first container except the third container is too large, a new container may be created, and the method for creating the new container may refer to the description in other embodiments, and is not described herein again. In this way, the number of containers for processing request information of the distributed system can be realized, and the containers are adjusted in real time according to the load of the current containers.
The container management device may also release resources of the third virtual machine. Further, the container management device may further determine whether the third virtual machine is a subscribed cloud virtual machine, and when the third virtual machine is a subscribed cloud virtual machine, the container management device may unsubscribe the third virtual machine.
In this way, the container management device may calculate an average resource utilization rate based on resource utilization rates of at least two first virtual machines used for running the first container, determine, when the average resource utilization rate is low, a third virtual machine from the at least two virtual machines, determine that the first container running on the third virtual machine is the third container, and control to release resources of the third container, thereby fully utilizing resources of the first virtual machine.
Based on the foregoing embodiments, an embodiment of the present invention provides a schematic structural diagram of a distributed system, as shown in fig. 5, the distributed system includes a first-tier load balancing server 51, a second-tier load balancing server 52, an application server 53, a database server 54, a management server 55, and a service registry 55. Each application server 53 may be any container in the above embodiments.
The first-tier load balancing server 51, the second-tier load balancing server 52, the application server 53, the database server 54, the management server 55, and the service registry 55 are all deployed through containerization. By containerized deployment it is understood that any of these servers or service registries 55 are containers that include applications deployed to handle the corresponding services.
The first-tier load balancing server 51 is configured to receive a service request sent by a user end device, and forward request information to the second-tier load balancing server 52. The service request may be request information in the above embodiment.
The second-tier load balancing server 52 is configured to receive the service request sent by the first-tier load balancing server 51, read the current attribute list from the service registry 55, and send the service request to the corresponding application server 53 based on the attribute information of the application server 53 corresponding to the current attribute list.
The application server 53 is configured to receive a service request sent by the two-tier load balancing server 52, call target data corresponding to the service request in the database server 54, obtain feedback information based on the service request of the target data processor, and send the feedback information to the two-tier load balancing server 52, so that the two-tier load balancing server 52 sends the feedback information to the ue through the one-tier load balancing server 51.
The management server 55 is configured to start a new virtual machine according to the resource usage of the application server 53 in the distributed system, start a new application server on the virtual machine, and complete a self-registration process through the service registration center and the second-tier load balancing, so that the newly started application server can share the system traffic.
When the system access amount is too large, the management server 55 may perform a judgment according to the load condition of the application server 53 and the condition of the comprehensive resource utilization rate on the virtual machine (the resource utilization condition of the virtual machine is determined according to performance data such as a CPU and/or a memory), directly create a new application server 53 if the resource utilization rate on the virtual machine is still sufficient, call an interface of the cloud platform if the resource utilization rate on the virtual machine is already insufficient, start the new virtual machine, and then read the image of the application program from the image library by the new virtual machine through an agent command, thereby starting the new application server 53 on the newly started virtual machine. The newly started application server 53 transmits its own attribute information to the service registration center for registration, and then the second-tier load balancing 52 reads new attribute information, and can transmit a new request to the new application server 53.
When the comprehensive resource utilization rate of the virtual machine is too low and the load of the application server 53 is small, the management server 55 stops part of the application servers 53 and releases the cloud host (virtual machine), thereby achieving the purpose of saving resources and cost.
The working principle of the distributed system is illustrated by the steps E1 to E7 as follows:
step E1: and uploading the mirror image of each server (comprising a first-layer load balancing server, a second-layer load balancing server, an application server, a database server, a management server and a service registration center) to the container mirror image library. The image of the server may be an image of an application program for processing a corresponding service.
Step E2: and deploying a first-layer load balancing server, a second-layer load balancing server, an application server, a database server, a management server and a service registration center by using a container deployment mode.
Step E3: when the management server monitors the load of the application server and the comprehensive resource utilization rate of the virtual machine, when the load of the application server is found to be too high, whether the resource utilization rate in the current virtual machine is sufficient is judged, if the resource utilization rate of the virtual machine is sufficient, the mirror image of the application program is read from the mirror image library through an agent command, and therefore a new container is directly started on the virtual machine.
Step E4: if the resource utilization rate of the virtual machine is insufficient, a new virtual machine is purchased by calling an interface of the cloud platform, and then the new virtual machine reads the application program mirror image from the container mirror image library through an agent command, so that a new container is started on the new virtual machine.
The application image here may be the program image in the above-described embodiment.
Step E5: and the management server sends the attribute information of the newly started container to a service registration center, and the service registration center is used for storing the attribute information of the container.
The attribute information of the application server may be the attribute information of the above-described embodiment.
Step E6: the two-tier load balancing server may read new attribute information from the service registry every n seconds, and after the new attribute information is read, may forward a new service request to a newly started container, where n is a real number greater than 0, and may be, for example, 1 second, 10 seconds, or 20 seconds.
Step E7: when the management server finds that the comprehensive resource utilization rate of the virtual machine is too low, the management server judges that the current virtual machine comprises a plurality of application servers, firstly sends a request, deletes the attribute information of the application servers in the service registration center, then waits for n seconds, enables the two-layer load balancing server to read the latest attribute information, and then calls a cloud platform unsubscribe interface to unsubscribe the virtual machine.
Based on the foregoing embodiments, an embodiment of the present invention provides a container management apparatus 6, where the container management apparatus 6 may be applied to a container management method provided in the embodiments corresponding to fig. 1 to 4, and as shown in fig. 6, the container management apparatus 6 may include: a processor 61, a memory 62, and a communication bus 63, wherein:
the communication bus 63 is used to implement a communication connection between the processor 61 and the memory 62.
The processor 61 is adapted to execute a program of a container management method stored in the memory 62 to implement the steps of:
acquiring the load of each first container with a first program deployed in the distributed system;
calculating the load to obtain a target load;
if the target load is greater than the first preset load, creating a second container with a second program deployed; the first program and the second program are both used for processing the target service.
In other embodiments of the invention, the processor 61 is configured to execute a program of a container management method stored in the memory 62 to implement the steps of:
if the target load is less than or equal to a second preset load, determining a third container from the first container; the second preset load is less than or equal to the first preset load;
controlling the release of resources of the third container.
In other embodiments of the present invention, the processor 61 is configured to execute the load stored in the memory 62 to calculate the target load, so as to implement the following steps:
acquiring performance parameters of each first container;
calculating a load sum based on each load and calculating a performance parameter sum based on each performance parameter;
and dividing the sum of the load and the performance parameter to obtain a target load.
In other embodiments of the present invention, the processor 61 is configured to execute the second container with the second program deployed therein if the target load stored in the memory 62 is greater than the first preset load, so as to implement the following steps:
if the target load is greater than the first preset load, acquiring the resource utilization rate of a first virtual machine for operating a first container;
if the resource utilization rate is less than or equal to a first threshold value, a second container is established on the first virtual machine;
if the resource utilization rate is larger than a first threshold value, calling a second virtual machine;
a second container is created on the second virtual machine.
In other embodiments of the present invention, the processor 61 is configured to execute the second container with the second program deployed therein if the target load stored in the memory 62 is greater than the first preset load, so as to implement the following steps:
if the target load is greater than the first preset load, acquiring a mirror library for storing mirror images of at least two different types of programs and the type of the first program;
determining a mirror image of the target program matched with the type of the first program from a mirror image library;
a second container is created based on the image of the target program.
In other embodiments of the present invention, processor 61 is configured to execute a container management method stored in memory 62 to implement the following steps:
acquiring first attribute information of a second container;
sending the first attribute information to the first electronic equipment; the first electronic equipment is used for adding first attribute information to the current attribute list to obtain a first target list; the first target list is used for enabling the second electronic equipment to forward request information for processing the target service to the first container or the second container based on the first target list.
In other embodiments of the present invention, the processor 61 is configured to determine a third container from the first containers if the target load stored in the memory 62 is less than or equal to a second preset load, so as to implement the following steps:
if the target load is less than or equal to a second preset load, acquiring resource utilization rates of at least two first virtual machines for operating the first container;
calculating an average resource utilization of the at least two resource utilizations;
if the average resource utilization rate is less than or equal to a second threshold value, determining a third virtual machine from the at least two first virtual machines;
determining that the first container running on the third virtual machine is a third container.
In other embodiments of the present invention, processor 61 is configured to execute a container control method stored in memory 62 to implement the following steps:
acquiring second attribute information of a third container;
sending the second attribute information to the first electronic equipment; the first electronic equipment is used for deleting second attribute information in the current attribute list to obtain a second target list; the second target list is used for enabling the second electronic equipment to forward the request information for processing the target service to the first container except the third container based on the second target list.
It should be noted that, for a specific implementation process of the steps executed by the processor in this embodiment, reference may be made to the implementation process in the container management method provided in the embodiments corresponding to fig. 1 to 4, and details are not described here again.
The container management device provided by the embodiment of the invention obtains the load of each first container with a first program deployed in a distributed system, calculates the load to obtain a target load, creates a second container with a second program deployed therein if the target load is greater than a first preset load, and because the first program and the second program are both used for processing a target service, the second container with the second program deployed therein for processing the target service can be automatically created when the load of the container in the distributed system is large, and further the target service can be processed by both the first container and the second container, so that the load pressure of the first container is reduced, thereby avoiding the occurrence of slow operation or downtime of the system when the distributed system in the related technology has large-flow impact, and improving the service processing capability of the distributed system.
Based on the foregoing embodiments, embodiments of the invention provide a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement the steps of the container management method as any one of the above.
The Processor or the CPU may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understood that the electronic device implementing the above-mentioned processor function may be other electronic devices, and the embodiments of the present application are not particularly limited.
The computer storage medium/Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM); but may also be various terminals such as mobile phones, computers, tablet devices, personal digital assistants, etc., that include one or any combination of the above-mentioned memories.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit. Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media capable of storing program codes, such as a removable Memory device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, and an optical disk.
The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.
Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.
The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for container management, the method comprising:
acquiring the load of each first container with a first program deployed in the distributed system;
calculating the load to obtain a target load;
if the target load is larger than a first preset load, creating a second container with a second program deployed; wherein the first program and the second program are both used for processing a target service.
2. The method of claim 1, further comprising:
if the target load is less than or equal to a second preset load, determining a third container from the first container; wherein the second preset load is less than or equal to the first preset load;
controlling the release of resources of the third container.
3. The method of claim 1 or 2, wherein the calculating the load to obtain a target load comprises:
acquiring performance parameters of each first container;
calculating a load sum based on each load and calculating a performance parameter sum based on each performance parameter;
and dividing the sum of the load by the sum of the performance parameters to obtain the target load.
4. The method according to claim 1 or 2, wherein the creating a second container with a second program deployed therein if the target load is greater than a first preset load comprises:
if the target load is greater than a first preset load, acquiring the resource utilization rate of a first virtual machine for operating the first container;
if the resource utilization rate is less than or equal to a first threshold value, creating the second container on the first virtual machine;
if the resource utilization rate is larger than the first threshold value, calling a second virtual machine;
creating the second container on the second virtual machine.
5. The method according to claim 1 or 2, wherein the creating a second container with a second program deployed therein if the target load is greater than a first preset load comprises:
if the target load is greater than the first preset load, acquiring a mirror library for storing mirror images of at least two different types of programs and the type of the first program;
determining an image of a target program matching the type of the first program from the image library;
creating the second container based on the image of the target program.
6. The method according to claim 1 or 2, wherein after creating the second container with the second program deployed therein if the target load is greater than the first preset load, the method further comprises:
acquiring first attribute information of the second container;
sending the first attribute information to first electronic equipment; the first electronic device is used for adding the first attribute information to a current attribute list to obtain a first target list; the first target list is used for enabling the second electronic equipment to forward request information for processing the target service to the first container or the second container based on the first target list.
7. The method of claim 2, wherein determining a third container from the first containers if the target load is less than or equal to a second predetermined load comprises:
if the target load is less than or equal to the second preset load, acquiring resource utilization rates of at least two first virtual machines for operating the first container;
calculating an average resource utilization of the at least two resource utilizations;
if the average resource utilization rate is smaller than or equal to a second threshold value, determining a third virtual machine from the at least two first virtual machines;
determining that the first container running on the third virtual machine is the third container.
8. The method of claim 2, wherein prior to the controlling releasing the resources of the third container, the method further comprises:
acquiring second attribute information of the third container;
sending the second attribute information to the first electronic equipment; the first electronic device is used for deleting the second attribute information in the current attribute list to obtain a second target list; the second target list is used for enabling the second electronic equipment to forward the request information for processing the target service to the first container except the third container based on the second target list.
9. A container management apparatus, characterized in that the container management apparatus comprises: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is configured to execute a program of a container management method in a memory to implement the steps of:
acquiring the load of each first container with a first program deployed in the distributed system;
calculating the load to obtain a target load;
if the target load is larger than a first preset load, creating a second container with a second program deployed; wherein the first program and the second program are both used for processing a target service.
10. A computer storage medium, characterized in that the computer storage medium stores one or more programs executable by one or more processors to implement the steps of the container management method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910775705.3A CN112416506A (en) | 2019-08-21 | 2019-08-21 | Container management method, device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910775705.3A CN112416506A (en) | 2019-08-21 | 2019-08-21 | Container management method, device and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112416506A true CN112416506A (en) | 2021-02-26 |
Family
ID=74779547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775705.3A Pending CN112416506A (en) | 2019-08-21 | 2019-08-21 | Container management method, device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112416506A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113296794A (en) * | 2021-05-25 | 2021-08-24 | 多点(深圳)数字科技有限公司 | Software engineering management method and device, storage medium and electronic equipment |
| CN114785793A (en) * | 2022-03-15 | 2022-07-22 | 浪潮云信息技术股份公司 | Elastic expansion load balancing method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106020930A (en) * | 2016-05-13 | 2016-10-12 | 深圳市中润四方信息技术有限公司 | Application container based application management method and system |
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN108399100A (en) * | 2017-02-06 | 2018-08-14 | 中国移动通信集团公司 | A kind of method of adjustment and device of container resource |
| CN108683720A (en) * | 2018-04-28 | 2018-10-19 | 金蝶软件（中国）有限公司 | A kind of container cluster service configuration method and device |
| CN109032755A (en) * | 2018-06-29 | 2018-12-18 | 优刻得科技股份有限公司 | A kind of container service mandatory system and provide the method for container service |
-
2019
- 2019-08-21 CN CN201910775705.3A patent/CN112416506A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106020930A (en) * | 2016-05-13 | 2016-10-12 | 深圳市中润四方信息技术有限公司 | Application container based application management method and system |
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN108399100A (en) * | 2017-02-06 | 2018-08-14 | 中国移动通信集团公司 | A kind of method of adjustment and device of container resource |
| CN108683720A (en) * | 2018-04-28 | 2018-10-19 | 金蝶软件（中国）有限公司 | A kind of container cluster service configuration method and device |
| CN109032755A (en) * | 2018-06-29 | 2018-12-18 | 优刻得科技股份有限公司 | A kind of container service mandatory system and provide the method for container service |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113296794A (en) * | 2021-05-25 | 2021-08-24 | 多点(深圳)数字科技有限公司 | Software engineering management method and device, storage medium and electronic equipment |
| CN114785793A (en) * | 2022-03-15 | 2022-07-22 | 浪潮云信息技术股份公司 | Elastic expansion load balancing method and system |
| CN114785793B (en) * | 2022-03-15 | 2024-08-09 | 浪潮云信息技术股份公司 | Method and system for balancing elastic telescopic load |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109729143B (en) | Deploying a network-based cloud platform on a terminal device | |
| CN111338774B (en) | Distributed timing task scheduling system and computing device | |
| CN105979007B (en) | Method and device for accelerating resource processing and network function virtualization system | |
| EP3125468B1 (en) | Cloud application processing method and application deployment method and relevant apparatus and system | |
| CN108255497B (en) | Application deployment method and device | |
| CN110149409B (en) | Cloud host metadata service management method, system, equipment and storage medium | |
| CN111345008B (en) | Mobile edge host service notification method and device | |
| CN110113441A (en) | Realize computer equipment, the system and method for load balancing | |
| CN109358967B (en) | ME platform APP instantiation migration method and server | |
| US10884880B2 (en) | Method for transmitting request message and apparatus | |
| US10705740B2 (en) | Managing of storage resources | |
| CN111858054A (en) | Resource scheduling system and method based on edge computing in heterogeneous environment | |
| CN110012054B (en) | Service processing method and system based on alliance link network | |
| CN113014608B (en) | Flow distribution control method and device, electronic equipment and storage medium | |
| CN111176803A (en) | Service processing method, device, server and storage medium | |
| CN113206877A (en) | Session keeping method and device | |
| CN112187864A (en) | Load balancing method and device, storage medium and electronic equipment | |
| CN112416506A (en) | Container management method, device and computer storage medium | |
| CN110162404B (en) | Method, system, equipment and computer medium for distributing resources of secure resource pool | |
| CN111835797A (en) | Data processing method, device and equipment | |
| US9317355B2 (en) | Dynamically determining an external systems management application to report system errors | |
| EP3014439A1 (en) | Automatic adjustment of application launch endpoints | |
| CN113535402A (en) | Load balancing processing method and device based on 5G MEC and electronic equipment | |
| CN111045778B (en) | Virtual machine creation method, device, server and storage medium | |
| WO2018014351A1 (en) | Method and apparatus for resource configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210226 |