Disclosure of Invention
The embodiment of the invention provides a memory management method, a memory management device, electronic equipment and a computer readable storage medium.
In a first aspect, an embodiment of the present invention provides a memory management method, which is applicable to a virtual machine.
Specifically, the memory management method includes:
determining a first memory resource locked in advance and a second memory resource locked in delay;
executing a look-ahead locking operation on the first memory resource in response to a preset virtual machine operation being triggered;
and responding to the completion of the operation of the preset virtual machine, releasing the first memory resource, and executing a delayed locking operation on the second memory resource.
With reference to the first aspect, in a first implementation manner of the first aspect, a sum of the first memory resource and the second memory resource is a sum of resources of memory resources of the virtual machine, where the first memory resource is a necessary memory resource required when the virtual machine is started.
With reference to the first aspect and the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the performing a delayed locking operation on the second memory resource is implemented as:
executing locking operation on the second memory resource in response to the first memory resource being released, and releasing the second memory resource when a preset release condition is met; or, in response to the first memory resource being released, performing a fractional locking operation on the second memory resource, and when the locked portion meets a corresponding preset release condition, releasing the locked portion of the second memory resource until all the second memory resources are locked and released.
With reference to the first aspect, the first implementation manner of the first aspect, and the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the executing an advance locking operation on the first memory resource in response to the preset virtual machine operation being triggered is implemented as:
responding to the triggering of the preset virtual machine operation, and acquiring a memory configuration table;
and setting the attribute of the first memory resource as available in the memory configuration table, setting the attribute of the second memory resource as reserved, and executing advanced locking operation on the first memory resource with the attribute as available.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, and the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the performing a delayed locking operation on the second memory resource is implemented as:
in response to the first memory resource being released, performing a locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the second memory resource as available, and releasing the second memory resource when the locked second memory resource meets a preset release condition; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the locked part as available, and releasing the locked part of the second memory resource when the locked part meets the corresponding preset release condition until the locking and releasing of all the second memory resources are completed.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, and the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, in response to the preset virtual machine operation being triggered, the performing an advanced locking operation on the first memory resource is implemented as:
and responding to the triggering of the operation of the preset virtual machine, loading a preset driver so that the preset driver occupies the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, the fourth implementation manner of the first aspect, and the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the performing a delayed locking operation on the second memory resource is implemented as:
in response to the first memory resource being released, executing a release operation on a second memory resource occupied by the preset drive, so that the second memory resource which is released from being occupied is locked, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on the second memory resource occupied by the preset drive so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, the fourth implementation manner of the first aspect, the fifth implementation manner of the first aspect, and the sixth implementation manner of the first aspect, in a seventh implementation manner of the first aspect, the performing an advanced locking operation on the first memory resource in response to the preset virtual machine operation being triggered includes:
and responding to the triggering of the operation of the preset virtual machine, starting a preset system level application to enable the preset system level application to occupy the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource, wherein the priority of the preset system level application is higher than that of the relevant application when the virtual machine is started.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, the fourth implementation manner of the first aspect, the fifth implementation manner of the first aspect, the sixth implementation manner of the first aspect, and the seventh implementation manner of the first aspect, in an eighth implementation manner of the first aspect, the performing a delayed locking operation on the second memory resource is implemented as:
in response to the first memory resource being released, performing a release operation on a second memory resource occupied by the preset system-level application, so that the second memory resource which is released from being occupied is subjected to a locking operation, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on a second memory resource occupied by the preset system-level application so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
In a second aspect, an embodiment of the present disclosure provides a memory management apparatus, which is suitable for a virtual machine.
Specifically, the memory management device includes:
a determining module configured to determine an anticipatory locked first memory resource and a delayed locked second memory resource;
a first locking module configured to perform an anticipatory locking operation on the first memory resource in response to a preset virtual machine operation being triggered;
and the second locking module is configured to release the first memory resource and execute a delayed locking operation on the second memory resource in response to the completion of the preset virtual machine operation.
With reference to the second aspect, in a first implementation manner of the second aspect, a sum of the first memory resource and the second memory resource is a sum of resources of memory resources of the virtual machine, where the first memory resource is a necessary memory resource required when the virtual machine is started.
With reference to the second aspect and the first implementation manner of the second aspect, in a second implementation manner of the second aspect, the second locking module is configured to:
executing locking operation on the second memory resource in response to the first memory resource being released, and releasing the second memory resource when a preset release condition is met; or, in response to the first memory resource being released, performing a fractional locking operation on the second memory resource, and when the locked portion meets a corresponding preset release condition, releasing the locked portion of the second memory resource until all the second memory resources are locked and released.
With reference to the second aspect, the first implementation manner of the second aspect, and the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the first locking module is configured to:
responding to the triggering of the preset virtual machine operation, and acquiring a memory configuration table;
and setting the attribute of the first memory resource as available in the memory configuration table, setting the attribute of the second memory resource as reserved, and executing advanced locking operation on the first memory resource with the attribute as available.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, and the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the second locking module is configured to:
in response to the first memory resource being released, performing a locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the second memory resource as available, and releasing the second memory resource when the locked second memory resource meets a preset release condition; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the locked part as available, and releasing the locked part of the second memory resource when the locked part meets the corresponding preset release condition until the locking and releasing of all the second memory resources are completed.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, and the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the first locking module is configured to:
and responding to the triggering of the operation of the preset virtual machine, loading a preset driver so that the preset driver occupies the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, the fourth implementation manner of the second aspect, and the fifth implementation manner of the second aspect, in a sixth implementation manner of the second aspect, the second locking module is configured to:
in response to the first memory resource being released, executing a release operation on a second memory resource occupied by the preset drive, so that the second memory resource which is released from being occupied is locked, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on the second memory resource occupied by the preset drive so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, the fourth implementation manner of the second aspect, the fifth implementation manner of the second aspect, and the sixth implementation manner of the second aspect, in a seventh implementation manner of the second aspect, the first locking module is configured to:
and responding to the triggering of the operation of the preset virtual machine, starting a preset system level application to enable the preset system level application to occupy the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource, wherein the priority of the preset system level application is higher than that of the relevant application when the virtual machine is started.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, the third implementation manner of the second aspect, the fourth implementation manner of the second aspect, the fifth implementation manner of the second aspect, the sixth implementation manner of the second aspect, and the seventh implementation manner of the second aspect, in an eighth implementation manner of the second aspect, the second locking module is configured to:
in response to the first memory resource being released, performing a release operation on a second memory resource occupied by the preset system-level application, so that the second memory resource which is released from being occupied is subjected to a locking operation, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on a second memory resource occupied by the preset system-level application so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a storage and a processor, where the storage is configured to store one or more computer instructions that support a memory management apparatus to execute the memory management method, and the processor is configured to execute the computer instructions stored in the storage. The memory management device may further include a communication interface for the memory management device to communicate with other devices or a communication network.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer instructions for a memory management device, where the computer instructions include computer instructions for executing the memory management method to be a memory management device.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
according to the technical scheme, the memory resources of the virtual machine are divided into the first memory resource locked in advance and the second memory resource locked in a delayed mode, the first memory resource is locked in advance when the operation of the preset virtual machine is triggered, the first memory resource is released after the operation of the preset virtual machine is completed, and the delayed locking operation is executed on the second memory resource. According to the technical scheme, even if large-capacity memory resources are used, the starting time of the virtual machine cannot be obviously increased, and the starting speed and the running efficiency of the virtual machine can be improved on the premise of guaranteeing the running quality of the virtual machine.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the invention.
Detailed Description
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the embodiments of the present invention, it is to be understood that terms such as "including" or "having", etc., are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may be present or added.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. Embodiments of the present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
According to the technical scheme provided by the embodiment of the invention, the memory resources of the virtual machine are divided into the first memory resource which is locked in advance and the second memory resource which is locked in a delayed mode, the first memory resource is locked in advance when the operation of the preset virtual machine is triggered, the first memory resource is released after the operation of the preset virtual machine is finished, and the delayed locking operation is executed on the second memory resource. According to the technical scheme, even if large-capacity memory resources are used, the starting time of the virtual machine cannot be obviously increased, and the starting speed and the running efficiency of the virtual machine can be improved on the premise of guaranteeing the running quality of the virtual machine.
Fig. 1 shows a flowchart of a memory management method according to an embodiment of the present invention, which is applicable to a virtual machine, and as shown in fig. 1, the memory management method includes the following steps S101 to S103:
in step S101, a first memory resource to be locked in advance and a second memory resource to be locked in delay are determined;
in step S102, in response to a preset virtual machine operation being triggered, performing a look-ahead locking operation on the first memory resource;
in step S103, in response to the completion of the operation of the preset virtual machine, the first memory resource is released, and a delayed locking operation is performed on the second memory resource.
As mentioned above, with the development of virtualization technology, because the KVM virtualization cloud service implemented based on the VFIO can provide a stable and efficient pass-through virtualization solution for the user, the performance of the KVM virtualization cloud service can be close to the use experience of a physical machine, and thus the application of the KVM virtualization cloud service is more and more extensive. However, with the rise of cloud computing and artificial intelligence, the demand for storage and computing resources is increasing, and the configuration of memory capacity is increasing day by day. When the memory resources based on VFIO are used for management, all the memory resources need to be locked first in the starting process of the virtual machine, obviously, when the memory capacity is small, the memory locking process only needs a few milliseconds, the influence on a user can be ignored, but with the increase of the memory capacity, the influence of the memory locking on the starting time of the virtual machine is increasingly increased, for example, for 384GB virtual machine memory configuration, the locking of a single memory resource consumes about 90 seconds, which seriously affects the operating efficiency and quality of the virtual machine.
Based on the above problems, it is considered that there is hysteresis in actual access to the memory resources by the end user during use, that is, the virtual machine is started from the boot to the system, then to the service or application start, and finally to the system load reaching a peak value, which is a relatively long time process. Therefore, in this embodiment, a staged memory management method is provided, in which a memory resource of a virtual machine is divided into a first memory resource locked in advance and a second memory resource locked in a delayed manner, the first memory resource is locked in advance when a preset virtual machine operation is triggered, the first memory resource is released after the preset virtual machine operation is completed, and the delayed locking operation is performed on the second memory resource. Therefore, the starting time of the virtual machine can be greatly shortened on the premise of ensuring that the normal use of a user is not influenced, the starting time of the virtual machine cannot be obviously increased even for large-capacity memory resources, and the starting speed and the running efficiency of the virtual machine can be improved on the premise of ensuring the running quality of the virtual machine.
In an embodiment of the present invention, the first memory resource refers to a necessary memory resource required when the virtual machine is started, for example, a resource required in a system boot of the virtual machine, a configuration of a service or an application environment related to the virtual machine, a service related to the virtual machine, or an application initialization stage; and the second memory resource refers to the memory resource left after the first memory resource is removed. The sum of the first memory resource and the second memory resource is a resource sum of memory resources of the virtual machine, and the first memory resource and the second memory resource respectively account for a preset proportion of the memory resource sum of the virtual machine, for example, the first memory resource accounts for 20% of the memory resource sum of the virtual machine, and the second memory resource accounts for 80% of the memory resource sum of the virtual machine.
In an embodiment of the present invention, the preset virtual machine operation refers to a virtual machine starting operation capable of triggering locking of a memory resource, for example, the preset virtual machine operation may be a virtual machine starting operation, a virtual machine system initialization operation, a service/application/driver application operation on the memory resource, and the like.
In an embodiment of the present invention, the delayed locking refers to an operation of locking the second memory resource after a preset time after the first memory resource is locked in advance. That is, the first memory resource is locked first, the second memory resource is locked again, and the time between two locking operations, i.e. the length of the preset time, is related to the sum of the time for completing the operation of the preset virtual machine and the time for completing the release of the first memory resource.
That is, in this embodiment, after the preset virtual machine operation is triggered, the locking operation is not performed on all memory resources as in the prior art, but a first memory resource necessary when the virtual machine is started is locked first, and then after the preset virtual machine operation is completed, the first memory resource that is locked previously is released, so that the memory resource that has been already locked and released can be used normally, and then the locking operation is performed on the remaining second memory resource. Therefore, the starting time of the virtual machine with large-capacity memory resources can be greatly reduced, the normal use of the virtual machine after starting is ensured, and the influence of the locking process of the memory resources on the use of the user is greatly reduced. When the preset virtual machine operation is a virtual machine starting operation, based on the above technical solution, since the second memory resource is locked after the virtual machine is started, and only the necessary first memory resource is locked before, and the proportion of the first memory resource to the total memory resource of the virtual machine is usually smaller than the proportion of the second memory resource to the total memory resource of the virtual machine, the starting time of the virtual machine can be greatly shortened, for example, for the 384GB virtual machine memory configuration, the scheme in the prior art is adopted, and the locking of the memory resource will consume about 90 seconds of starting time, and by adopting the above technical solution of the present disclosure, 10% of the 384GB, that is, the memory resource of 38G is set as the first memory resource locking, and the remaining 90%, that is, the memory resource of 346G is locked before the virtual machine is started, the start-up time of the virtual machine can be shortened to 9 seconds, i.e. 81 seconds is saved.
In an embodiment of the present invention, the step of performing the delayed locking operation on the second memory resource in step S103 may be implemented as:
executing locking operation on the second memory resource in response to the first memory resource being released, and releasing the second memory resource when a preset release condition is met; or, in response to the first memory resource being released, performing a fractional locking operation on the second memory resource, and when the locked portion meets a corresponding preset release condition, releasing the locked portion of the second memory resource until all the second memory resources are locked and released.
In this embodiment, when the first memory resource is unlocked and released and the delayed locking operation is performed on the second memory resource, the second memory resource may be locked once and then released once, that is, in response to the first memory resource being released, the locking operation is performed on all the second memory resources, and when the second memory resource meets a preset release condition, the locked all the second memory resources are released once. The second memory resource may also be gradually and periodically locked and gradually released, that is, in response to the first memory resource being released, the second memory resource is partially locked, and when the locked portion meets the corresponding preset release condition, the locked portion of the second memory resource is released until all the second memory resources are locked and released. For example, if the second memory resource is divided into three parts to perform locking respectively, after the first memory resource is released, first performing locking on the part 1 of the second memory resource, where both the released first memory resource and the parts 2 and 3 of the second memory resource that are not locked can be used; after the part 1 of the second memory resource is unlocked and released, locking the part 2 of the second memory resource, wherein the released part 1 of the first memory resource and the second memory resource and the part 3 of the second memory resource which is not locked can be used; after the part 2 of the second memory resource is unlocked and released, locking the part 3 of the second memory resource, wherein the released first memory resource, the part 1 and the part 2 of the second memory resource can be used; after the part 3 of the second memory resource is unlocked and released, all the memory resources can be used at this time. The gradual and staged locking and the gradual and staged releasing of the second memory resource can further reduce the influence on the use of the memory resource and further improve the running quality of the virtual machine.
The preset release condition may be, for example, that a lock time expires, a previously locked memory resource is released, a preset virtual machine operation is triggered, and the like, and a person skilled in the art may set a specific release condition according to a need of an actual application, which is not specifically limited by the present disclosure.
In practical applications, the first memory resource and the second memory resource may be divided and the memory resources may be locked step by step in a plurality of manners, for example, a manner of managing a memory configuration table, a manner of loading a preset driver, a manner of starting a preset system-level application, and the like, and then the foregoing manners are explained and explained respectively.
In an embodiment of the present invention, the step S102, executed in response to the preset virtual machine operation being triggered, of executing an advance locking operation on the first memory resource, may be implemented as:
responding to the triggering of the preset virtual machine operation, and acquiring a memory configuration table;
and setting the attribute of the first memory resource as available in the memory configuration table, setting the attribute of the second memory resource as reserved, and executing advanced locking operation on the first memory resource with the attribute as available.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by managing the memory configuration table. Specifically, when the preset virtual machine operation is triggered, such as system initialization, a memory configuration table, such as an E820 table, is obtained; then, by modifying the memory configuration table, setting the attribute of the first memory resource with a predetermined proportion as available in the memory configuration table, that is, the memory resource with the available attribute can be normally locked and released according to the process of starting the virtual machine; and simultaneously reserving a part of memory resources, namely setting the attribute of the second memory resource with a predetermined proportion as reservation, wherein the reserved part of memory resources are unavailable at present, so that the locking cannot be carried out according to the process of starting the virtual machine, and thus, the operations of carrying out advanced locking on the first memory resource with the available attribute and carrying out delayed locking on the second memory resource with the reserved attribute are realized.
In this embodiment, the step of performing the delayed locking operation on the second memory resource in step S103 may be implemented as:
in response to the first memory resource being released, performing a locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the second memory resource as available, and releasing the second memory resource when the locked second memory resource meets a preset release condition; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing step-by-step locking operation on the second memory resource, informing the memory configuration table to set the attribute of the locked part as available, and releasing the locked part of the second memory resource when the locked part meets the corresponding preset release condition until the locking and releasing of all the second memory resources are completed.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of managing a memory configuration table, similar to the above description, there are two embodiments for the delayed locking operation performed on the second memory resource, i.e., performing locking operation and releasing operation on the second memory resource once, specifically, after the first memory resource is released, then performing locking operation on the second memory resource, and notifying the memory configuration table to modify the attribute of the second memory resource to be usable, and when the locked second memory resource meets a preset release condition, performing releasing operation on the second memory resource once. II, secondly: performing stepwise locking and stepwise releasing step by step on the second memory resource, specifically, after the first memory resource is released, performing fractional locking on the second memory resource, notifying the memory configuration table to modify the attribute of the locked portion to be usable, and when the locked portion meets the corresponding preset release condition, releasing the locked portion of the second memory resource until all the locking and releasing of the second memory resource are completed, for example, dividing the second memory resource into three portions to respectively perform locking, when the first memory resource is released, first performing locking on the portion 1 of the second memory resource, notifying the memory configuration table to modify the attribute of the portion 1 to be usable, and after the portion 1 of the second memory resource is unlocked and released, performing locking on the portion 2 of the second memory resource, and informing the memory configuration table to modify the attribute of the part 2 to be usable, similarly, after the part 2 of the second memory resource is unlocked and released, locking the part 3 of the second memory resource, and then informing the memory configuration table to modify the attribute of the part 3 to be usable.
In another embodiment of the present invention, the step S102, executed by the first memory resource, of executing an advance locking operation in response to the preset virtual machine operation being triggered, may be implemented as:
and responding to the triggering of the operation of the preset virtual machine, loading a preset driver so that the preset driver occupies the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by loading a preset driver. Specifically, when the operation of the preset virtual machine is triggered, for example, when a system is initialized, or before a user service, an application, or a driver applies for a memory resource, a preset driver is loaded in advance to preempt a predetermined proportion of the second memory resource, so that the virtual machine performs a preemptive locking operation on the remaining first memory resource, and the preempted second memory resource is reserved until the first memory resource is released. The preset drive may be, for example, a virtio-balloon drive.
In this embodiment, the step of performing the delayed locking operation on the second memory resource in step S103 may be implemented as:
in response to the first memory resource being released, executing a release operation on a second memory resource occupied by the preset drive, so that the second memory resource which is released from being occupied is locked, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on the second memory resource occupied by the preset drive so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of loading a preset driver, similar to the above description, there are two implementation manners for performing a delayed locking operation on the second memory resource, i.e., performing a one-time locking operation and a one-time releasing operation on the second memory resource, and specifically, after the first memory resource is released, performing a release occupation operation on all the second memory resources occupied by the preset driver, so that the second memory resources that are released from occupation can be locked and, when the second memory resources satisfy a preset release condition, all the second memory resources are released. II, secondly: performing stepwise locking and stepwise releasing step by step on the second memory resource, specifically, after the first memory resource is released, performing a partial occupation removing operation on the second memory resource occupied by the preset drive, so that an occupation removing part is subjected to a locking operation, and when the occupation removing part meets a corresponding preset releasing condition, releasing the occupation removing part until all the second memory resources are locked and released, for example, as an example above, dividing the second memory resource into three parts to respectively perform locking, when the first memory resource is released, first removing occupation on a part 1 of the second memory resource, performing locking on a part 1 removed from occupation, and after the part 1 of the second memory resource is unlocked and released, removing occupation on a part 2 of the second memory resource, and locking the unoccupied part 2, similarly, after the part 2 of the second memory resource is unlocked and released, then the part 3 of the second memory resource is unoccupied, and locking the unoccupied part 3.
In another embodiment of the present invention, the step S102, namely the step of executing the preemptive locking operation on the first memory resource in response to the preset virtual machine operation being triggered, may be implemented as:
and responding to the triggering of the operation of the preset virtual machine, starting a preset system level application to enable the preset system level application to occupy the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource, wherein the priority of the preset system level application is higher than that of the relevant application when the virtual machine is started.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by starting a preset system-level application. Specifically, when the operation of the preset virtual machine is triggered, for example, before a user service, an application, or a driver applies for a memory resource, a preset system-level application is started, so that the preset system-level application can occupy a predetermined proportion of the second memory resource in advance, and perform a preemptive locking operation on the remaining unoccupied first memory resource, it should be noted that the priority of the preset system-level application should be higher than the priority of the relevant application when the virtual machine is started, so as to achieve preemption of the memory resource.
In this embodiment, the step of performing the delayed locking operation on the second memory resource in step S103 may be implemented as:
in response to the first memory resource being released, performing a release operation on a second memory resource occupied by the preset system-level application, so that the second memory resource which is released from being occupied is subjected to a locking operation, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on a second memory resource occupied by the preset system-level application so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of starting a preset system-level application, similar to the above description, there are two implementation manners for performing a delayed locking operation on the second memory resource, that is, performing a one-time locking operation and a one-time releasing operation on the second memory resource, specifically, after the first memory resource is released, notifying the preset system-level application to perform an occupation removing operation on the occupied second memory resource, so that the occupied second memory resource is locked and the occupied second memory resource is released once when the second memory resource meets a preset releasing condition. II, secondly: performing stepwise locking and stepwise releasing of the second memory resource, specifically, after the first memory resource is released, notifying the preset system-level application to perform fractional deallocation on of the second memory resource, so that the deallocated portion is continuously locked, and when the deallocated portion meets a corresponding preset release condition, releasing the deallocated portion until all locking and releasing of the second memory resource are completed, for example, still taking the above example as an example, dividing the second memory resource into three portions to respectively perform locking, after the first memory resource is released, first notifying the preset system-level application to deallocate a portion 1 of the second memory resource, performing locking on the portion 1 of the deallocated portion, and after the portion 1 of the second memory resource is unlocked and released, and similarly, after the part 2 of the second memory resource is unlocked and released, the preset system-level application is notified to release the occupation of the part 3 of the second memory resource, and the part 3 of the second memory resource is locked.
Fig. 2A is a schematic view of an application scenario of a memory management method according to the prior art, and fig. 2B is a schematic view of an application scenario of a memory management method according to an embodiment of the present disclosure, in the application scenario, the operation of the preset virtual machine is a virtual machine system initialization operation, as shown in fig. 2A, in the prior art, after a virtual machine is started and enters a system initialization process, all memory resources need to be locked, after the system initialization is completed, the memory resources can be released, then initialization of user services/applications/drivers and operation of the user services/applications/drivers can be started, and for 384GB of virtual machine memory configuration, the locking of the memory resources will consume about 90 seconds of start time. As shown in fig. 2B, in the present disclosure, a 10% portion of the memory resources is used as the first memory resource that is locked in advance, and a 90% portion of the memory resources is used as the second memory resource that is locked in a delayed manner. When the virtual machine is started and enters a system initialization process, the first memory resource part is locked first, after the system initialization is completed, the first memory resource which is locked previously is released, the initialization and the operation of the user service/application/driver and the subsection delay locking of the second memory resource are started at the same time, along with the operation of the user service/application/driver, the actual occupation amount of the memory resources slowly reaches the peak value, and the locking of the second memory resource is also completed in the process. Therefore, the starting time of the virtual machine is greatly shortened under the condition that the normal use of a user is not influenced, and the starting speed and the running efficiency of the virtual machine can be improved on the premise of ensuring the running quality of the virtual machine. For the memory configuration of the virtual machine of 384GB, by adopting the above technical solution of the present disclosure, 10% of the memory resources in 384GB, that is, 38G, are set as the first memory resources to be locked first, and the remaining 90% of the memory resources, that is, 346G, are locked after the virtual machine is started, so that the start time of the virtual machine can be shortened to 9 seconds, that is, the time of 81 seconds is saved.
The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention.
Fig. 3 is a block diagram illustrating a memory management apparatus according to an embodiment of the present invention, which may be implemented as part or all of an electronic device, and may be implemented as a virtual machine, through software, hardware, or a combination of the two. As shown in fig. 3, the memory management device includes:
a determining module 301 configured to determine a first memory resource locked in advance and a second memory resource locked in delay;
a first locking module 302 configured to perform an advance locking operation on the first memory resource in response to a preset virtual machine operation being triggered;
a second locking module 303, configured to release the first memory resource and perform a delayed locking operation on the second memory resource in response to the preset virtual machine operation being completed.
As mentioned above, with the development of virtualization technology, because the KVM virtualization cloud service implemented based on the VFIO can provide a stable and efficient pass-through virtualization solution for the user, the performance of the KVM virtualization cloud service can be close to the use experience of a physical machine, and thus the application of the KVM virtualization cloud service is more and more extensive. However, with the rise of cloud computing and artificial intelligence, the demand for storage and computing resources is increasing, and the configuration of memory capacity is increasing day by day. When the memory resources based on VFIO are used for management, all the memory resources need to be locked first in the starting process of the virtual machine, obviously, when the memory capacity is small, the memory locking process only needs a few milliseconds, the influence on a user can be ignored, but with the increase of the memory capacity, the influence of the memory locking on the starting time of the virtual machine is increasingly increased, for example, for 384GB virtual machine memory configuration, the locking of a single memory resource consumes about 90 seconds, which seriously affects the operating efficiency and quality of the virtual machine.
Based on the above problems, it is considered that there is hysteresis in actual access to the memory resources by the end user during use, that is, the virtual machine is started from the boot to the system, then to the service or application start, and finally to the system load reaching a peak value, which is a relatively long time process. Therefore, in this embodiment, a staged memory management device is provided, in which a memory resource of a virtual machine is divided into a first memory resource locked in advance and a second memory resource locked in a delayed manner, the first memory resource is locked in advance when a preset virtual machine operation is triggered, the first memory resource is released after the preset virtual machine operation is completed, and the delayed locking operation is performed on the second memory resource. Therefore, the starting time of the virtual machine can be greatly shortened on the premise of ensuring that the normal use of a user is not influenced, the starting time of the virtual machine cannot be obviously increased even for large-capacity memory resources, and the starting speed and the running efficiency of the virtual machine can be improved on the premise of ensuring the running quality of the virtual machine.
In an embodiment of the present invention, the first memory resource refers to a necessary memory resource required when the virtual machine is started, for example, a resource required in a system boot of the virtual machine, a configuration of a service or an application environment related to the virtual machine, a service related to the virtual machine, or an application initialization stage; and the second memory resource refers to the memory resource left after the first memory resource is removed. The sum of the first memory resource and the second memory resource is a resource sum of memory resources of the virtual machine, and the first memory resource and the second memory resource respectively account for a preset proportion of the memory resource sum of the virtual machine, for example, the first memory resource accounts for 20% of the memory resource sum of the virtual machine, and the second memory resource accounts for 80% of the memory resource sum of the virtual machine.
In an embodiment of the present invention, the preset virtual machine operation refers to a virtual machine starting operation capable of triggering locking of a memory resource, for example, the preset virtual machine operation may be a virtual machine starting operation, a virtual machine system initialization operation, a service/application/driver application operation on the memory resource, and the like.
In an embodiment of the present invention, the delayed locking refers to an operation of locking the second memory resource after a preset time after the first memory resource is locked in advance. That is, the first memory resource is locked first, the second memory resource is locked again, and the time between two locking operations, i.e. the length of the preset time, is related to the sum of the time for completing the operation of the preset virtual machine and the time for completing the release of the first memory resource.
That is, in this embodiment, after the preset virtual machine operation is triggered, the locking operation is not performed on all memory resources as in the prior art, but a first memory resource necessary when the virtual machine is started is locked first, and then after the preset virtual machine operation is completed, the first memory resource that is locked previously is released, so that the memory resource that has been already locked and released can be used normally, and then the locking operation is performed on the remaining second memory resource. Therefore, the starting time of the virtual machine with large-capacity memory resources can be greatly reduced, the normal use of the virtual machine after starting is ensured, and the influence of the locking process of the memory resources on the use of the user is greatly reduced. When the preset virtual machine operation is a virtual machine starting operation, based on the above technical solution, since the second memory resource is locked after the virtual machine is started, and only the necessary first memory resource is locked before, and the proportion of the first memory resource to the total memory resource of the virtual machine is usually smaller than the proportion of the second memory resource to the total memory resource of the virtual machine, the starting time of the virtual machine can be greatly shortened, for example, for the 384GB virtual machine memory configuration, the scheme in the prior art is adopted, and the locking of the memory resource will consume about 90 seconds of starting time, and by adopting the above technical solution of the present disclosure, 10% of the 384GB, that is, the memory resource of 38G is set as the first memory resource locking, and the remaining 90%, that is, the memory resource of 346G is locked before the virtual machine is started, the start-up time of the virtual machine can be shortened to 9 seconds, i.e. 81 seconds is saved.
In an embodiment of the present invention, the second locking module 303 may be configured to:
executing locking operation on the second memory resource in response to the first memory resource being released, and releasing the second memory resource when a preset release condition is met; or, in response to the first memory resource being released, performing a fractional locking operation on the second memory resource, and when the locked portion meets a corresponding preset release condition, releasing the locked portion of the second memory resource until all the second memory resources are locked and released.
In this embodiment, when the first memory resource is unlocked and released and the delayed locking operation is performed on the second memory resource, the second memory resource may be locked once and then released once, that is, in response to the first memory resource being released, the locking operation is performed on all the second memory resources, and when the second memory resource meets a preset release condition, the locked all the second memory resources are released once. The second memory resource may also be gradually and periodically locked and gradually released, that is, in response to the first memory resource being released, the second memory resource is partially locked, and when the locked portion meets the corresponding preset release condition, the locked portion of the second memory resource is released until all the second memory resources are locked and released. For example, if the second memory resource is divided into three parts to perform locking respectively, after the first memory resource is released, first performing locking on the part 1 of the second memory resource, where both the released first memory resource and the parts 2 and 3 of the second memory resource that are not locked can be used; after the part 1 of the second memory resource is unlocked and released, locking the part 2 of the second memory resource, wherein the released part 1 of the first memory resource and the second memory resource and the part 3 of the second memory resource which is not locked can be used; after the part 2 of the second memory resource is unlocked and released, locking the part 3 of the second memory resource, wherein the released first memory resource, the part 1 and the part 2 of the second memory resource can be used; after the part 3 of the second memory resource is unlocked and released, all the memory resources can be used at this time. The gradual and staged locking and the gradual and staged releasing of the second memory resource can further reduce the influence on the use of the memory resource and further improve the running quality of the virtual machine.
The preset release condition may be, for example, that a lock time expires, a previously locked memory resource is released, a preset virtual machine operation is triggered, and the like, and a person skilled in the art may set a specific release condition according to a need of an actual application, which is not specifically limited by the present disclosure.
In practical applications, the first memory resource and the second memory resource may be divided and the memory resources may be locked step by step in a plurality of manners, for example, a manner of managing a memory configuration table, a manner of loading a preset driver, a manner of starting a preset system-level application, and the like, and then the foregoing manners are explained and explained respectively.
In an embodiment of the present invention, the first locking module 302 may be configured to:
responding to the triggering of the preset virtual machine operation, and acquiring a memory configuration table;
and setting the attribute of the first memory resource as available in the memory configuration table, setting the attribute of the second memory resource as reserved, and executing advanced locking operation on the first memory resource with the attribute as available.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by managing the memory configuration table. Specifically, when the preset virtual machine operation is triggered, such as system initialization, a memory configuration table, such as an E820 table, is obtained; then, by modifying the memory configuration table, setting the attribute of the first memory resource with a predetermined proportion as available in the memory configuration table, that is, the memory resource with the available attribute can be normally locked and released according to the process of starting the virtual machine; and simultaneously reserving a part of memory resources, namely setting the attribute of the second memory resource with a predetermined proportion as reservation, wherein the reserved part of memory resources are unavailable at present, so that the locking cannot be carried out according to the process of starting the virtual machine, and thus, the operations of carrying out advanced locking on the first memory resource with the available attribute and carrying out delayed locking on the second memory resource with the reserved attribute are realized.
In this embodiment, the second locking module 303 may be configured to:
in response to the first memory resource being released, performing a locking operation on the second memory resource, notifying the memory configuration table to set the attribute of the second memory resource as available, and releasing the second memory resource when the locked second memory resource meets a preset release condition; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing step-by-step locking operation on the second memory resource, informing the memory configuration table to set the attribute of the locked part as available, and releasing the locked part of the second memory resource when the locked part meets the corresponding preset release condition until the locking and releasing of all the second memory resources are completed.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of managing a memory configuration table, similar to the above description, there are two embodiments for the delayed locking operation performed on the second memory resource, i.e., performing locking operation and releasing operation on the second memory resource once, specifically, after the first memory resource is released, then performing locking operation on the second memory resource, and notifying the memory configuration table to modify the attribute of the second memory resource to be usable, and when the locked second memory resource meets a preset release condition, performing releasing operation on the second memory resource once. II, secondly: performing stepwise locking and stepwise releasing step by step on the second memory resource, specifically, after the first memory resource is released, performing fractional locking on the second memory resource, notifying the memory configuration table to modify the attribute of the locked portion to be usable, and when the locked portion meets the corresponding preset release condition, releasing the locked portion of the second memory resource until all the locking and releasing of the second memory resource are completed, for example, dividing the second memory resource into three portions to respectively perform locking, when the first memory resource is released, first performing locking on the portion 1 of the second memory resource, notifying the memory configuration table to modify the attribute of the portion 1 to be usable, and after the portion 1 of the second memory resource is unlocked and released, performing locking on the portion 2 of the second memory resource, and informing the memory configuration table to modify the attribute of the part 2 to be usable, similarly, after the part 2 of the second memory resource is unlocked and released, locking the part 3 of the second memory resource, and then informing the memory configuration table to modify the attribute of the part 3 to be usable.
In another embodiment of the present invention, the first locking module 302 may be configured to:
and responding to the triggering of the operation of the preset virtual machine, loading a preset driver so that the preset driver occupies the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by loading a preset driver. Specifically, when the operation of the preset virtual machine is triggered, for example, when a system is initialized, or before a user service, an application, or a driver applies for a memory resource, a preset driver is loaded in advance to preempt a predetermined proportion of the second memory resource, so that the virtual machine performs a preemptive locking operation on the remaining first memory resource, and the preempted second memory resource is reserved until the first memory resource is released. The preset drive may be, for example, a virtio-balloon drive.
In this embodiment, the second locking module 303 may be configured to:
in response to the first memory resource being released, executing a release operation on a second memory resource occupied by the preset drive, so that the second memory resource which is released from being occupied is locked, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on the second memory resource occupied by the preset drive so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of loading a preset driver, similar to the above description, there are two implementation manners for performing a delayed locking operation on the second memory resource, i.e., performing a one-time locking operation and a one-time releasing operation on the second memory resource, and specifically, after the first memory resource is released, performing a release occupation operation on all the second memory resources occupied by the preset driver, so that the second memory resources that are released from occupation can be locked and, when the second memory resources satisfy a preset release condition, all the second memory resources are released. II, secondly: performing stepwise locking and stepwise releasing step by step on the second memory resource, specifically, after the first memory resource is released, performing a partial occupation removing operation on the second memory resource occupied by the preset drive, so that an occupation removing part is subjected to a locking operation, and when the occupation removing part meets a corresponding preset releasing condition, releasing the occupation removing part until all the second memory resources are locked and released, for example, as an example above, dividing the second memory resource into three parts to respectively perform locking, when the first memory resource is released, first removing occupation on a part 1 of the second memory resource, performing locking on a part 1 removed from occupation, and after the part 1 of the second memory resource is unlocked and released, removing occupation on a part 2 of the second memory resource, and locking the unoccupied part 2, similarly, after the part 2 of the second memory resource is unlocked and released, then the part 3 of the second memory resource is unoccupied, and locking the unoccupied part 3.
In yet another embodiment of the present invention, the first locking module 302 may be configured to:
and responding to the triggering of the operation of the preset virtual machine, starting a preset system level application to enable the preset system level application to occupy the second memory resource in advance, and executing advanced locking operation on the remaining first memory resource, wherein the priority of the preset system level application is higher than that of the relevant application when the virtual machine is started.
In this embodiment, the first memory resource and the second memory resource are divided and the memory resources are locked step by starting a preset system-level application. Specifically, when the operation of the preset virtual machine is triggered, for example, before a user service, an application, or a driver applies for a memory resource, a preset system-level application is started, so that the preset system-level application can occupy a predetermined proportion of the second memory resource in advance, and perform a preemptive locking operation on the remaining unoccupied first memory resource, it should be noted that the priority of the preset system-level application should be higher than the priority of the relevant application when the virtual machine is started, so as to achieve preemption of the memory resource.
In this embodiment, the second locking module 303 may be configured to:
in response to the first memory resource being released, performing a release operation on a second memory resource occupied by the preset system-level application, so that the second memory resource which is released from being occupied is subjected to a locking operation, and when the second memory resource meets a preset release condition, releasing the second memory resource; alternatively, the first and second electrodes may be,
and in response to the first memory resource being released, executing a subsection unoccupied operation on a second memory resource occupied by the preset system-level application so as to enable an unoccupied part to be locked, and releasing the unoccupied part when the unoccupied part meets corresponding preset release conditions until all the second memory resources are locked and released.
When the first memory resource and the second memory resource are divided and the memory resources are locked step by step in a manner of starting a preset system-level application, similar to the above description, there are two implementation manners for performing a delayed locking operation on the second memory resource, that is, performing a one-time locking operation and a one-time releasing operation on the second memory resource, specifically, after the first memory resource is released, notifying the preset system-level application to perform an occupation removing operation on the occupied second memory resource, so that the occupied second memory resource is locked and the occupied second memory resource is released once when the second memory resource meets a preset releasing condition. II, secondly: performing stepwise locking and stepwise releasing of the second memory resource, specifically, after the first memory resource is released, notifying the preset system-level application to perform fractional deallocation on of the second memory resource, so that the deallocated portion is continuously locked, and when the deallocated portion meets a corresponding preset release condition, releasing the deallocated portion until all locking and releasing of the second memory resource are completed, for example, still taking the above example as an example, dividing the second memory resource into three portions to respectively perform locking, after the first memory resource is released, first notifying the preset system-level application to deallocate a portion 1 of the second memory resource, performing locking on the portion 1 of the deallocated portion, and after the portion 1 of the second memory resource is unlocked and released, and similarly, after the part 2 of the second memory resource is unlocked and released, the preset system-level application is notified to release the occupation of the part 3 of the second memory resource, and the part 3 of the second memory resource is locked.
Fig. 4 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 4, the electronic device 400 includes a memory 401 and a processor 402; wherein the content of the first and second substances,
the memory 401 is used to store one or more computer instructions, which are executed by the processor 402 to implement any of the method steps described above.
Fig. 5 is a schematic structural diagram of a computer system suitable for implementing a memory management method according to an embodiment of the present invention.
As shown in fig. 5, the computer system 500 includes a processing unit 501 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the system 500 are also stored. The processing unit 501, the ROM502, and the RAM503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary. The processing unit 501 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.
In particular, the above described method may be implemented as a computer software program according to an embodiment of the present invention. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the memory management method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may be a computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present invention.
The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the embodiments of the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present invention are mutually replaced to form the technical solution.