WO2022057754A1 - 内存控制方法和装置 - Google Patents

内存控制方法和装置 Download PDF

Info

Publication number
WO2022057754A1
WO2022057754A1 PCT/CN2021/117914 CN2021117914W WO2022057754A1 WO 2022057754 A1 WO2022057754 A1 WO 2022057754A1 CN 2021117914 W CN2021117914 W CN 2021117914W WO 2022057754 A1 WO2022057754 A1 WO 2022057754A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
job
node
control group
ratio
Prior art date
Application number
PCT/CN2021/117914
Other languages
English (en)
French (fr)
Inventor
丁肇辉
朱波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022057754A1 publication Critical patent/WO2022057754A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present application relates to the field of computer technology, because it relates to a memory control method and device.
  • HPC high performance computing
  • the embodiments of the present application provide a memory control method and device, which can make full use of system memory and effectively prevent memory overflow.
  • an embodiment of the present application provides a memory control method, which is applied to a first node.
  • the method includes: acquiring a first total occupied memory of each job included in a job control group, where the job control group includes the first total occupied memory. Unfinished jobs on a node; when the first total occupied memory satisfies a first preset condition, reduce the total occupied memory of each job included in the job control group, where the first preset condition includes greater than The maximum available memory, or the first preset condition includes a first preset ratio that is less than or equal to the maximum available memory and greater than the maximum available memory.
  • the first node calls the control group system monitoring process to obtain the first total occupied memory of each job included in the job control group.
  • the number of job control groups is one, and each job acquired by the first node is added to the job control group by the first node.
  • the first node After the first node acquires the job, it starts a task management process that manages the job, the first node invokes the task management process to start the job, and adds a process corresponding to the job to the job In the control group, the job is bound to the job control group.
  • a method for determining the maximum used memory may be as follows: the first node reads the maximum available memory ratio in the configuration file, and determines the maximum used memory according to the maximum used memory ratio and the total memory of the first node.
  • the maximum used memory may also be the system memory of the first node, which is not limited in this solution.
  • the control group system monitoring process has the function of monitoring the job control group in real time, and the unprocessed jobs on the first node are located in the same job control group, the first node can obtain the unprocessed jobs on the first node in real time.
  • the total occupied memory of each job and only when the total occupied memory is greater than the maximum available memory, or less than or equal to the maximum available memory, and greater than the first preset ratio of the maximum available memory, the job control group includes each job.
  • the total memory is occupied. Therefore, the method of this solution can make full use of the system memory of the first node while avoiding memory overflow.
  • the reducing the total occupied memory of each job included in the job control group includes: using a first processing manner to process a process corresponding to the first job in the job control group, the The first processing method is to suspend or terminate.
  • the first job may be a job that satisfies the following conditions: the first actual used memory of the first job is greater than the first maximum available memory of the first job, and the memory of the first job exceeds the highest proportion.
  • the memory excess ratio of the first job is the ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actual used memory and the first maximum available memory.
  • the first optional manner can specifically suspend or terminate the job with the highest memory exceeding ratio, so as to ensure the normal execution of other jobs on the first node.
  • the first job may be the job that actually uses the highest memory.
  • the second optional manner can quickly reduce the total memory occupied by each job included in the control group.
  • the method further includes: the first node reads processing mode indication information in the configuration file, where the processing mode indication information indicates The first treatment method. That is, the first node can obtain the first processing mode according to the processing mode indication information read from the configuration file. In this optional manner, the first node can determine the processing method of the first job, so as to realize the correct processing of the first job and facilitate the management of the job by the first node.
  • This solution reduces the total memory occupied by each job included in the job control group by suspending or terminating the first job, thereby ensuring the normal execution of other jobs except the first job.
  • the method before the process corresponding to the first job in the job control group is processed in the first processing manner, the method further includes: determining that the first total occupied memory is less than or equal to the maximum memory Use memory; determine first memory information according to the first total occupied memory and the maximum used memory, where the first memory information indicates that the first total occupied memory is greater than a first preset ratio of the maximum available memory .
  • the first memory information includes: a first remaining total available memory ratio is less than a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 100%;
  • a remaining total available memory ratio is a ratio of a first difference to the maximum available memory, where the first difference is a difference between the maximum available memory and the first total occupied memory.
  • the second preset ratio is the minimum threshold of the remaining total memory ratio of the first node. The second preset ratio may be read by the first node from the configuration file.
  • the first memory information includes: a ratio of the total used memory is greater than the first preset ratio; and the ratio of the total used memory is a ratio of the first total occupied memory to the maximum available memory.
  • the first preset ratio is the minimum threshold of the used total memory ratio of the first node. The first preset ratio may be read by the first node from a configuration file.
  • the The first processing mode processes the process corresponding to the first job in the job control group. Since the first processing method is used to process the process corresponding to the first job in the job control group after the first total occupied memory is greater than the first preset ratio of the maximum available memory, this solution can effectively reduce job control The probability that the total memory usage of each job included in the group is greater than the maximum memory usage.
  • the method before obtaining the first total occupied memory of each job included in the job control group, the method further includes: obtaining the second total occupied memory of each job included in the job control group; 2. The total occupied memory is greater than the maximum available memory, then the part of the memory data corresponding to each job included in the job control group is migrated to the swap partition, so that the total occupied memory of each job included in the job control group is The first total occupied memory.
  • the first processing method is used to process the job For the first job in control, the first processing method is to suspend or terminate.
  • the memory overflow of the first node can be effectively prevented, and because the total occupied memory of each job in the control group is less than or equal to the maximum available memory and greater than the first preset ratio of the maximum available memory, the system will hang up.
  • Starting or terminating the first job can reduce the probability that the total memory of each job in the control group is greater than the maximum available memory, that is, it can reduce the probability of migrating part of the memory data corresponding to the job control group to the swap partition of the first node, and then Reduced the probability that the job execution time will be longer due to the migration of memory data to the swap partition.
  • the first processing method is suspend, and the first total occupied memory satisfies a first preset condition and the first preset condition is less than or equal to the maximum used memory and greater than the maximum used memory
  • the method further includes: acquiring a third total occupied memory of each job included in the job control group ; determining that the third total occupied memory is less than a third preset ratio of the maximum available memory, and the third preset ratio is less than or equal to the first preset ratio; wake up the process corresponding to the first job.
  • the first node when the first total occupied memory satisfies the first preset condition and the first preset condition is less than or equal to the maximum used memory and greater than the first preset ratio of the maximum available memory, After the first node suspends the process corresponding to the first job, if it monitors that the third total occupied memory is less than the third preset ratio of the maximum available memory, it can wake up the process corresponding to the first job, so that the first job continues to be processed. Execution increases the reliability of the first node.
  • the first processing method is suspension, and the first processing method is adopted when the first total occupied memory satisfies a first preset condition and the first preset condition is greater than the maximum used memory
  • the method further includes: acquiring the third total occupied memory of each job included in the job control group; determining that the third total occupied memory is less than the maximum available memory A fourth preset ratio; wake up the process corresponding to the first job.
  • the reducing the total memory occupied by each job included in the job control group includes: migrating at least part of memory data corresponding to at least one job included in the job control group to a swap partition , so that the total occupied memory of each job included in the job control group is less than or equal to the maximum available memory.
  • At least part of the memory data corresponding to at least one job included in the job control group is migrated to the swap partition, so that the total occupied memory of each job included in the job control group is less than or equal to the maximum available memory, which can effectively to prevent memory overflow.
  • the method further includes: if the swap partition is occupied by the job control group The first occupied storage space is larger than the maximum available storage space of the swap partition, and the process corresponding to the second job in the job control group is processed by the first processing method, so that the swap partition is controlled by the job control group.
  • the occupied storage space is less than or equal to the maximum available storage space, and the first processing method is to suspend or terminate.
  • the second job is any job in the job control group that occupies the swap partition; or, the second job is the storage space of the swap partition occupied in the job control group most homework.
  • the method for the first node to determine the maximum available storage space of the swap partition may be as follows: the first ratio of the maximum used memory of the configuration file read by the first node to the maximum available storage space of the swap partition; The maximum used memory is used to determine the maximum available storage space of the swap partition.
  • This solution can prevent the storage space of the swap partition from being overused.
  • an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores data that can be executed by the at least one processor The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect or any possible implementation manner of the first aspect.
  • an embodiment of the present application provides a memory control system, including a second node and at least one first node; the second node is configured to send job information to the first node; the first node, is configured to obtain a job based on the job information, and add the job to a job control group, where the job control group includes each unprocessed job on the first node; the first node is further configured to obtain The first total occupied memory of each job included in the job control group, where the job control group includes each unprocessed job on the first node; when the first total occupied memory satisfies the first preset condition, reduce The total occupied memory of each job included in the job control group, the first preset condition includes greater than the maximum available memory, or the first preset condition includes less than or equal to the maximum available memory, and greater than the The first preset ratio of the maximum available memory.
  • the first node is specifically configured to: call a control group system monitoring process to acquire the first total occupied memory of each job included in the job control group.
  • the number of the job control group is one, and each job acquired by the first node is added to the job control group by the first node.
  • the first node is specifically configured to: adopt a first processing manner to process a process corresponding to the first job in the job control group, where the first processing manner is suspending or terminating.
  • the first node before the first node processes the process corresponding to the first job in the job control group by using the first processing manner, the first node is further configured to: determine the first job The total occupied memory is less than or equal to the maximum used memory; first memory information is determined according to the first total occupied memory and the maximum used memory, and the first memory information indicates that the first total occupied memory is greater than the The first preset ratio of the maximum available memory.
  • the first memory information includes: the ratio of the total used memory is greater than the first preset ratio; the ratio of the total used memory is the ratio of the first total occupied memory and the maximum The ratio of available memory.
  • the first node is further configured to: read the first preset ratio in the configuration file.
  • the first memory information includes: a first remaining total available memory ratio is less than a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 100 %; the ratio of the first remaining total available memory is the ratio of the first difference to the maximum available memory, and the first difference is the difference between the maximum available memory and the first total occupied memory.
  • the first node is further configured to: read the second preset ratio in the configuration file.
  • the first node before the first node obtains the first total occupied memory of each job included in the job control group, the first node is further configured to: obtain the first total occupied memory of each job included in the job control group 2. The total occupied memory; if the second total occupied memory is greater than the maximum available memory, the part of the memory data corresponding to each job included in the job control group is migrated to the swap partition, so that the job control group includes The total occupied memory of each job is the first total occupied memory.
  • the first actually used memory of the first job is greater than the first maximum available memory of the first job, and the memory excess ratio of the first job is the highest;
  • the memory excess ratio is the ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actually used memory and the first maximum available memory.
  • the first processing mode is suspending, and after the first node uses the first processing mode to process the process corresponding to the first job in the job control group, the first processing The node is further configured to: obtain a third total occupied memory of each job included in the job control group; determine that the third total occupied memory is less than a third preset proportion of the maximum available memory, where the third preset proportion is less than or equal to the first preset ratio; wake up the process corresponding to the first job.
  • the first node before the first node uses the first processing mode to process the process corresponding to the first job in the job control group, the first node is further configured to: read the information in the configuration file The processing mode indication information, the processing mode indication information indicates the first processing mode.
  • the first node is specifically configured to: migrate at least part of the memory data corresponding to at least one job included in the job control group to the swap partition, so that each job included in the job control group The total occupied memory is less than or equal to the maximum available memory.
  • the first node is further configured to: read the maximum available memory ratio in the configuration file; and determine the maximum available memory ratio according to the maximum used memory ratio and the total memory of the first node Use memory.
  • the first node is a computing node or a cloud server in a distributed computing system
  • the second node is a management node in the distributed computing system
  • the job information includes all describe the work.
  • the second node is a terminal device, and the first node is an application server; the job information includes a user request, and the job is used to execute the user request.
  • an embodiment of the present application provides a memory control device, the device includes: an obtaining module configured to obtain a first total occupied memory of each job included in a job control group, the job control group including on the first node Jobs that have not been processed. a processing module, configured to reduce the total occupied memory of each job included in the job control group when the first total occupied memory satisfies a first preset condition, the first preset condition includes greater than the maximum available memory, or , the first preset condition includes a first preset ratio less than or equal to the maximum available memory and greater than the maximum available memory.
  • the obtaining module is specifically configured to: call a control group system monitoring process to obtain the first total occupied memory of each job included in the job control group.
  • the number of the job control group is one, and each job acquired by the first node is added to the job control group by the first node.
  • the processing module is specifically configured to: adopt a first processing manner to process the process corresponding to the first job in the job control group, where the first processing manner is suspending or terminating.
  • the processing module before the processing module adopts the first processing manner to process the process corresponding to the first job in the job control group, the processing module is further configured to: determine the first total occupied memory is less than or equal to the maximum used memory; first memory information is determined according to the first total occupied memory and the maximum used memory, and the first memory information indicates that the first total occupied memory is greater than the maximum available memory the first preset ratio.
  • the first memory information includes: the proportion of the total used memory is greater than the first preset proportion; the proportion of the total used memory is the first total occupied memory and the maximum available memory ratio of memory.
  • the processing module is further configured to: read the first preset ratio in the configuration file.
  • the first memory information includes: a first remaining total available memory ratio is less than a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 100% ;
  • the first remaining total available memory ratio is the ratio of the first difference to the maximum available memory, and the first difference is the difference between the maximum available memory and the first total occupied memory.
  • the processing module is further configured to: read the second preset ratio in the configuration file.
  • the acquiring module before the acquiring module acquires the first total occupied memory of each job included in the job control group: the acquiring module is further configured to: acquire the second total memory usage of each job included in the job control group Occupied memory; if the second total occupied memory is greater than the maximum available memory, the processing module is further configured to: migrate part of the memory data corresponding to each job included in the job control group to the swap partition, so that the The total occupied memory of each job included in the job control group is the first total occupied memory.
  • the first actually used memory of the first job is greater than the first maximum available memory of the first job, and the memory excess ratio of the first job is the highest; the memory of the first job
  • the excess ratio is the ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actually used memory and the first maximum available memory.
  • the first processing manner is suspension, and after the processing module adopts the first processing manner to process the process corresponding to the first job in the job control group: the obtaining module is further configured to: Acquiring the third total occupied memory of each job included in the job control group; the processing module is further configured to: determine a third preset ratio in which the third total occupied memory is smaller than the maximum available memory, and the third preset The ratio is less than or equal to the first preset ratio; and a process corresponding to the first operation is awakened.
  • the processing module before the processing module adopts the first processing mode to process the process corresponding to the first job in the job control group, the processing module is further configured to: read the processing mode in the configuration file indication information, where the processing mode indication information indicates the first processing mode.
  • the processing module is specifically configured to: migrate at least part of the memory data corresponding to at least one job included in the job control group to the swap partition, so that the total amount of each job included in the job control group is The occupied memory is less than or equal to the maximum available memory.
  • the processing module migrates part of the memory data corresponding to each job included in the job control group to the swap partition, the processing module is further configured to: if the swap partition is The first occupied storage space occupied by the job control group is greater than the maximum available storage space of the swap partition, and the first processing method is used to process the process corresponding to the second job in the job control group, so that the swap partition is If the storage space occupied by the job control group is less than or equal to the maximum available storage space, the first processing method is to suspend or terminate.
  • the second job is any one of the jobs that occupy the swap partition in the job control group; or, the second job is all jobs occupied in the job control group The job with the most storage space in the swap partition.
  • the processing module is further configured to: read a first ratio between the maximum used memory in the configuration file and the maximum available storage space of the swap partition; according to the first ratio and the maximum Using memory, determine the maximum available storage space of the swap partition.
  • the processing module is further configured to: read the maximum available memory ratio in the configuration file; determine the maximum used memory ratio according to the maximum used memory ratio and the total memory of the first node .
  • an embodiment of the present application provides a storage medium, where the storage medium includes a computer program, and the computer program is used to implement the method described in the first aspect or any possible implementation manner of the first aspect.
  • an embodiment of the present application provides a chip, including a processor, a memory, and a communication interface, wherein the processor and the memory are connected to the communication interface, wherein the processor is configured to read and execute the A computer program stored in a memory to perform the method described in the first aspect or any possible implementation of the first aspect.
  • FIG. 1 is a schematic diagram of a current memory control method
  • FIG. 2 is a schematic diagram of another current memory control method
  • 3A is a system architecture diagram provided by an embodiment of the present application.
  • 3B is a specific system architecture diagram under a distributed computing system provided by an embodiment of the present application.
  • FIG. 4 is a schematic block diagram of an electronic device according to an embodiment of the application.
  • FIG. 5 is a flowchart 1 of a memory control method provided by an embodiment of the present application.
  • FIG. 6 is a second flowchart of a memory control method provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the principle of the method embodiment shown in FIG. 6;
  • FIG. 8 is a schematic process diagram of the method embodiment shown in FIG. 6;
  • FIG. 9 is a third flowchart of a memory control method provided by an embodiment of the present application.
  • FIG. 10 is a fourth flowchart of a memory control method provided by an embodiment of the present application.
  • FIG. 11 is a flowchart 5 of a memory control method provided by an embodiment of the present application.
  • FIG. 12 is a schematic block diagram of a memory control apparatus provided by an embodiment of the present application.
  • FIG. 13 provides a schematic block diagram of a memory control system according to an embodiment of the present application.
  • a distributed computing system includes multiple computers interconnected through a network, such as multiple HPC computers interconnected through a network.
  • the multiple computers in the distributed computing system may include at least one management node and multiple computing nodes.
  • the management node is used for receiving jobs submitted by users and assigning the jobs to the computing nodes, and the computing nodes execute the jobs assigned by the management nodes.
  • a job can be assigned to one compute node or multiple compute nodes for execution.
  • Job A set of program instances that need to be executed to complete a specific computing business, usually corresponding to a set of processes, containers or other runtime entities on one or more computers. That is, a job corresponds to multiple processes. For example, in a distributed management system, after receiving a user's request, the management node will generate at least one job for processing the request, and assign the job to the computing node for execution. For another example, in some other scenarios, after receiving a user's request for an application, the terminal device sends the request to the application server of the application, and the application server generates at least one job for processing the request and executes at least one job. .
  • Control group which provides a mechanism for controlling the use of resources by a specific set of processes.
  • a cgroup binds a collection of processes to one or more subsystems.
  • a subsystem is a module that manages a collection of processes through the tools and interfaces provided by cgroup.
  • a subsystem is a typical "resource controller", which is used to schedule resources or control the upper limit of resource usage.
  • the Job Limit method is a memory control method applied in distributed computing systems.
  • the management node receives the job submitted by the user, and the job carries the maximum available memory (job-level limit) of the job.
  • the management node allocates the job to the computing node and carries the maximum available memory of the job. RAM.
  • a computing node executes a job, it periodically monitors the actual memory used by the job. If the actual memory used by the job is greater than the maximum available memory for the job, the computing node suspends or terminates the job.
  • the maximum available memory of a job is 10G. When the actual memory used by the job is greater than 10G, the job will be suspended or terminated.
  • the computing node when the computing node monitors the actual memory usage of the job, it cannot perform real-time monitoring. During the monitoring interval, the actual memory usage of the job may be greater than the maximum available memory of the job, and OOM may occur. On the other hand, when a job actually uses more memory than the maximum available memory for the job, even if there is a lot of free memory in the system memory of the computing node, the job is suspended or terminated, so the memory cannot be fully utilized.
  • Hard resource limits are another memory control method applied in distributed computing systems.
  • the computing node creates a control group (cgroup) for each job.
  • the computing node calls the control group system monitoring process to monitor the actual memory used by the job, when the actual memory used by the monitored job is greater than
  • the maximum used memory of the job part of the memory data of the job is migrated to the swap partition, when the actual used memory of the job is greater than the maximum used memory of the job and the storage space of the swap partition occupied by the job is larger than the swap partition of the job
  • the partition uses the maximum storage space (job swap limit)
  • the compute node will suspend or terminate the job.
  • the computing node can call the control group system monitoring process to realize real-time monitoring of the actual memory used by the job, and almost avoids OOM.
  • the actual used memory of the job is greater than the maximum used memory of the job, even if there is a large amount of idle memory in the system memory of the computing node, part of the memory data of the job is migrated to the swap partition. The memory cannot be fully utilized.
  • FIG. 3A is a system architecture diagram provided by an embodiment of the present application.
  • the system architecture includes at least one second node and at least one first node.
  • the second node may be a management node in a distributed computing system, a terminal device, or another server; the first node may be a computing node or a cloud server in a distributed computing system, or other servers, such as cloud servers or application servers.
  • the first node may be a computing node or a cloud server in the distributed computing system and the number of the first nodes is multiple.
  • the second node is a terminal device
  • the first node may be an application server.
  • the first node runs a job, and the job in the first node may be allocated by the second node, or may be generated based on a request sent by the first node.
  • FIG. 3B a specific system architecture diagram may be as shown in FIG. 3B .
  • the system architecture includes a management node and a computing node, and may also include an operation server.
  • the computing node and the cloud server may be the first node shown in FIG. 3A, and the management node may be the second node shown in FIG. 3B. .
  • FIG. 4 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the electronic device in this embodiment may be the first node in the foregoing system architecture, or may be a chip, a chip system, or a processor that supports the first node to implement the following method, and the electronic device may be used to implement the following method embodiments
  • the method corresponding to the first node described in for details, please refer to the description in the method embodiment.
  • the electronic device may include one or more processors 401, and the processors 401 may also be referred to as processing units, which may implement certain control functions.
  • the processor 401 may be a general-purpose processor or a special-purpose processor, or the like.
  • the processor 401 may also store instructions and/or data 403, and the instructions and/or data 403 may be executed by the processor, so that the electronic device executes the following method embodiments method described in .
  • the processor 401 may include a transceiver unit for implementing the functions of receiving and transmitting.
  • the transceiver unit may be a transceiver circuit, or an interface, or an interface circuit.
  • Transceiver circuits, interfaces or interface circuits used to implement receiving and transmitting functions may be separate or integrated.
  • the above-mentioned transceiver circuit, interface or interface circuit can be used for reading and writing code/data, or the above-mentioned transceiver circuit, interface or interface circuit can be used for signal transmission or transmission.
  • the electronic device may include one or more memories 402 on which instructions 404 may be stored, and the instructions may be executed on the processor, so that the electronic device executes the following method embodiments method described in .
  • data may also be stored in the memory.
  • instructions and/or data may also be stored in the processor.
  • the processor and the memory can be provided separately or integrated together. For example, the corresponding relationships described in the following method embodiments may be stored in a memory or in a processor.
  • the electronic device may further include a transceiver 405 and/or an antenna 406 .
  • the processor 401 may be referred to as a processing unit, and controls the electronic device.
  • the transceiver 405 may be referred to as a transceiver unit, a transceiver, a transceiver circuit or a transceiver, etc., and is used to implement a transceiver function.
  • FIG. 5 is a flowchart 1 of a memory control method provided by an embodiment of the present application, and the execution subject of this embodiment may be the first node in FIG. 3A .
  • the method of this embodiment includes:
  • Step S501 the first node acquires the first total occupied memory of each job included in the job control group, and the job control group includes each unprocessed job on the first node.
  • the first node creates a job control group.
  • the first node assigns the job to the job control group, or binds the process corresponding to the job to the job control group.
  • the first node After the first node obtains the job, it starts a task management process that manages the job, the first node calls the task management process to start the job, and adds the process corresponding to the job to the job control. group to realize the binding of the job to the job control group. That is to say, the job control group includes all jobs that have not been processed on the first node, wherein all the jobs that have not been processed include jobs that are being executed and jobs that are suspended. That is, the first node includes a job control group, and each job acquired by the first node is assigned to the job control group by the first node.
  • the first node may call the control group system monitoring process to monitor the first total occupied memory of each job included in the job control group. Because the control group system monitoring process has the function of monitoring the job control group in real time, when the computing node calls the control group system monitoring process to monitor the total memory occupied by each job included in the job control group, the real-time monitoring is excellent, that is, the first node
  • the first total occupied memory of each job included in the job control group can be acquired in real time, or the first total occupied memory of each unprocessed job on the first node can be acquired in real time.
  • Step S502 When the first total occupied memory satisfies a first preset condition, the first node reduces the total occupied memory of each job included in the job control group, the first preset condition includes greater than the maximum available memory, or the first preset The setting condition includes a first preset ratio that is less than or equal to the maximum available memory and greater than the maximum available memory.
  • the maximum available memory may be the total system memory of the first node.
  • the maximum available memory may be the product of the total system memory of the first node and the maximum available memory ratio (jobcgroup.memory.limit_rate).
  • jobcgroup.memory.limit_rate the maximum available memory ratio
  • the product of the maximum available memory ratio can be read from the configuration file by the first node. After the first node obtains the maximum available memory ratio, it obtains the product of the total system memory of the first node and the maximum available memory ratio to obtain the maximum available memory ratio. available memory.
  • the maximum available memory ratio may be any value from 70% to 95%. For example, if the total system memory of the first node is 100G, and the maximum available memory ratio is 90%, the maximum available memory is 90G.
  • the target node can receive a configuration instruction of the maximum available memory ratio, and the configuration instruction can be used to generate a configuration file.
  • the target node may be the first node, the second node, or other devices, that is, the target node itself is the first node in this embodiment, or the target node is established with the first node in this embodiment.
  • the first node determines, according to the first total occupied memory and the maximum available memory, whether the first total occupied memory satisfies a first preset condition, where the first preset condition includes greater than the maximum available memory, or the first preset condition includes less than or equal to the maximum available memory. or equal to the maximum available memory and greater than the first preset ratio of the maximum available memory.
  • the first node reduces the total occupied memory of each job included in the job control group, for example, by suspending or terminating the job, reducing the total occupied memory of each job included in the job control group
  • the total occupied memory for example, reduces the total occupied memory of each job included in the job control group by migrating part of the memory data to the swap partition.
  • each unprocessed job on the first node is located in the same job control group, so that the first node can obtain the total memory occupied by each unprocessed job on the first node in real time, and in the total Only when the occupied memory is greater than the maximum available memory, or less than or equal to the maximum available memory and greater than the first preset ratio of the maximum available memory, the total occupied memory of each job included in the job control group is reduced.
  • the system memory of the first node can be fully utilized.
  • FIG. 5 The embodiment shown in FIG. 5 will be described in detail below by using a specific embodiment.
  • step S502 shown in FIG. 5 "the first node reduces the total occupied memory of each job included in the job control group" "The specific implementation of step S603 in this embodiment, and the method of this embodiment further includes step S601 and step S602 before step S603:
  • Step S601 the first node determines that the first total occupied memory is less than or equal to the maximum used memory.
  • Step S602 The first node determines first memory information according to the first total occupied memory and the maximum used memory, where the first memory information indicates that the first total occupied memory is greater than a first preset ratio of the maximum available memory.
  • the first node determines the first memory information according to the first total occupied memory and the maximum used memory, including the following a1 to a2:
  • the first node determines the ratio of the first remaining total available memory according to the first total occupied memory and the maximum used memory.
  • the first node may first obtain a first difference between the maximum used memory and the first total occupied memory, and determine the ratio of the first difference to the maximum used memory as the first remaining total available memory ratio. Alternatively, the first node may also obtain the used memory ratio first, and determine the difference between 100% and the used memory ratio as the first remaining total available memory ratio.
  • the first node determines that the first remaining total available memory ratio is smaller than the second preset ratio.
  • the first memory information includes that the ratio of the first remaining total available memory is smaller than the second preset ratio. Because the sum of the first remaining total available memory ratio and the used memory ratio is 100%, and the sum of the second preset ratio and the first preset ratio is 100%, therefore, if the first remaining total available memory ratio is less than the second preset ratio If the ratio is set, the used memory ratio is greater than the first preset ratio. Combined with the above description of the used memory ratio, when the first memory information includes that the first remaining total available memory ratio is smaller than the second preset ratio, the first memory information It may indicate a first preset ratio that the first total occupied memory is greater than the maximum available memory.
  • the second preset ratio is the minimum threshold of the remaining total memory ratio of the first node.
  • the second preset ratio may be read by the first node from the configuration file.
  • the target node further receives a second preset ratio configuration instruction, where the configuration instruction is used to generate a configuration file.
  • the configuration file may include one of the first preset ratio and the second preset ratio.
  • the first node determines the first memory information according to the first total occupied memory and the maximum used memory, including the following b1 to b2:
  • the first node determines the proportion of the total used memory according to the first total occupied memory and the maximum used memory.
  • the first node determines the proportion of the total used memory after determining that the first total occupied memory is less than or equal to the maximum used memory.
  • the used total memory ratio is the ratio of the first total occupied memory to the maximum available memory.
  • the first node determines that the used total memory ratio is greater than the first preset ratio.
  • the first memory information includes: the used total memory ratio is greater than the first preset ratio, and since the used total memory ratio is the ratio of the first total occupied memory to the maximum available memory, the first memory information may indicate that the first memory The total occupied memory is greater than the first preset proportion of the maximum available memory.
  • the first preset ratio is the maximum threshold of the used total memory ratio of the first node.
  • the first preset ratio may be read by the first node from a configuration file.
  • the target node also receives a first preset ratio configuration instruction, where the configuration instruction is used to generate a configuration file.
  • the first memory information in the solutions shown in b1 to b2 includes: the ratio of the total used memory is greater than the first preset ratio. Since the ratio of the total used memory can be directly determined according to the first total occupied memory and the maximum used memory, there is no need to obtain the first The difference between the total occupied memory and the maximum used memory, and the ratio of the difference to the maximum used memory to determine the proportion of the remaining total available memory. Therefore, the solutions shown in b1 to b2 are compared with the solutions shown in a1 to a2. The efficiency of acquiring the first memory information is high.
  • Step S603 the first node processes the process corresponding to the first job in the job control group by using the first processing method, and the first processing method is suspending or terminating.
  • the first node suspends or terminates the first job in the job control group .
  • the number of first jobs may be one or more.
  • the first job may be a job that satisfies the following conditions: the first actual used memory of the first job is greater than the first maximum available memory of the first job, and the memory excess ratio of the first job is the highest .
  • the memory excess ratio of the first job is the ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actual used memory and the first maximum available memory.
  • the first maximum available memory (job-level limit) of the first job may be sent to the first node when the second node allocates the first job to the first node in this embodiment.
  • the first optional manner can specifically suspend or terminate the job with the highest memory exceeding ratio, so as to ensure the normal execution of other jobs on the first node.
  • the first job may be the job that actually uses the highest memory.
  • the second optional manner can quickly reduce the total memory occupied by each job included in the control group.
  • the method further includes: the first node reads processing mode indication information in the configuration file, where the processing mode indication information indicates The first treatment method.
  • the first node can determine the processing method of the first job, so as to realize the correct processing of the first job and facilitate the management of the job by the first node.
  • the configuration file includes: processing mode indication information of at least one job, where the at least one job is used to reduce the total memory occupied by the job control group.
  • the first processing method is suspend, that is, the first node suspends the process corresponding to the first job, then after the first node uses the first processing method to process the process corresponding to the first job in the job control group, it also suspends the process corresponding to the first job. It may include: the first node obtains the third total occupied memory of each job included in the job control group, and when the third total occupied memory is less than a third preset ratio of the maximum available memory, the first node wakes up the process corresponding to the first job, The third preset ratio is less than or equal to the first preset ratio.
  • the third preset ratio may be read by the first node from the configuration file.
  • the target node also receives a third preset ratio configuration instruction input by the user, where the configuration instruction is used to generate a configuration file.
  • the process corresponding to the first job can be woken up, so that the The first job continues to be executed, increasing the reliability of the first node.
  • step S501 before step S501, it further includes step S500, where the first node obtains the second total occupied memory of each job included in the job control group, if the second total occupied memory is greater than the maximum available memory, then The first node migrates part of the memory data corresponding to the job control group to the swap partition of the first node, so that the total occupied memory of each job included in the job control group is the first total occupied memory, and the first total occupied memory is less than or equal to the maximum occupied memory available memory.
  • the first node may call the control group system monitoring process to obtain the second total occupied memory of each job included in the job control group.
  • the schematic diagram corresponding to this embodiment may be shown in FIG. 7
  • the schematic diagram corresponding to the process of this embodiment may be shown in FIG. 8 . shown.
  • a job control group is created.
  • the first node receives a job, it first adds the job to the job queue, and then starts the job and adds the process corresponding to the job to the job control group.
  • the first node calls the control group system monitoring process to obtain the total occupied memory of each job included in the job control group. If the total occupied memory is greater than the maximum available memory, the first node migrates part of the memory data corresponding to the job control group to swap partition. After that, the first node continues to call the control group system monitoring process to obtain the total occupied memory of each job included in the job control group.
  • the first A node obtains first memory information. If the first memory information indicates that the total occupied memory is greater than the first preset ratio of the maximum available memory, the first node suspends the job 1 with the highest memory usage ratio, so as to reduce the number of job control groups including The total memory occupied by each job.
  • suspending or terminating the first job can reduce the number of jobs in the control group.
  • the probability that the total memory of each job is greater than the maximum available memory, thereby reducing the probability of OOM occurrence.
  • the first job will be suspended or terminated, which can reduce the total memory of each job in the control group greater than the maximum available memory.
  • the probability of memory that is, the probability of migrating part of the memory data corresponding to the job control group to the swap partition of the first node can be reduced, thereby reducing the probability of the job execution time being prolonged due to the migration of memory data to the swap partition.
  • FIG. 9 is a third flowchart of a memory control method provided by an embodiment of the present application. This embodiment is applicable to a scenario where the first preset condition includes greater than the maximum available memory. Referring to FIG. 9 , the difference between this embodiment and the embodiment shown in FIG. 5 is that: in this embodiment, step S502 in the embodiment shown in FIG. 5 "the first node reduces each job included in the job control group" The specific implementation of step S902 in this embodiment, and the method of this embodiment further includes step S901 before step S902:
  • Step S901 the first node determines that the first total occupied memory is greater than the maximum available memory.
  • Step S902 the first node processes the process corresponding to the first job in the job control group by using a first processing method, where the first processing method is suspending or terminating.
  • the first processing method may further include: the first node obtains the information included in the job control group.
  • the third total occupied memory of each job when the third total occupied memory is less than the fourth preset ratio of the maximum available memory or less than the maximum available memory, the first node wakes up the process corresponding to the first job.
  • the fourth preset ratio may be the same as or different from the third preset ratio.
  • the fourth preset ratio may be any ratio from 70% to 90%.
  • the first node suspends or terminates the first job, which can reduce the probability of OOM and fully use the system memory of the first node.
  • FIG. 10 is a fourth flowchart of a memory control method provided by an embodiment of the present application. This embodiment is applicable to a scenario where the first preset condition includes greater than the maximum available memory. Referring to FIG. 10 , the difference between this embodiment and the embodiment shown in FIG. 5 is that: in this embodiment, step S502 in the embodiment shown in FIG. 5 "the first node reduces each job included in the job control group" The specific implementation of step S1002 in this embodiment, and the method of this embodiment further includes step S1001 before step S1002:
  • Step S1001 the first node determines that the first total occupied memory is greater than the maximum available memory.
  • Step S1002 The first node migrates at least part of the memory data corresponding to the job control group to the swap partition of the first node, so that the total occupied memory of each job included in the job control group is less than or equal to the maximum available memory.
  • the first node when the first total occupied memory is greater than the maximum available memory, at least part of the memory data corresponding to the job control group is migrated to the swap partition of the first node, so that the total occupied memory of each job included in the job control group is less than or equal to the maximum available memory. That is, when the first total occupied memory is less than or equal to the maximum available memory, the first node does not perform any operation.
  • the first node migrates at least part of the memory data corresponding to the job control group to the swap partition of the first node without suspending or terminating the first node. For some jobs on the node, on the basis of reducing the probability of OOM occurrence and fully using the system memory of the first node, each job on the first node can also run normally.
  • FIG. 11 is a flowchart of the memory control method provided by the Example methods include:
  • Step S1101 the first node determines the first occupied storage space of the swap partition occupied by the job control group.
  • Step S1102 If the first occupied storage space is greater than the maximum available storage space of the swap partition, the first node uses the first processing method to process the process corresponding to the second job in the job control group, so that the swap partition is occupied by the job control group. If the storage space is less than or equal to the maximum available storage space, the first processing method is to suspend or terminate.
  • the second job is the job that occupies the most storage space of the swap partition.
  • the second job is any job in the job control group that occupies the storage space of the swap partition.
  • the method for the first node to obtain the maximum available storage space may be as follows: the first node obtains a first ratio of the maximum used memory to the maximum available storage space of the swap partition, and determines the maximum available storage space of the swap partition according to the first ratio and the maximum used memory storage. For example, if the maximum used memory is 90G, and the first ratio of the maximum used memory to the maximum available storage space of the swap partition is 1:2, then the maximum available storage space of the swap partition is 180G.
  • the first ratio of the maximum used memory to the maximum available storage space of the swap partition may be stored in a configuration file, and the first node reads the first ratio from the configuration file.
  • the target node also receives a configuration instruction of the first ratio input by the user, where the configuration instruction is used to generate a configuration file.
  • the meaning of the target node in this embodiment is the same as the meaning of the target node in the foregoing embodiment, and details are not repeated here.
  • the following c1 to c3 are further included:
  • the first node determines the second occupied storage space of the swap partition occupied by the job control group.
  • the first node determines that the second occupied storage space is less than or equal to the maximum occupied storage space of the swap partition.
  • the first node determines the first storage space information according to the second occupied storage space and the maximum occupied storage space of the swap partition, where the first storage space information indicates that the second occupied storage space is less than or equal to the maximum available storage space of the swap partition. Five preset ratios.
  • the fifth preset ratio may be any value from 60% to 80%.
  • the first node determines the first storage space information according to the second occupied storage space and the maximum occupied storage space of the swap partition, including the following c31-c32:
  • the first node determines the proportion of the total used storage space according to the second occupied storage space and the maximum occupied storage space of the swap partition.
  • the ratio of the total used storage space is the ratio of the second occupied storage space to the maximum occupied storage space of the swap partition.
  • the first node determines that the proportion of the total used storage space is less than or equal to the fifth preset proportion.
  • the first storage space information includes that the ratio of the total storage space used is less than or equal to the preset ratio. Since the ratio of the total storage space used is the ratio of the second occupied storage space to the maximum occupied storage space of the swap partition, the first The storage space information may indicate that the second occupied storage space is less than or equal to a fifth preset ratio of the maximum available storage space of the swap partition.
  • the fifth preset ratio is the maximum threshold value of the used total storage space ratio of the job control group.
  • the fifth preset ratio may be read by the first node from the configuration file.
  • the target node further receives a fifth preset ratio configuration instruction input by the user, where the configuration instruction is used to generate a configuration file.
  • the first node determines the first storage space information according to the second occupied storage space and the maximum occupied storage space of the swap partition, including the following c33 to c34:
  • the first node determines the proportion of the remaining total available storage space according to the second occupied storage space and the maximum occupied storage space of the swap partition.
  • the first node may first obtain the third difference between the maximum occupied storage space of the swap partition and the second occupied storage space, and determine the ratio of the third difference to the maximum occupied storage space of the swap partition as the ratio of the remaining total available storage space.
  • the first node may also obtain the ratio of the total used storage space first, and determine the difference between 100% and the ratio of the total used storage space as the ratio of the remaining total available storage space.
  • the first node determines that the proportion of the remaining total available storage space is greater than or equal to the sixth preset proportion.
  • the first storage space information is information that the ratio of the remaining total available storage space is greater than or equal to the sixth preset ratio. Because the sum of the remaining total available storage space ratio and the used total storage space ratio is 100%, and the sum of the fifth preset ratio and the sixth preset ratio is 100%, therefore, if the remaining total available storage space ratio is less than or equal to the first Six preset ratios, the used total storage space ratio is greater than the fifth preset ratio, combined with the aforementioned description of the used total storage space ratio, the first memory information including the remaining total available storage space ratio is less than or equal to the sixth preset ratio When the ratio is used, the first storage space information may indicate that the second occupied storage space is less than or equal to a fifth preset ratio of the maximum available storage space of the swap partition.
  • the sixth preset ratio is the minimum threshold of the remaining total storage space ratio of the job control group.
  • the sixth preset ratio may be read by the first node from the configuration file.
  • the target node also receives a sixth preset ratio configuration instruction input by the user, where the configuration instruction is used to generate a configuration file.
  • the configuration file may include one of the fifth preset ratio and the sixth preset ratio.
  • the first node wakes up the process corresponding to the second job.
  • the method of this embodiment can prevent the storage space of the swap partition from being used excessively.
  • FIG. 12 is a schematic block diagram of a memory control apparatus provided by an embodiment of the present application.
  • the apparatus of this embodiment includes an acquisition module 1201 and a processing module 1202 .
  • the obtaining module 1201 is configured to obtain the first total occupied memory of each job included in the job control group.
  • the job control group includes every unprocessed job on the first node.
  • a processing module 1202 configured to reduce the total occupied memory of each job included in the job control group when the first total occupied memory satisfies a first preset condition, where the first preset condition includes greater than the maximum available memory, Alternatively, the first preset condition includes a first preset ratio that is less than or equal to the maximum available memory and greater than the maximum available memory.
  • the obtaining module 1201 is specifically configured to: call the control group system monitoring process to obtain the first total occupied memory of each job included in the job control group.
  • the number of the job control group is one, and each job acquired by the first node is added to the job control group by the first node.
  • the processing module 1202 is specifically configured to: adopt a first processing manner to process the process corresponding to the first job in the job control group, where the first processing manner is suspending or terminating.
  • the processing module 1202 before the processing module 1202 adopts the first processing manner to process the process corresponding to the first job in the job control group, the processing module 1202 is further configured to:
  • First memory information is determined according to the first total occupied memory and the maximum used memory, where the first memory information indicates that the first total occupied memory is greater than a first preset ratio of the maximum available memory.
  • the first memory information includes: a ratio of the total used memory is greater than the first preset ratio; and the ratio of the total used memory is a ratio of the first total occupied memory to the maximum available memory.
  • processing module 1202 is further configured to: read the first preset ratio in the configuration file.
  • the first memory information includes: a first remaining total available memory ratio is less than a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 100%; A remaining total available memory ratio is a ratio of a first difference to the maximum available memory, where the first difference is a difference between the maximum available memory and the first total occupied memory.
  • processing module 1202 is further configured to: read the second preset ratio in the configuration file.
  • the obtaining module 1201 is further configured to: obtain the second total occupied memory of each job included in the job control group If the second total occupied memory is greater than the maximum available memory, the processing module 1202 is further configured to: migrate part of the memory data corresponding to each job included in the job control group to the swap partition, so that the job The total occupied memory of each job included in the control group is the first total occupied memory.
  • the first actual used memory of the first job is greater than the first maximum available memory of the first job, and the memory excess ratio of the first job is the highest; the memory excess ratio of the first job is the th The ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actual used memory and the first maximum available memory.
  • the first processing method is suspension, and after the processing module 1202 adopts the first processing method to process the process corresponding to the first job in the job control group: the obtaining module 1201 is further configured to: obtain the job controlling the third total occupied memory of each job included in the group; the processing module 1202 is further configured to: determine that the third total occupied memory is less than a third preset proportion of the maximum available memory, the third preset proportion is less than or equal to the first preset ratio; and wakes up the process corresponding to the first operation.
  • the processing module 1202 is further configured to: read the processing mode indication information in the configuration file , and the processing mode indication information indicates the first processing mode.
  • the processing module 1202 is specifically configured to: migrate at least part of the memory data corresponding to at least one job included in the job control group to the swap partition, so that the total occupied memory of each job included in the job control group is less than or equal to the maximum available memory.
  • the processing module 1202 migrates part of the memory data corresponding to each job included in the job control group to the swap partition, the processing module 1202 is further configured to: if the swap partition is used by the job The first occupied storage space occupied by the control group is larger than the maximum available storage space of the swap partition, and the process corresponding to the second job in the job control group is processed by the first processing method, so that the swap partition is If the storage space occupied by the job control group is less than or equal to the maximum available storage space, the first processing method is to suspend or terminate.
  • the second job is any one of the jobs in the job control group that occupy the swap partition; or, the second job is the swap partition occupied in the job control group. Jobs with the most storage space.
  • the processing module 1202 is further configured to: read the first ratio between the maximum used memory in the configuration file and the maximum available storage space of the swap partition; according to the first ratio and the maximum used memory, Determine the maximum available storage space for the swap partition.
  • processing module 1202 is further configured to: read the maximum available memory ratio in the configuration file; and determine the maximum used memory according to the maximum used memory ratio and the total memory of the first node.
  • the apparatus in this embodiment can be used to execute the technical solutions in the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.
  • FIG. 13 is a schematic block diagram of a memory control system provided by an embodiment of the present application.
  • the system in this embodiment includes at least a first node 1301 and a second node 1302;
  • the second node 1302, configured to send job information to the first node 1301;
  • the first node 1301 configured to obtain a job based on the job information, and add the job to a job control group, where the job control group includes each unprocessed job on the first node 1301;
  • the first node 1301 is further configured to acquire the first total occupied memory of each job included in a job control group, where the job control group includes each unprocessed job on the first node 1301;
  • the first preset condition includes greater than the maximum available memory, or the first The preset condition includes a first preset ratio that is less than or equal to the maximum available memory and greater than the maximum available memory.
  • the first node 1301 is specifically configured to: invoke the control group system monitoring process to obtain the first total occupied memory of each job included in the job control group.
  • the number of the job control group is one, and each job acquired by the first node 1301 is added to the job control group by the first node 1301 .
  • the first node 1301 is specifically configured to: adopt a first processing manner to process the process corresponding to the first job in the job control group, where the first processing manner is suspending or terminating.
  • the first node 1301 before the first node 1301 uses the first processing mode to process the process corresponding to the first job in the job control group, the first node 1301 is further configured to:
  • First memory information is determined according to the first total occupied memory and the maximum used memory, where the first memory information indicates that the first total occupied memory is greater than a first preset ratio of the maximum available memory.
  • the first memory information includes: a ratio of the total used memory is greater than the first preset ratio; and the ratio of the total used memory is a ratio of the first total occupied memory to the maximum available memory.
  • the first node 1301 is further configured to: read the first preset ratio in the configuration file.
  • the first memory information includes: a first remaining total available memory ratio is less than a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 100%; A remaining total available memory ratio is a ratio of a first difference to the maximum available memory, where the first difference is a difference between the maximum available memory and the first total occupied memory.
  • the first node 1301 is further configured to: read the second preset ratio in the configuration file.
  • the first node 1301 before the first node 1301 acquires the first total occupied memory of each job included in the job control group, the first node 1301 is further configured to:
  • the part of memory data corresponding to each job included in the job control group is migrated to the swap partition, so that the total amount of the jobs included in the job control group is The occupied memory is the first total occupied memory.
  • the first actually used memory of the first job is greater than the first maximum available memory of the first job and the memory excess ratio of the first job is the highest;
  • the memory excess ratio of the first job is the ratio of the second difference to the first maximum used memory, and the second difference is the difference between the first actual used memory and the first maximum available memory .
  • the first processing method is suspend, and after the first node 1301 uses the first processing method to process the process corresponding to the first job in the job control group, the first node 1301 also uses the first processing method.
  • the first node 1301 before the first node 1301 uses the first processing mode to process the process corresponding to the first job in the job control group, the first node 1301 is further configured to:
  • the processing mode indication information in the configuration file is read, where the processing mode indication information indicates the first processing mode.
  • the first node 1301 is specifically configured to: migrate at least part of the memory data corresponding to at least one job included in the job control group to the swap partition, so that the total memory occupied by each job included in the job control group Less than or equal to the maximum available memory.
  • the first node 1301 is also used for:
  • the maximum used memory is determined according to the maximum used memory ratio and the total memory of the first node 1301 .
  • the first node 1301 is a computing node or a cloud server in a distributed computing system
  • the second node 1302 is a management node in the distributed computing system
  • the job information includes the job
  • the second node 1302 is a terminal device, and the first node 1301 is an application server; the job information includes a user request, and the job is used to execute the user request.
  • the embodiments of the present application further provide a computer-readable medium on which a computer program is stored, and when the computer program is executed by a computer, implements the method described in any of the foregoing method embodiments.
  • the embodiments of the present application also provide a computer program product, which implements the method described in any of the above method embodiments when the computer program product is executed by a computer.
  • An embodiment of the present application further provides a chip, including a processor, a memory, and a communication interface, wherein the processor and the memory are connected to the communication interface, and it is characterized in that, the processor is configured to read and execute the data stored in the memory.
  • CMOS complementary metal oxide semiconductor
  • NMOS nMetal-oxide-semiconductor
  • PMOS P-type metal oxide semiconductor
  • BJT bipolar junction transistor
  • BiCMOS bipolar CMOS
  • SiGe silicon germanium
  • GaAs gallium arsenide
  • the processor in this embodiment of the present application may be an integrated circuit chip, which has a signal processing capability.
  • each step of the above method embodiments may be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software.
  • the above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other possible Programming logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • the memory in this embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile memory may be random access memory (RAM), which acts as an external cache.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous link dynamic random access memory
  • direct rambus RAM direct rambus RAM
  • the above-described embodiments are implemented using software, they may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line, DSL) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, high-density digital video discs (DVDs)), or semiconductor media (eg, solid state disks, SSD)) etc.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the character "/" generally indicates that the associated objects are an "or” relationship.
  • At least one of or “at least one of” herein mean all or any combination of the listed items, eg, "at least one of A, B, and C", It can be expressed as: A alone exists, B alone exists, C alone exists, A and B exist simultaneously, B and C exist simultaneously, and A, B and C exist simultaneously, where A can be singular or plural, and B can be Singular or plural, C can be singular or plural.
  • B corresponding to A means that B is associated with A, and B can be determined according to A.
  • determining B according to A does not mean that B is only determined according to A, and B may also be determined according to A and/or other information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种内存控制方法和装置，应用于第一节点，该方法包括：第一节点获取作业控制组包括的各作业的第一总占用内存，作业控制组包括该第一节点上未处理完毕的各作业（S501）；第一节点在第一总占用内存满足第一预设条件时，减少作业控制组包括的各作业的总占用内存，第一预设条件包括大于最大可用内存，或者，包括小于或等于该最大可用内存、且大于该最大可用内存的第一预设比例（S502）。第一节点上未处理完毕的各作业均位于同一个作业控制组中，使得第一节点可以实时的获取第一节点上的各作业的总占用内存，且在总占用内存满足上述的第一预设条件时，才减少作业控制组包括的各作业的总占用内存，在避免OOM的同时，还可以充分利用第一节点的系统内存。

Description

内存控制方法和装置 技术领域
本申请涉及计算机技术领域，由于涉及一种内存控制方法和装置。
背景技术
高性能计算(high performance computing，HPC)服务的主要目的是提高运算速度及能力，以达到每秒万亿次级的计算速度。其可以解决大规模科学问题的计算和海量数据的处理，如气象预报、汽车模拟仿真、军事研究、生物制药、基因测序、核爆炸模拟等等。可以提供高性能计算服务的计算机可称之为“高性能计算机”或“HPC计算机”。也就是说，HPC计算机的负载较大，比如运行有大量的作业，因此对HPC计算机的内存控制具有重要的意义。
目前HPC计算机的内存控制的方法有的存在内存溢出(out of memory，简称OOM)的问题，有的存在无法充分使用HPC计算机的系统内存的问题。
发明内容
本申请实施例提供一种内存控制方法和装置，可以充分利用系统内存以及有效的防止内存溢出。
第一方面，本申请实施例提供一种内存控制方法，应用于第一节点，所述方法包括：获取作业控制组包括的各作业的第一总占用内存，所述作业控制组包括所述第一节点上未处理完毕的各作业；在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
可以理解的是，本方案中第一节点调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
本方案中，作业控制组的数量为一个，第一节点获取到的各作业被所述第一节点均添加至作业控制组中。在一种具体的实现中，在第一节点获取到作业后，启动管理该作业的一个任务管理进程，第一节点调用该任务管理进程启动该作业，并将该作业对应的进程添加至该作业控制组中，以实现该作业与该作业控制组的绑定。
其中，最大使用内存的一种确定方法可如下：第一节点读取配置文件中的最大可用内存比例，根据最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。此外，最大使用内存还可以是第一节点的系统内存，本方案中并不限定。
因为控制组系统监控进程具有实时监控作业控制组的功能以及第一节点上未处理完毕的各作业均位于同一个作业控制组中，因此，第一节点可以实时的获取第一节点上未处理完毕的各作业的总占用内存，且在总占用内存大于最大可用内存，或者，小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，才减少作业控制组包括的各作业的总占用内存，因此，本方案的方法在避免内存溢出的同时，还可以充分利用第一节点的系统内存。
在一种可能的实施方式中，所述减少所述作业控制组包括的各作业的总占用内存，包括：采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起 或终止。
其中，在第一种可选的方式中，第一作业可以是满足如下条件的作业：第一作业的第一实际使用内存大于第一作业的第一最大可用内存、且第一作业的内存超出比例最高。其中，第一作业的内存超出比例为第二差值与第一最大使用内存的比值，第二差值为第一实际使用内存与第一最大可用内存的差值。该第一种可选的方式可以有针对性的挂起或终止内存超出比例最高的作业，保证第一节点上的其它作业的正常执行。
在第二种可选的方式中，第一作业可以是实际使用内存最高的作业。该第二种可选的方式可以快速的降低控制组包括的各作业的总占用内存。
可选的，在第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程之前，还包括：第一节点读取配置文件中的处理方式指示信息，该处理方式指示信息指示第一处理方式。也就是说，第一节点可根据从配置文件中读取的处理方式指示信息得到的第一处理方式。该可选的方式，可以使得第一节点确定第一作业的处理方式，以实现对第一作业的正确处理，方便了第一节点对作业的管理。
本方案通过挂起或终止第一作业的方式减少所述作业控制组包括的各作业的总占用内存，保证了除了第一作业以外的其它作业的正常执行。
在一种可能的实施方式中，在所述采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，还包括：确定所述第一总占用内存小于或等于所述最大使用内存；根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
可选的，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。其中，第二预设比例为第一节点的剩余总内存比例的最小阈值。第二预设比例可以是第一节点从配置文件中读取的。
可选的，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。其中，第一预设比例为第一节点的已用总内存比例的最小阈值。第一预设比例可以是第一节点从配置文件中读取的。
也就是说，本方案在第一总占用内存满足第一预设条件且第一预设条件为小于或等于所述最大使用内存、且大于所述最大可用内存的第一预设比例时，采用第一处理方式处理所述作业控制组中的第一作业对应的进程。由于在第一总占用内存大于最大可用内存的第一预设比例后，就采用第一处理方式处理所述作业控制组中的第一作业对应的进程，因此，本方案可以有效的降低作业控制组包括的各作业的总占用内存大于最大使用内存的几率。
在一种可能的实施方式中，在所述获取作业控制组包括的各作业的第一总占用内存之前，还包括：获取作业控制组包括的各作业的第二总占用内存；若所述第二总占用内存大于所述最大可用内存，则所述将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。在第一总占用内存满足第一预设条件且第一预设条件为小于或等于所述最大使用内存、且大于所述最大可用内存的第一预设比例时，采用第一处理方式处理作业控制中的第一作业，第一处理方式为挂起或终止。
在本方案中，可以有效防止第一节点的内存溢出，且由于在控制组中的各作业的总占用内存小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，便挂起或终止第一 作业，可以降低控制组中的各作业的总内存大于最大可用内存的几率，也就是可以降低将作业控制组对应的部分内存数据迁移至第一节点的swap分区的概率，进而降低了作业由于内存数据被迁移至swap分区造成的作业执行时间变长的概率。
在一种可能的实施方式中，所述第一处理方式为挂起，在第一总占用内存满足第一预设条件且第一预设条件为小于或等于所述最大使用内存、且大于所述最大可用内存的第一预设比例时，采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，还包括：获取作业控制组包括的各作业的第三总占用内存；确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；唤醒所述第一作业对应的进程。
也就是说，本方案中在第一总占用内存满足第一预设条件且第一预设条件为小于或等于所述最大使用内存、且大于所述最大可用内存的第一预设比例时，第一节点将第一作业对应的进程挂起之后，若监控到第三总占用内存小于最大可用内存的第三预设比例，则可以唤醒第一作业对应的进程，以使得第一作业继续被执行，增加了第一节点的可靠性。
在一种可能的实施方式中，所述第一处理方式为挂起，在第一总占用内存满足第一预设条件且第一预设条件为大于所述最大使用内存时，采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，还包括：获取作业控制组包括的各作业的第三总占用内存；确定所述第三总占用内存小于所述最大可用内存的第四预设比例；唤醒所述第一作业对应的进程。
也就是说，本方案中在第一总占用内存满足第一预设条件且第一预设条件为大于最大可用内存时，第一节点将第一作业对应的进程挂起之后，若监控到第三总占用内存小于最大可用内存的第四预设比例，则可以唤醒第一作业对应的进程，以使得第一作业继续被执行，增加了第一节点的可靠性。
在一种可能的实施方式中，所述减少所述作业控制组包括的各作业的总占用内存，包括：将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
本方案中将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存，可以有效的防止内存溢出。
在一种可能的实施方式中，第一节点将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区之后，还包括：若所述swap分区被所述作业控制组所占用的第一占用存储空间满足大于所述swap分区的最大可用存储空间，采用第一处理方式处理所述作业控制组中的第二作业对应的进程，以使所述swap分区被所述作业控制组所占用存储空间小于或等于所述最大可用存储空间，第一处理方式为挂起或终止。
其中，所述第二作业为所述作业控制组中占用所述swap分区的作业中的任意一个作业；或者，所述第二作业为所述作业控制组中占用的所述swap分区的存储空间最多的作业。第一节点确定swap分区的最大可用存储空间的方法可如下：第一节点读取配置文件的最大使用内存与所述swap分区的最大可用存储空间的第一比值；根据所述第一比值和所述最大使用内存，确定所述swap分区的最大可用存储空间。
本方案可以防止swap分区的存储空间被过度使用。
第二方面，本申请实施例提供一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指 令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行第一方面或者第一方面任一可能的实施方式中所述的方法。
第三方面，本申请实施例提供一种内存控制系统，包括第二节点和至少一个第一节点；所述第二节点，用于向所述第一节点发送作业信息；所述第一节点，用于基于所述作业信息得到作业，以及将所述作业添加至作业控制组，所述作业控制组包括所述第一节点上未处理完毕的各作业；所述第一节点，还用于获取作业控制组包括的各作业的第一总占用内存，所述作业控制组包括所述第一节点上未处理完毕的各作业；在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
在一种可能的实施方式中，所述第一节点具体用于：调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
在一种可能的实施方式中，所述作业控制组的数量为一个，所述第一节点获取到的各作业被所述第一节点添加至所述作业控制组中。
在一种可能的实施方式中，所述第一节点具体用于：采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
在一种可能的实施方式中，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点还用于：确定所述第一总占用内存小于或等于所述最大使用内存；根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
在一种可能的实施方式中，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
在一种可能的实施方式中，所述第一节点还用于：读取配置文件中的所述第一预设比例。
在一种可能的实施方式中，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
在一种可能的实施方式中，所述第一节点还用于：读取配置文件中的所述第二预设比例。
在一种可能的实施方式中，在所述第一节点获取作业控制组包括的各作业的第一总占用内存之前，所述第一节点还用于：获取作业控制组包括的各作业的第二总占用内存；若所述第二总占用内存大于所述最大可用内存，则所述将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
在一种可能的实施方式中，所述第一作业的第一实际使用内存大于所述第一作业的第一最大可用内存且所述第一作业的内存超出比例最高；所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
在一种可能的实施方式中，所述第一处理方式为挂起，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，所述第一节点还用于：获取作业控制组包括的各作业的第三总占用内存；确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；唤醒所述第一作业对应的进程。
在一种可能的实施方式中，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点还用于：读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
在一种可能的实施方式中，所述第一节点具体用于：将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
在一种可能的实施方式中，所述第一节点还用于：读取配置文件中的最大可用内存比例；根据所述最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。
在一种可能的实施方式中，所述第一节点为分布式计算系统中的计算节点或云服务器，所述第二节点为所述分布式计算系统中的管理节点；所述作业信息包括所述作业。
在一种可能的实施方式中，所述第二节点为终端设备，所述第一节点为应用服务器；所述作业信息包括用户请求，所述作业用于执行所述用户请求。
第四方面，本申请实施例提供一种内存控制装置，该装置包括：获取模块，用于获取作业控制组包括的各作业的第一总占用内存所述作业控制组包括所述第一节点上未处理完毕的各作业。处理模块，用于在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
在一种可能实施方式中，所述获取模块具体用于：调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
在一种可能实施方式中，所述作业控制组的数量为一个，所述第一节点获取到的各作业被所述第一节点添加至所述作业控制组中。
在一种可能实施方式中，所述处理模块具体用于：采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
在一种可能实施方式中，在所述处理模块采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述处理模块还用于：确定所述第一总占用内存小于或等于所述最大使用内存；根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
在一种可能实施方式中，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
在一种可能实施方式中，所述处理模块还用于：读取配置文件中的所述第一预设比例。
在一种可能实施方式中，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
在一种可能实施方式中，所述处理模块还用于：读取配置文件中的所述第二预设比例。
在一种可能实施方式中，在所述获取模块在获取作业控制组包括的各作业的第一总占用内存之前：所述获取模块还用于：获取作业控制组包括的各作业的第二总占用内存；若所述第二总占用内存大于所述最大可用内存，则所述处理模块还用于：将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
在一种可能实施方式中，所述第一作业的第一实际使用内存大于所述第一作业的第一最 大可用内存且所述第一作业的内存超出比例最高；所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
在一种可能实施方式中，所述第一处理方式为挂起，在处理模块采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后：所述获取模块还用于：获取作业控制组包括的各作业的第三总占用内存；所述处理模块还用于：确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；以及唤醒所述第一作业对应的进程。
在一种可能实施方式中，在所述处理模块采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述处理模块还用于：读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
在一种可能实施方式中，所述处理模块具体用于：将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
在一种可能实施方式中，在所述处理模块将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区之后，所述处理模块还用于：若所述swap分区被所述作业控制组所占用的第一占用存储空间满足大于所述swap分区的最大可用存储空间，采用第一处理方式处理所述作业控制组中的第二作业对应的进程，以使所述swap分区被所述作业控制组所占用存储空间小于或等于所述最大可用存储空间，第一处理方式为挂起或终止。
在一种可能实施方式中，所述第二作业为所述作业控制组中占用所述swap分区的作业中的任意一个作业；或者，所述第二作业为所述作业控制组中占用的所述swap分区的存储空间最多的作业。
在一种可能实施方式中，所述处理模块还用于：读取配置文件中的最大使用内存与所述swap分区的最大可用存储空间的第一比值；根据所述第一比值和所述最大使用内存，确定所述swap分区的最大可用存储空间。
在一种可能实施方式中，所述处理模块还用于：读取配置文件中的最大可用内存比例；根据所述最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。
第五方面，本申请实施例提供一种存储介质，所述存储介质包括计算机程序，所述计算机程序用于实现如第一方面或第一方面任一可能的实施方式中所述的方法。
第六方面，本申请实施例提供一种芯片，包括处理器、存储器和通信接口，所述处理器、存储器与所述通信接口连接，其特征在于，所述处理器用于读取并执行所述存储器中存储的计算机程序，以执行第一方面或第一方面任一可能的实施方式中所述的方法。
附图说明
图1为目前的一种内存控制方法的原理示意图；
图2为目前的另一种内存控制方法的原理示意图；
图3A为本申请实施例提供的一种系统架构图；
图3B为本申请实施例提供的分布式计算系统下一种具体系统架构图；
图4为本申请实施例的电子设备的一种示意性框图；
图5为本申请实施例提供的内存控制方法的流程图一；
图6为本申请实施例提供的内存控制方法的流程图二；
图7为图6所示的方法实施例的原理示意图；
图8为图6所示的方法实施例的过程示意图；
图9为本申请实施例提供的内存控制方法的流程图三；
图10为本申请实施例提供的内存控制方法的流程图四；
图11为本申请实施例提供的内存控制方法的流程图五；
图12为本申请实施例提供的内存控制装置的示意性框图；
图13为本申请实施例提供内存控制系统的示意性框图。
具体实施方式
首先对本申请涉及的要素进行说明。
1、分布式计算系统：分布式计算系统包括多台通过网络互联的计算机，比如多台通过网络互联的HPC计算机。分布式计算系统中的多台计算机可包括至少一个管理节点和多个计算节点，管理节点用于接收用户提交的作业，并将作业分配至计算节点，计算节点执行管理节点分配的作业。一个作业可被分配至一个计算节点或多个计算节点来执行。
2、作业(job)：完成一项特定的计算业务所需要执行的一组程序实例的集合，通常对应于一台或多台计算机上的一组进程、容器或其他运行时实体。也就是说，一个作业对应多个进程。比如，在分布式管理系统中，管理节点接收用户的请求后，会生成至少一个处理该请求的作业，并将作业分配至计算节点中执行。又比如在其它的一些场景中，终端设备接收到用户针对某一应用的请求后，将该请求发送至该应用的应用服务器，该应用服务器生成至少一个处理该请求的作业，并执行至少一个作业。
3、内存过度使用或过载(overuse)：会导致同一台机器上的多个作业恶性竞争资源，作业执行慢或异常退出：如：处理器被其他作业占用，系统进程(如nfsd)受影响导致系统服务响应慢，严重时可能导致机器因内存溢出(out of memory，OOM)而宕机。
4、内存未充分使用(underuse)：内存资源利用率低，可能导致作业排队等待时间更久，吞吐量降低。
5、控制组(cgroup)，它提供了一套机制用于控制一组特定进程对资源的使用。cgroup绑定一个进程集合到一个或多个子系统上。其中，子系统是一个通过cgroup提供的工具和接口来管理进程集合的模块。一个子系统就是一个典型的“资源控制器”，用来调度资源或者控制资源使用的上限。
为了更好的理解本申请，下面对目前存在的技术问题进行说明。
作业内存限制(Job Limit)方法是分布式计算系统中应用的一种内存控制方法。对于该种方法，参见图1，管理节点接收用户提交的作业，作业中携带有作业的最大可用内存(job-level limit)，管理节点将该作业分配至计算节点，并携带该作业的最大可用内存。计算节点执行作业时，周期性的监控作业的实际使用内存，若该作业的实际使用内存大于该作业的最大可用内存，则计算节点挂起或终止该作业。比如，某个作业的最大可用内存为10G，当该作业的实际使用内存大于10G，该作业就会被挂起或终止。
该方法一方面由于计算节点在监控作业的实际使用内存时，无法做到实时监控，在监控的间隔期，存在作业的实际使用内存大于该作业的最大可用内存的可能，从而有可能发生OOM。另一方面，在某个作业实际使用内存大于该作业的最大可用内存时，即使此时计算节 点的系统内存有大量的闲置内存，该作业还是被挂起或终止，因此无法充分利用内存。
硬资源限制是分布式计算系统中应用的另一种内存控制方法。对于该种方法，参见图2，计算节点为每个作业创建一个控制组(cgroup)，当计算节点调用控制组系统监控进程，来监控作业的实际使用内存，当监控到作业的实际使用内存大于该作业的最大使用内存时，将该作业的部分内存数据迁移至swap分区中，当该作业的实际使用内存大于该作业的最大使用内存且该作业占用的swap分区的存储空间大于该作业的swap分区最大使用存储空间(job swap limit)时，计算节点才会挂起或者终止该作业。
该方法由于为每个作业创建一个控制组，计算节点可以调用控制组系统监控进程实现对作业的实际使用内存的实时监控，几乎避免了OOM。但是，该方法中当作业的实际使用内存大于该作业的最大使用内存时，即使此时计算节点的系统内存有大量的闲置内存，还是将该作业的部分内存数据迁移至swap分区中，因此也无法充分利用内存。
为了解决上述技术问题，提出了本申请实施例的方法。
图3A为本申请实施例提供的一种系统架构图。参见图3A，该系统架构包括至少一个第二节点和至少一个第一节点。其中，第二节点可以是分布式计算系统中的管理节点，也可以是终端设备，还可以是其它的服务器；第一节点可以是分布式计算系统中的计算节点或云服务器，也可以是其它的服务器，比如云服务器或应用服务器。例如，在第二节点为分布式计算系统中的管理节点时，第一节点可以是分布式计算系统中的计算节点或云服务器且第一节点的数量为多个。又例如，在第二节点为终端设备时，第一节点可以是应用服务器。
其中，第一节点运行有作业，第一节点中的作业可以是第二节点分配的，也可以是基于第一节点发送的请求生成的。
例如，在本实施例的方法应用于分布式计算系统时，一种具体的系统架构图可如图3B所示。参见图3B，该系统架构包括管理节点和计算节点，还可包括运服务器，计算节点和云服务器可为图3A中所示的第一节点，管理节点可为图3B中所示的第二节点。
图4为本申请实施例的电子设备的一种示意性框图。本实施例的电子设备可以是上述系统架构中的第一节点，也可以是支持第一节点实现下述方法的芯片、芯片系统、或处理器等，该电子设备可用于实现下述方法实施例中描述的第一节点对应的方法，具体可以参见述方法实施例中的说明。
所述电子设备可以包括一个或多个处理器401，所述处理器401也可以称为处理单元，可以实现一定的控制功能。所述处理器401可以是通用处理器或者专用处理器等。
在一种可选的设计中，处理器401也可以存有指令和/或数据403，所述指令和/或数据403可以被所述处理器运行，使得所述电子设备执行下述方法实施例中描述的方法。
在另一种可选的设计中，处理器401中可以包括用于实现接收和发送功能的收发单元。例如该收发单元可以是收发电路，或者是接口，或者是接口电路。用于实现接收和发送功能的收发电路、接口或接口电路可以是分开的，也可以集成在一起。上述收发电路、接口或接口电路可以用于代码/数据的读写，或者，上述收发电路、接口或接口电路可以用于信号的传输或传递。
可选的，所述电子设备中可以包括一个或多个存储器402，其上可以存有指令404，所述指令可在所述处理器上被运行，使得所述电子设备执行下述方法实施例中描述的方法。可选的，所述存储器中还可以存储有数据。可选的，处理器中也可以存储指令和/或数据。所述处理器和存储器可以单独设置，也可以集成在一起。例如，下述方法实施例中所描述的对应关系可以存储在存储器中，或者存储在处理器中。
可选的，所述电子设备还可以包括收发器405和/或天线406。所述处理器401可以称为处理单元，对所述电子设备进行控制。所述收发器405可以称为收发单元、收发机、收发电路或者收发器等，用于实现收发功能。
下面采用具体的实施例对本申请的内存控制方法进行说明。
图5为本申请实施例提供的内存控制方法的流程图一，本实施例的执行主体可为图3A中的第一节点。参见图5，本实施例的方法包括：
步骤S501、第一节点获取作业控制组包括的各作业的第一总占用内存，作业控制组包括第一节点上未处理完毕的各作业。
其中，在第一节点的初始化阶段，第一节点创建一个作业控制组。当第一节点获取到作业时，第一节点将该作业分配至该作业控制组，或者说将该作业对应的进程绑定至该作业控制组。一种具体的实现中，在第一节点获取到作业后，启动管理该作业的一个任务管理进程，第一节点调用该任务管理进程启动该作业，并将该作业对应的进程添加至该作业控制组中，以实现该作业与该作业控制组的绑定。也就是说，作业控制组包括第一节点上未处理完毕的所有作业，其中，未处理完毕的所有作业包括正在被执行的作业和被挂起的作业。也就是说，第一节点包括一个作业控制组，第一节点获取到的每个作业均被第一节点分配至该作业控制组中。
第一节点可调用控制组系统监控进程，来监控作业控制组包括的各作业的第一总占用内存。因为控制组系统监控进程具有实时监控作业控制组的功能，因此，计算节点调用控制组系统监控进程，来监控作业控制组包括的各作业的总占用内存时，监控实时性优异，即第一节点可以实时的获取作业控制组包括的各作业的第一总占用内存或者说可以实时的获取第一节点上未处理完毕的各作业的第一总占用内存。
步骤S502、第一节点在该第一总占用内存满足第一预设条件时，减少作业控制组包括的各作业的总占用内存，第一预设条件包括大于最大可用内存，或者，第一预设条件包括小于或等于最大可用内存、且大于最大可用内存的第一预设比例。
一种方案中，最大可用内存可以是第一节点的系统总内存。
另一种方案中，最大可用内存可以是第一节点的系统总内存与最大可用内存比例(jobcgroup.memory.limit_rate)的乘积。其中，最大可用内存比例的乘积可以是第一节点从配置文件中读取的，第一节点获取到最大可用内存比例后，获取第一节点的系统总内存与最大可用内存比例的乘积，得到最大可用内存。可选的，最大可用内存比例可为70％～95％中的任一数值。比如，第一节点的系统总内存为100G，最大可用内存比例为90％，则最大可用内存为90G。
其中，目标节点可接收最大可用内存比例配置指令，该配置指令可用于生成配置文件。目标节点可以是该第一节点，可以是第二节点，还可以是其它的设备，也就是说目标节点为本实施例的第一节点本身，或者目标节点为与本实施例的第一节点建立有通信连接的且能够使得本实施例的第一节点读取到该配置文件的节点。由于配置文件中包括用户设置的最大可用内存比例，在第一节点从配置文件中读取最大可用内存比例，以得到最大可用内存时，可使得第一节点确定的最大可用内存比较合理。
第一节点根据该第一总占用内存和最大可用内存，判断该第一总占用内存是否满足第一预设条件，第一预设条件包括大于最大可用内存，或者，第一预设条件包括小于或等于最大可用内存、且大于最大可用内存的第一预设比例。若该第一总占用内存满足第一预设条件，则第一节点减少作业控制组包括的各作业的总占用内存，比如可通过挂起或终止作业的方式 减少作业控制组包括的各作业的总占用内存，又比如，通过将部分内存数据迁移至swap分区的方式减少作业控制组包括的各作业的总占用内存。
本实施例中，第一节点上未处理完毕的各作业均位于同一个作业控制组中，使得第一节点可以实时的获取第一节点上未处理完毕的各作业的总占用内存，且在总占用内存大于最大可用内存，或者，小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，才减少作业控制组包括的各作业的总占用内存，在避免OOM的同时，还可以充分利用第一节点的系统内存。
下面采用具体的实施例对图5所示的实施例进行详细说明。
图6为本申请实施例提供的内存控制方法的流程图二，本实施例适用于第一预设条件包括小于或等于最大可用内存、且大于最大可用内存的第一预设比例的场景。参见图6，本实施例的方法与图5所示的方法不同之处在于：在本实施例中，图5所示的步骤S502“第一节点减少作业控制组包括的各作业的总占用内存”的具体实现为本实施例中步骤S603，以及本实施例的方法在步骤S603之前还包括步骤S601和步骤S602：
步骤S601、第一节点确定第一总占用内存小于或等于最大使用内存。
步骤S602、第一节点根据第一总占用内存和最大使用内存确定第一内存信息，第一内存信息指示第一总占用内存大于最大可用内存的第一预设比例。
在一种方案中，第一节点根据第一总占用内存和最大使用内存，确定第一内存信息，包括如下的a1～a2：
a1、第一节点根据第一总占用内存和最大使用内存，确定第一剩余总可用内存比例。
其中，第一节点可先获取最大使用内存与第一总占用内存的第一差值，确定第一差值和最大使用内存的比值为第一剩余总可用内存比例。或者，第一节点还可以先获取已用内存比例，确定100％与已用内存比例的差值为第一剩余总可用内存比例。
a2、第一节点确定第一剩余总可用内存比例小于第二预设比例。
此时，第一内存信息包括第一剩余总可用内存比例小于第二预设比例。因为第一剩余总可用内存比例与已用内存比例的和为100％，第二预设比例和第一预设比例的和为100％，因此，若第一剩余总可用内存比例小于第二预设比例，则已用内存比例大于第一预设比例，结合前述对已用内存比例的说明，在第一内存信息包括第一剩余总可用内存比例小于第二预设比例时，第一内存信息可指示第一总占用内存大于最大可用内存的第一预设比例。
其中，第二预设比例为第一节点的剩余总内存比例的最小阈值。第二预设比例可以是第一节点从配置文件中读取的。相应地，目标节点还接收第二预设比例配置指令，该配置指令用于生成配置文件。可以理解的是，配置文件中可包括第一预设比例、第二预设比例中的一项。
在另一种方案中，第一节点根据第一总占用内存和最大使用内存，确定第一内存信息，包括如下的b1～b2：
b1、第一节点根据第一总占用内存和最大使用内存，确定已用总内存比例。
其中，第一节点确定第一总占用内存小于或等于最大使用内存之后，确定已用总内存比例。已用总内存比例为第一总占用内存与最大可用内存的比值。
b2、第一节点确定已用总内存比例大于第一预设比例。
此时，第一内存信息包括：已用总内存比例大于第一预设比例，由于已用总内存比例为第一总占用内存与最大可用内存的比值，因此，第一内存信息可指示第一总占用内存大于最大可用内存的第一预设比例。
其中，第一预设比例为第一节点的已用总内存比例的最大阈值。第一预设比例可以是第一节点从配置文件中读取的。相应地，目标节点还接收第一预设比例配置指令，该配置指令用于生成配置文件。
b1～b2所示的方案中的第一内存信息包括：已用总内存比例大于第一预设比例，由于已用总内存比例可直接根据第一总占用内存和最大使用内存确定，无需获取第一总占用内存和最大使用内存的差值、以及根据该差值与最大使用内存的比值确定剩余总可用内存比例，因此b1～b2所示的方案相对于a1～a2所示的方案而言，获取第一内存信息的效率较高。
步骤S603、第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程，第一处理方式为挂起或终止。
也就是说，本实施例中在第一总占用内存满足小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，第一节点挂起或终止作业控制组中的第一作业。第一作业的数量可以是一个或多个。
在第一种可选的方式中，第一作业可以是满足如下条件的作业：第一作业的第一实际使用内存大于第一作业的第一最大可用内存、且第一作业的内存超出比例最高。其中，第一作业的内存超出比例为第二差值与第一最大使用内存的比值，第二差值为第一实际使用内存与第一最大可用内存的差值。第一作业的第一最大可用内存(job-level limit)可以是第二节点向本实施例中的第一节点分配第一作业时，发送至第一节点的。
该第一种可选的方式可以有针对性的挂起或终止内存超出比例最高的作业，保证第一节点上的其它作业的正常执行。
在第二种可选的方式中，第一作业可以是实际使用内存最高的作业。该第二种可选的方式可以快速的降低控制组包括的各作业的总占用内存。
可选的，在第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程之前，还包括：第一节点读取配置文件中的处理方式指示信息，该处理方式指示信息指示第一处理方式。该可选的方式，可以使得第一节点确定第一作业的处理方式，以实现对第一作业的正确处理，方便了第一节点对作业的管理。
也就是说，配置文件中包括：至少一个作业的处理方式指示信息，该至少一个作业用于减少作业控制组的总占用内存时。
可选的，第一处理方式为挂起，即第一节点挂起第一作业对应的进程，那么在第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程之后，还可包括：第一节点获取作业控制组包括的各作业的第三总占用内存，在第三总占用内存小于最大可用内存的第三预设比例时，第一节点唤醒第一作业对应的进程，第三预设比例小于或等于第一预设比例。
在第三预设比例和第一预设比例不同时，第三预设比例可以是第一节点从配置文件中读取的。相应地，目标节点还接收用户输入的第三预设比例配置指令，该配置指令用于生成配置文件。
也就是说，在第一节点将第一作业对应的进程挂起之后，若监控到第三总占用内存小于最大可用内存的第三预设比例，则可以唤醒第一作业对应的进程，以使得第一作业继续被执行，增加了第一节点的可靠性。
可选的，在本实施例中，在步骤S501之前，还包括步骤S500、第一节点获取作业控制组包括的各作业的第二总占用内存，若第二总占用内存大于最大可用内存，则第一节点将作业控制组对应的部分内存数据迁移至第一节点的swap分区，以使作业控制组包括的各作业的总占用内存为第一总占用内存，第一总占用内存小于或等于最大可用内存。其中，第一节点可 调用控制组系统监控进程，来获取作业控制组包括的各作业的第二总占用内存。此时，在第一节点为分布式计算系统中的计算节点，第二节点为管理节点时，该实施例对应的原理示意图可如图7所示，该实施例对应的过程示意图可如图8所示。
参见图8，第一节点启动后，创建作业控制组。第一节点每接收到一个作业，先将该作业加入到作业队列中，然后启动作业后将该作业对应的进程加入到作业控制组中。第一节点调用控制组系统监控进程，来获取作业控制组包括的各作业的总占用内存，若该总占用内存大于最大可用内存，则第一节点将作业控制组对应的部分内存数据迁移至swap分区。之后，第一节点继续调用控制组系统监控进程，来获取作业控制组包括的各作业的总占用内存，若此时作业控制组包括的各作业的总占用内存小于或等于最大可用内存，则第一节点获取第一内存信息，若第一内存信息指示该总占用内存大于最大可用内存的第一预设比例，则第一节点挂起内存使用超出比例最高的作业1，以减少作业控制组包括的各作业的总占用内存。
本实施例中，在控制组中的各作业的总占用内存小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，挂起或终止第一作业，可以降低控制组中的各作业的总内存大于最大可用内存的几率，进而降低OOM发生的概率。
此外，在若控制组中的各作业的总占用内存大于最大可用内存，则将作业控制组对应的部分内存数据迁移至第一节点的swap分区的场景下，本实施例中的在控制组中的各作业的总占用内存小于或等于最大可用内存、且大于最大可用内存的第一预设比例时，便挂起或终止第一作业，可以降低控制组中的各作业的总内存大于最大可用内存的几率，也就是可以降低将作业控制组对应的部分内存数据迁移至第一节点的swap分区的概率，进而降低了作业由于内存数据被迁移至swap分区造成的作业执行时间变长的概率。
图9为本申请实施例提供的内存控制方法的流程图三，本实施例适用于第一预设条件包括大于最大可用内存的场景。参见图9，本实施例与图5所示的实施例的不同之处在于：在本实施例中，图5所示的实施例中的步骤S502“第一节点减少作业控制组包括的各作业的总占用内存”的具体实现为本实施例中步骤S902，以及本实施例的方法在步骤S902之前还包括步骤S901：
步骤S901、第一节点确定该第一总占用内存大于最大可用内存。
步骤S902、第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程，第一处理方式为挂起或终止。
本实施例中的第一处理方式、第一作业参照上一实施例中的说明，此处不再赘述。
可选的，若第一处理方式为挂起，那么在第一节点采用第一处理方式处理作业控制组中的第一作业对应的进程之后，还可包括：第一节点获取作业控制组包括的各作业的第三总占用内存，在第三总占用内存小于最大可用内存的第四预设比例或者小于最大可用内存，第一节点唤醒第一作业对应的进程。其中，第四预设比例可以与第三预设比例相同，与可以不相同。可选的，第四预设比例可为70％～90％中的任意一个比例。
本实施例中在控制组中的各作业的总占用内存大于最大可用内存时，第一节点挂起或终止第一作业，可以降低OOM发生的概率以及充分使用第一节点的系统内存。
图10为本申请实施例提供的内存控制方法的流程图四。本实施例适用于第一预设条件包括大于最大可用内存的场景。参见图10，本实施例与图5所示的实施例的不同之处在于：在本实施例中，图5所示的实施例中的步骤S502“第一节点减少作业控制组包括的各作业的总占用内存”的具体实现为本实施例中步骤S1002，以及本实施例的方法在步骤S1002之前还包括步骤S1001：
步骤S1001、第一节点确定该第一总占用内存大于最大可用内存。
步骤S1002、第一节点将作业控制组对应的至少部分内存数据迁移至第一节点的swap分区，以使作业控制组包括的各作业的总占用内存小于或等于最大可用内存。
本实施例中在该第一总占用内存大于最大可用内存时，将作业控制组对应的至少部分内存数据迁移至第一节点的swap分区，以使作业控制组包括的各作业的总占用内存小于或等于最大可用内存即可。也就是说，在该第一总占用内存小于或等于最大可用内存时，第一节点不执行任何操作。
本实施例中在控制组中的各作业的总占用内存大于最大可用内存时，第一节点将作业控制组对应的至少部分内存数据迁移至第一节点的swap分区，无需挂起或终止第一节点上的部分作业，在降低OOM发生的概率以及充分使用第一节点的系统内存的基础上，还可使得第一节点上的各作业正常运行。
针对上述实施例中在控制组中的各作业的总占用内存大于最大可用内存时，第一节点将作业控制组对应的至少部分内存数据迁移至第一节点的swap分区的场景，为了防止swap分区的存储空间被过度使用(overuse)，本实施例在图上述实施例的基础上做了进一步的改进，图11为本申请实施例提供的内存控制方法的流程图五，参见图11，本实施例的方法包括：
步骤S1101、第一节点确定swap分区被作业控制组所占用的第一占用存储空间。
步骤S1102、若第一占用存储空间大于swap分区的最大可用存储空间，则第一节点采用第一处理方式处理作业控制组中的第二作业对应的进程，以使swap分区被作业控制组所占用存储空间小于或等于该最大可用存储空间，第一处理方式为挂起或终止。
可选的，第二作业为占用的swap分区的存储空间最多的作业。
可选的，第二作业为作业控制组中占用swap分区的存储空间的作业中的任意一个作业。
其中，第一节点获取最大可用存储空间的方法可如下：第一节点获取最大使用内存与swap分区的最大可用存储空间的第一比值，根据第一比值和最大使用内存，确定swap分区的最大可用存储空间。比如，最大使用内存为90G，最大使用内存与swap分区的最大可用存储空间的第一比值为1:2，则swap分区的最大可用存储空间为180G。
其中，该最大使用内存与swap分区的最大可用存储空间的第一比值可以存储在配置文件中，第一节点从配置文件中读取该第一比值。相应地，目标节点还接收用户输入的该第一比值的配置指令，该配置指令用于生成配置文件。本实施例中的目标节点的含义和上述实施例中的目标节点的含义相同，此处不再赘述。
可选的，若第一处理方式为挂起，则在第一节点采用第一处理方式处理作业控制组中的第二作业对应的进程之后，还包括如下的c1～c3：
c1、第一节点确定swap分区被作业控制组所占用的第二占用存储空间。
c2、第一节点确定第二占用存储空间小于或等于swap分区的最大占用存储空间。
c3、第一节点根据第二占用存储空间和swap分区的最大占用存储空间，确定第一存储空间信息，第一存储空间信息指示第二占用存储空间小于或等于swap分区的最大可用存储空间的第五预设比例。
可选的，第五预设比例可为60％～80％中的任意一个数值。
在一种方案中，第一节点根据第二占用存储空间和swap分区的最大占用存储空间，确定第一存储空间信息，包括如下的c31～c32：
c31、第一节点根据第二占用存储空间和swap分区的最大占用存储空间，确定已用总存储空间比例。
其中，已用总存储空间比例为第二占用存储空间与swap分区的最大占用存储空间的比值。
c32、第一节点确定已用总存储空间比例小于或等于第五预设比例。
此时，第一存储空间信息包括已用总存储空间比例小于或等于预设比例，由于已用总存储空间比例为第二占用存储空间与swap分区的最大占用存储空间的比值，因此，第一存储空间信息可指示第二占用存储空间小于或等于swap分区的最大可用存储空间的第五预设比例。
其中，第五预设比例为作业控制组的已用总存储空间比例的最大阈值。第五预设比例可以是第一节点从配置文件中读取的。相应地，目标节点还接收用户输入的第五预设比例配置指令，该配置指令用于生成配置文件。
在又一种方案中，第一节点根据第二占用存储空间和swap分区的最大占用存储空间，确定第一存储空间信息，包括如下的c33～c34：
c33、第一节点根据第二占用存储空间和swap分区的最大占用存储空间，确定剩余总可用存储空间比例。
其中，第一节点可先获取swap分区的最大占用存储空间与第二占用存储空间的第三差值，确定第三差值和swap分区的最大占用存储空间的比值为剩余总可用存储空间比例。或者，第一节点还可以先获取已用总存储空间比例，确定100％与已用总存储空间比例的差值为剩余总可用存储空间比例。
c34、第一节点确定剩余总可用存储空间比例大于或等于第六预设比例。
此时，第一存储空间信息为剩余总可用存储空间比例大于或等于第六预设比例的信息。因为剩余总可用存储空间比例与已用总存储空间比例的和为100％，第五预设比例和第六预设比例的和为100％，因此，若剩余总可用存储空间比例小于或等于第六预设比例，则已用总存储空间比例大于第五预设比例，结合前述对已用总存储空间比例的说明，在第一内存信息包括剩余总可用存储空间比例小于或等于第六预设比例时，第一存储空间信息可指示第二占用存储空间小于或等于swap分区的最大可用存储空间的第五预设比例。
其中，第六预设比例为作业控制组的剩余总存储空间比例的最小阈值。第六预设比例可以是第一节点从配置文件中读取的。相应地，目标节点还接收用户输入的第六预设比例配置指令，该配置指令用于生成配置文件。可以理解的是，配置文件中可包括第五预设比例、第六预设比例中的一项。
c4、第一节点唤醒第二作业对应的进程。
本实施例的方法可以防止swap分区的存储空间被过度被使用。
以上对本申请涉及的方法进行了说明，下面对本申请涉及的装置进行说明。
图12为本申请实施例提供的内存控制装置的示意性框图，参见图11，本实施例的装置包括：获取模块1201和处理模块1202。
获取模块1201，用于获取作业控制组包括的各作业的第一总占用内存所述作业控制组包括所述第一节点上未处理完毕的各作业。
处理模块1202，用于在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
可选的，所述获取模块1201具体用于：调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
可选的，所述作业控制组的数量为一个，所述第一节点获取到的各作业被所述第一节点添加至所述作业控制组中。
可选的，所述处理模块1202具体用于：采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
可选的，在所述处理模块1202采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述处理模块1202还用于：
确定所述第一总占用内存小于或等于所述最大使用内存；
根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
可选的，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
可选的，所述处理模块1202还用于：读取配置文件中的所述第一预设比例。
可选的，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
可选的，所述处理模块1202还用于：读取配置文件中的所述第二预设比例。
可选的，在所述获取模块1201在获取作业控制组包括的各作业的第一总占用内存之前：所述获取模块1201还用于：获取作业控制组包括的各作业的第二总占用内存；若所述第二总占用内存大于所述最大可用内存，则所述处理模块1202还用于：将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
可选的，所述第一作业的第一实际使用内存大于所述第一作业的第一最大可用内存且所述第一作业的内存超出比例最高；所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
可选的，所述第一处理方式为挂起，在处理模块1202采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后：所述获取模块1201还用于：获取作业控制组包括的各作业的第三总占用内存；所述处理模块1202还用于：确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；以及唤醒所述第一作业对应的进程。
可选的，在所述处理模块1202采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述处理模块1202还用于：读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
可选的，所述处理模块1202具体用于：将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
可选的，在所述处理模块1202将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区之后，所述处理模块1202还用于：若所述swap分区被所述作业控制组所占用的第一占用存储空间满足大于所述swap分区的最大可用存储空间，采用第一处理方式处理所述作业控制组中的第二作业对应的进程，以使所述swap分区被所述作业控制组所占用存储空间小于或等于所述最大可用存储空间，第一处理方式为挂起或终止。
可选的，所述第二作业为所述作业控制组中占用所述swap分区的作业中的任意一个作业；或者，所述第二作业为所述作业控制组中占用的所述swap分区的存储空间最多的作业。
可选的，所述处理模块1202还用于：读取配置文件中的最大使用内存与所述swap分区的 最大可用存储空间的第一比值；根据所述第一比值和所述最大使用内存，确定所述swap分区的最大可用存储空间。
可选的，所述处理模块1202还用于：读取配置文件中的最大可用内存比例；根据所述最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。
本实施例的装置可以用于执行上述方法实施例中的技术方案，其实现原理和技术效果类似，此处不再赘述。
图13为本申请实施例提供的内存控制系统的示意性框图。参见图13，本实施例中的系统包括至少一个第一节点1301和第二节点1302；
所述第二节点1302，用于向所述第一节点1301发送作业信息；
所述第一节点1301，用于基于所述作业信息得到作业，以及将所述作业添加至作业控制组，所述作业控制组包括所述第一节点1301上未处理完毕的各作业；
所述第一节点1301，还用于获取作业控制组包括的各作业的第一总占用内存，所述作业控制组包括所述第一节点1301上未处理完毕的各作业；
在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
可选的，所述第一节点1301具体用于：调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
可选的，所述作业控制组的数量为一个，所述第一节点1301获取到的各作业被所述第一节点1301添加至所述作业控制组中。
可选的，所述第一节点1301具体用于：采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
可选的，在所述第一节点1301采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点1301还用于：
确定所述第一总占用内存小于或等于所述最大使用内存；
根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
可选的，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
可选的，所述第一节点1301还用于：读取配置文件中的所述第一预设比例。
可选的，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
可选的，所述第一节点1301还用于：读取配置文件中的所述第二预设比例。
可选的，在所述第一节点1301获取作业控制组包括的各作业的第一总占用内存之前，所述第一节点1301还用于：
获取作业控制组包括的各作业的第二总占用内存；
若所述第二总占用内存大于所述最大可用内存，则所述将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
可选的，所述第一作业的第一实际使用内存大于所述第一作业的第一最大可用内存且所 述第一作业的内存超出比例最高；
所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
可选的，所述第一处理方式为挂起，在所述第一节点1301采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，所述第一节点1301还用于：
获取作业控制组包括的各作业的第三总占用内存；
确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；
唤醒所述第一作业对应的进程。
可选的，在所述第一节点1301采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点1301还用于：
读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
可选的，所述第一节点1301具体用于：将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
可选的，所述第一节点1301还用于：
读取配置文件中的最大可用内存比例；
根据所述最大使用内存比例和所述第一节点1301的总内存，确定所述最大使用内存。
可选的，所述第一节点1301为分布式计算系统中的计算节点或云服务器，所述第二节点1302为所述分布式计算系统中的管理节点；所述作业信息包括所述作业。
可选的，所述第二节点1302为终端设备，所述第一节点1301为应用服务器；所述作业信息包括用户请求，所述作业用于执行所述用户请求。
本实施例的系统可以用于执行上述方法实施例中的技术方案，其实现原理和技术效果类似，此处不再赘述。
本申请实施例还提供了一种计算机可读介质，其上存储有计算机程序，该计算机程序被计算机执行时实现上述任一方法实施例所述的方法。
本申请实施例还提供了一种计算机程序产品，该计算机程序产品被计算机执行时实现上述任一方法实施例所述的方法。
本申请实施例还提供了一种芯片，包括处理器、存储器和通信接口，所述处理器、存储器与所述通信接口连接，其特征在于，所述处理器用于读取并执行所述存储器中存储的计算机程序，以执行上述任一方法实施例所述的方法。
本申请实施例中描述的处理器和收发器可以用各种IC工艺技术来制造，例如互补金属氧化物半导体(complementary metal oxide semiconductor,CMOS)、N型金属氧化物半导体(nMetal-oxide-semiconductor，NMOS)、P型金属氧化物半导体(positive channel metal oxide semiconductor,PMOS)、双极结型晶体管(Bipolar Junction Transistor，BJT)、双极CMOS(BiCMOS)、硅锗(SiGe)、砷化镓(GaAs)等。
应理解，本申请实施例中的处理器可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array，FPGA)或者其它可编程逻辑器件、分立门 或者晶体管逻辑器件、分立硬件组件。
可以理解，本申请实施例中的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
在上述实施例使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(digital video disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以 是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。
应理解，说明书通篇中提到的“实施例”意味着与实施例有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各个实施例未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
还应理解，在本申请中，“当…时”、“若”以及“如果”均指在某种客观情况下第一节点会做出相应的处理，并非是限定时间，且也不要求第一节点实现时一定要有判断的动作，也不意味着存在其它限定。
本申请中对于使用单数表示的元素旨在用于表示“一个或多个”，而并非表示“一个且仅一个”，除非有特别说明。本申请中，在没有特别说明的情况下，“至少一个”旨在用于表示“一个或者多个”，“多个”旨在用于表示“两个或两个以上”。
另外，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况，其中A可以是单数或者复数，B可以是单数或者复数。
字符“/”一般表示前后关联对象是一种“或”的关系。
本文中术语“……中的至少一个”或“……中的至少一种”，表示所列出的各项的全部或任意组合，例如，“A、B和C中的至少一种”，可以表示：单独存在A，单独存在B，单独存在C，同时存在A和B，同时存在B和C，同时存在A、B和C这六种情况，其中A可以是单数或者复数，B可以是单数或者复数，C可以是单数或者复数。
应理解，在本申请各实施例中，“与A相应的B”表示B与A相关联，根据A可以确定B。但还应理解，根据A确定B并不意味着仅仅根据A确定B，还可以根据A和/或其它信息确定B。

Claims (35)

  1. 一种内存控制方法，其特征在于，应用于第一节点，所述方法包括：
    获取作业控制组包括的各作业的第一总占用内存，所述作业控制组包括所述第一节点上未处理完毕的各作业；
    在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
  2. 根据权利要求1所述的方法，其特征在于，所述获取作业控制组包括的各作业的第一总占用内存，包括：
    调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
  3. 根据权利要求1或2所述的方法，其特征在于，所述作业控制组的数量为一个，所述第一节点获取到的各作业被所述第一节点添加至所述作业控制组中。
  4. 根据权利要求1～3任一项所述的方法，其特征在于，所述减少所述作业控制组包括的各作业的总占用内存，包括：
    采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
  5. 根据权利要求4所述的方法，其特征在于，在所述采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，还包括：
    确定所述第一总占用内存小于或等于所述最大使用内存；
    根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
  6. 根据权利要求5所述的方法，其特征在于，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；
    所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
  7. 根据权利要求6所述的方法，其特征在于，还包括：
    读取配置文件中的所述第一预设比例。
  8. 根据权利要求5所述的方法，其特征在于，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；
    所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
  9. 根据权利要求8所述的方法，其特征在于，还包括：
    读取配置文件中的所述第二预设比例。
  10. 根据权利要求5～9任一项所述的方法，其特征在于，在所述获取作业控制组包括的各作业的第一总占用内存之前，还包括：
    获取作业控制组包括的各作业的第二总占用内存；
    若所述第二总占用内存大于所述最大可用内存，则所述将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
  11. 根据权利要求4～10任一项所述的方法，其特征在于，所述第一作业的第一实际使用内存大于所述第一作业的第一最大可用内存且所述第一作业的内存超出比例最高；
    所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
  12. 根据权利要求5～11任一项所述的方法，其特征在于，所述第一处理方式为挂起，在所述采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，还包括：
    获取作业控制组包括的各作业的第三总占用内存；
    确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；
    唤醒所述第一作业对应的进程。
  13. 根据权利要求4～12任一项所述的方法，其特征在于，在采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，还包括：
    读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
  14. 根据权利要求1～3任一项所述的方法，其特征在于，所述减少所述作业控制组包括的各作业的总占用内存，包括：
    将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
  15. 根据权利要求1～14任一项所述的方法，其特征在于，还包括：
    读取配置文件中的最大可用内存比例；
    根据所述最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。
  16. 一种电子设备，其特征在于，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1～15中任一项所述的方法。
  17. 一种内存控制系统，其特征在于，包括第二节点和至少一个第一节点；
    所述第二节点，用于向所述第一节点发送作业信息；
    所述第一节点，用于基于所述作业信息得到作业，以及将所述作业添加至作业控制组，所述作业控制组包括所述第一节点上未处理完毕的各作业；
    所述第一节点，还用于获取作业控制组包括的各作业的第一总占用内存，所述作业控制组包括所述第一节点上未处理完毕的各作业；
    在所述第一总占用内存满足第一预设条件时，减少所述作业控制组包括的各作业的总占用内存，所述第一预设条件包括大于最大可用内存，或者，所述第一预设条件包括小于或等于所述最大可用内存、且大于所述最大可用内存的第一预设比例。
  18. 根据权利要求17所述的系统，其特征在于，所述第一节点具体用于：
    调用控制组系统监控进程，获取作业控制组包括的各作业的第一总占用内存。
  19. 根据权利要求17或18所述的系统，其特征在于，所述作业控制组的数量为一个，所述第一节点获取到的各作业被所述第一节点添加至所述作业控制组中。
  20. 根据权利要求17～19任一项所述的系统，其特征在于，所述第一节点具体用于：
    采用第一处理方式处理所述作业控制组中的第一作业对应的进程，所述第一处理方式为挂起或终止。
  21. 根据权利要求20所述的系统，其特征在于，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点还用于：
    确定所述第一总占用内存小于或等于所述最大使用内存；
    根据所述第一总占用内存和所述最大使用内存，确定第一内存信息，所述第一内存信息指示所述第一总占用内存大于所述最大可用内存的第一预设比例。
  22. 根据权利要求21所述的系统，其特征在于，所述第一内存信息包括：已用总内存比例大于所述第一预设比例；
    所述已用总内存比例为所述第一总占用内存与所述最大可用内存的比值。
  23. 根据权利要求22所述的系统，其特征在于，所述第一节点还用于：
    读取配置文件中的所述第一预设比例。
  24. 根据权利要求21所述的系统，其特征在于，所述第一内存信息包括：第一剩余总可用内存比例小于第二预设比例，所述第一预设比例和所述第二预设比例的和为100％；
    所述第一剩余总可用内存比例为第一差值与所述最大可用内存的比值，所述第一差值为所述最大可用内存与所述第一总占用内存的差值。
  25. 根据权利要求24所述的系统，其特征在于，所述第一节点还用于：
    读取配置文件中的所述第二预设比例。
  26. 根据权利要求21～25任一项所述的系统，其特征在于，在所述第一节点获取作业控制组包括的各作业的第一总占用内存之前，所述第一节点还用于：
    获取作业控制组包括的各作业的第二总占用内存；
    若所述第二总占用内存大于所述最大可用内存，则所述将所述作业控制组包括的各作业所对应的部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存为所述第一总占用内存。
  27. 根据权利要求20～26任一项所述的系统，其特征在于，所述第一作业的第一实际使用内存大于所述第一作业的第一最大可用内存且所述第一作业的内存超出比例最高；
    所述第一作业的内存超出比例为第二差值与所述第一最大使用内存的比值，所述第二差值为所述第一实际使用内存与所述第一最大可用内存的差值。
  28. 根据权利要求21～27任一项所述的系统，其特征在于，所述第一处理方式为挂起，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之后，所述第一节点还用于：
    获取作业控制组包括的各作业的第三总占用内存；
    确定所述第三总占用内存小于所述最大可用内存的第三预设比例，所述第三预设比例小于或等于所述第一预设比例；
    唤醒所述第一作业对应的进程。
  29. 根据权利要求20～28任一项所述的系统，其特征在于，在所述第一节点采用第一处理方式处理所述作业控制组中的第一作业对应的进程之前，所述第一节点还用于：
    读取配置文件中的处理方式指示信息，所述处理方式指示信息指示所述第一处理方式。
  30. 根据权利要求17～19任一项所述的系统，其特征在于，所述第一节点具体用于：
    将所述作业控制组包括的至少一个作业所对应的至少部分内存数据迁移至swap分区，以使作业控制组包括的各作业的总占用内存小于或等于所述最大可用内存。
  31. 根据权利要求17～30任一项所述的系统，其特征在于，所述第一节点还用于：
    读取配置文件中的最大可用内存比例；
    根据所述最大使用内存比例和所述第一节点的总内存，确定所述最大使用内存。
  32. 根据权利要求17～31任一项所述的系统，其特征在于，所述第一节点为分布式计算系 统中的计算节点或云服务器，所述第二节点为所述分布式计算系统中的管理节点；
    所述作业信息包括所述作业。
  33. 根据权利要求17～31任一项所述的系统，其特征在于，所述第二节点为终端设备，所述第一节点为应用服务器；
    所述作业信息包括用户请求，所述作业用于执行所述用户请求。
  34. 一种存储介质，其特征在于，所述存储介质包括计算机程序，所述计算机程序用于实现如权利要求1～15任一项所述的方法。
  35. 一种芯片，包括处理器、存储器和通信接口，所述处理器、存储器与所述通信接口连接，其特征在于，所述处理器用于读取并执行所述存储器中存储的计算机程序，以执行前述权利要求1～15任一项所述的方法。
PCT/CN2021/117914 2020-09-21 2021-09-13 内存控制方法和装置 WO2022057754A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010996974.5 2020-09-21
CN202010996974.5A CN114253457B (zh) 2020-09-21 2020-09-21 内存控制方法和装置

Publications (1)

Publication Number Publication Date
WO2022057754A1 true WO2022057754A1 (zh) 2022-03-24

Family

ID=80776479

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/117914 WO2022057754A1 (zh) 2020-09-21 2021-09-13 内存控制方法和装置

Country Status (2)

Country Link
CN (1) CN114253457B (zh)
WO (1) WO2022057754A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101833512A (zh) * 2010-04-22 2010-09-15 中兴通讯股份有限公司 一种内存回收方法及其装置
CN106407010A (zh) * 2016-09-06 2017-02-15 北京珠穆朗玛移动通信有限公司 一种内存管理方法及移动终端
US9946577B1 (en) * 2017-08-14 2018-04-17 10X Genomics, Inc. Systems and methods for distributed resource management
CN109379246A (zh) * 2018-09-21 2019-02-22 锐捷网络股份有限公司 一种内存的检测方法及装置
CN110221921A (zh) * 2019-06-13 2019-09-10 深圳Tcl新技术有限公司 内存管理方法、终端及计算机可读存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521029A (zh) * 2011-12-02 2012-06-27 曙光信息产业(北京)有限公司 一种基于内存独占的作业调度方法
CN107665146B (zh) * 2016-07-29 2020-07-07 华为技术有限公司 内存管理装置和方法
CN107066316A (zh) * 2017-04-25 2017-08-18 华中科技大学 分布式数据处理系统中缓解内存压力的调度方法和系统
CN109992471A (zh) * 2018-01-02 2019-07-09 中国移动通信有限公司研究院 一种内存监控的方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101833512A (zh) * 2010-04-22 2010-09-15 中兴通讯股份有限公司 一种内存回收方法及其装置
CN106407010A (zh) * 2016-09-06 2017-02-15 北京珠穆朗玛移动通信有限公司 一种内存管理方法及移动终端
US9946577B1 (en) * 2017-08-14 2018-04-17 10X Genomics, Inc. Systems and methods for distributed resource management
CN109379246A (zh) * 2018-09-21 2019-02-22 锐捷网络股份有限公司 一种内存的检测方法及装置
CN110221921A (zh) * 2019-06-13 2019-09-10 深圳Tcl新技术有限公司 内存管理方法、终端及计算机可读存储介质

Also Published As

Publication number Publication date
CN114253457B (zh) 2024-06-25
CN114253457A (zh) 2022-03-29

Similar Documents

Publication Publication Date Title
CN109375872B (zh) 数据访问请求的处理方法、装置和设备及存储介质
EP3358463B1 (en) Method, device and system for implementing hardware acceleration processing
US8413158B2 (en) Processor thread load balancing manager
US8478926B1 (en) Co-processing acceleration method, apparatus, and system
US7565653B2 (en) Methods and apparatus for processor task migration in a multi-processor system
WO2019223596A1 (zh) 事件处理方法、装置、设备及存储介质
CN113918101B (zh) 一种写数据高速缓存的方法、系统、设备和存储介质
CN109726005B (zh) 用于管理资源的方法、服务器系统和计算机可读介质
CN108900626B (zh) 一种云环境下数据存储方法、装置及系统
WO2024016596A1 (zh) 容器集群调度的方法、装置、设备及存储介质
WO2012177359A2 (en) Native cloud computing via network segmentation
JP2013513174A (ja) 仮想マシンのストレージスペースおよび物理ホストを管理するための方法およびシステム
WO2015101091A1 (zh) 一种分布式资源调度方法及装置
US9632958B2 (en) System for migrating stash transactions
CN107562685B (zh) 一种基于延时补偿的多核处理器核心间数据交互的方法
WO2024119988A1 (zh) 多cpu环境下的进程调度方法、装置、电子设备和介质
WO2024037239A1 (zh) 一种加速器调度方法及相关装置
US20100269119A1 (en) Event-based dynamic resource provisioning
US20200201691A1 (en) Enhanced message control banks
WO2022057754A1 (zh) 内存控制方法和装置
CN115543222B (zh) 一种存储优化方法、系统、设备及可读存储介质
CN116048756A (zh) 一种队列调度方法、装置及相关设备
US20130247065A1 (en) Apparatus and method for executing multi-operating systems
WO2024001851A1 (zh) 一种资源调度方法、装置及系统
CN117908772B (zh) 多mb的数据处理方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21868578

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21868578

Country of ref document: EP

Kind code of ref document: A1