CN102945214B - Based on the method for IO distribution optimization time delay interrupt processing task - Google Patents
Based on the method for IO distribution optimization time delay interrupt processing task Download PDFInfo
- Publication number
- CN102945214B CN102945214B CN201210401747.9A CN201210401747A CN102945214B CN 102945214 B CN102945214 B CN 102945214B CN 201210401747 A CN201210401747 A CN 201210401747A CN 102945214 B CN102945214 B CN 102945214B
- Authority
- CN
- China
- Prior art keywords
- request
- memory device
- distribution
- interrupt
- processing task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
Provide the method based on IO distribution optimization time delay interrupt processing task.Computer system comprises the first processing unit and the second processing unit, and described computer system is coupled with memory device; Described method comprises: receive the first I/O request for accessing described memory device; Record the start time of described first I/O request; In response to receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, record the end time of described first I/O request; The very first time calculating the end time of described first I/O request and the start time of described first I/O request is poor; Calculate the distribution of described very first time difference in a predetermined time period; And based on the distribution that the described very first time differs from, optimize the execution of the interrupt processing task be associated with the I/O request of described memory device.
Description
Technical field
The present invention relates to solid storage device (SolidStorageDevice, SSD), more specifically, the present invention relates to the interruption that memory device sends to main frame.
Background technology
Similar with mechanical type hard disk, solid storage device (SSD) is also Large Copacity, non-volatile memory device for computer system.Solid storage device is generally using flash memory (Flash) as storage medium.High performance solid storage device is used to high-performance computer.High-performance computer has multiple central processing unit (CPU, CentrolProcessUnit) usually, and each CPU can comprise multiple CPU core.
In modern operating system, almost each interrupt processing operation has been divided into two parts, i.e. interrupt service routine (ISR) and the latter half handling procedure (BottomHalfHandler); The first half (TopHalf) of interrupt processing in other words and the latter half (BottomHalf); In some scenarios, first order interrupt handling routine (FLIH, First-LevelInterruptHandler) and second level interrupt handling routine (SLIH, Second-LevelInterruptHandlers) is also referred to as.Difference is between the two: in the first half of interrupt processing, interruption is forbidden, speaking by the book is have at least an interruption (Current interrupt) (current C PU) on a CPU to be forbidden, and in some cases, whole interruptions of possible current C PU are all prohibited; And in the latter half of interrupt processing, the interruption in system is all allow, new interruption can seize the latter half handling procedure at any time.By interrupt processing is divided into two parts, and allow the first half process function only do the fastest minimum work--such as copy data from device buffer zone to internal memory, and the work of follow-up more time-consuming is left for the latter half handling procedure go perform, effectively reduce the time of interrupt choking effect, thus improve the interrupt processing efficiency of system.
In U.S. Patent application US2011/0087814A1, provide and strengthen IO(InputOutput in multicomputer system by interrupting compatibility mechanism) scheme of performance.Referring to Fig. 1, main frame 100 comprises CPU101,102,103 and 104, and application program 111,112,113 and 114 operates on respective CPU respectively.High-speed cache 121,122,123 and 124 respectively same CPU101,102,103 and 104 is coupled, for storing the instruction and data being respectively used to application program 111,112,113 and 114.And the storer 131,132,133 and 134 with larger capacity is coupled to high-speed cache 121,122,123 and 124 respectively.Host bus adaptor (HBA, HostBusAdapter) 150 pass through IOAPIC(AdvancedProgrammableInterruptController, Advanced Programmable Interrupt Controllers APICs) hub 140 is coupled to main frame 100, and be configured to same Storage are network (SAN, StorageArea, Network) 160 communications.SAN160 is connected to the computer memory device 170 of multiple such as hard disk.Interrupt compatibility mechanism to mean, when I/OAPIC hub 140 receives the interruption from HBA150, determine by which in multiple CPU of main frame 100 to process interruption.Such as, record sends the CPU identifier of I/O request, and when receiving the interruption corresponding to this I/O request, the CPU identifier based on record is selected to process the CPU interrupted.
Some computing machines have NUMA(Non-UniformMemoryAccess) or NUIO(Non-UniformInputOutput) structure.Provide in U.S. Patent application US2005/0060460A1 in NUMA system as the I/O device in multiple node assigns the scheme of interrupting.Comprise by interconnection network node 201,211,221 and 231 coupled to each other referring to Fig. 2, NUMA system 200.Each node 201,211,221 and 231 can comprise processor and storer.The storer of given node is local relative to the processor of this node, and the storer of other nodes is long-range relative to this processor.System 200 comprises I/O device 202,212 and 242, and for the Interrupt Service Routine (ISR, InterruptServiceRoutine) 203,213 and 223 of I/O device 202,212 and 242.System 200 also comprises interrupt distribution software 222, for Interrupt Service Routine 203,213 and 223 is distributed to I/O device 202,212 and 242.The distribution of Interrupt Service Routine can based on following factors: the node that equipment connects, the node that the ISR for equipment is positioned at, and the processor of node.?
www.acpi.info/DOWNLOADS/ACPIspec50.pdfprovide each NUMA domain information obtaining NUMA system obtainable " AdvancedConfigurationandPowerInterfaceSpecification " (on Dec 6th, 2011).
But the IO processing power of high-performance solid storage device significantly improves, the quantity of I/O request and the increase of data volume, cause burden to the process of CPU.Thus, the process to interrupt request is needed to be optimized.
Summary of the invention
According to a first aspect of the invention, provide a kind of method of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processing unit, and described memory device is by PCIE bus coupling in described multiple processing unit, and described method comprises:
Obtain the identifier of described memory device;
According to the identifier of described memory device, determine the first processing unit that described memory device is coupled;
The first half of the interrupt processing task be associated with described memory device is set to perform on described first processing unit;
The latter half of described interrupt processing task is set to perform on the second processing unit;
Wherein said first processing unit and described second processing unit share cache memory.
According to a second aspect of the invention, provide a kind of method of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processor slot, each processor slot is furnished with polycaryon processor, described polycaryon processor has multiple processor core, described memory device is by PCIE bus coupling in described multiple processor slot, and described method comprises:
Obtain the identifier of described memory device;
According to the identifier of described memory device, determine the first processor slot that described memory device is coupled;
The first half of the interrupt processing task be associated with described memory device is set to perform on the first processor core being arranged in the polycaryon processor on described first processor slot;
The latter half of described interrupt processing task is set to perform on the second processor core being arranged in the polycaryon processor on described first processor slot;
Wherein said first processor core and described second processor core share cache memory.
The method provided according to a second aspect of the invention, comprises further, shows the coupled relation of described memory device and described first processor slot and/or described first processor core.
According to a first aspect of the invention or the method that provides of second aspect, comprise further, show the coupled relation of described memory device and described first processing unit.
Method provided according to a second aspect of the invention, also comprise and the application program of the described memory device of access be set to perform on the 3rd processor core being arranged in the polycaryon processor on described first processor slot, described first processor core, described second processor core and described 3rd processor core share cache memory.
According to a first aspect of the invention or the method that provides of second aspect, also comprise and obtain and show passage (Lane) quantity of described PCIE bus.
According to a first aspect of the invention or the method that provides of second aspect, also comprise obtain and show described memory device have the first time delay IO access quantity, and the quantity with the IO access of the second time delay of described memory device, described first time delay is different from described second time delay.
According to a first aspect of the invention or the method that provides of second aspect, also comprise obtain and show described memory device have first time delay scope IO access quantity, and described memory device have second time delay scope IO access quantity, described first time delay scope be different from described second time delay scope.
The method provided according to a first aspect of the invention, is set to perform on the second processor by the application program of the described memory device of access.
The method provided according to a first aspect of the invention, the application program of the described memory device of access be set to perform on the 3rd processing unit, the 3rd processing unit and described first processing unit and the second processing unit share high-speed cache.
According to a first aspect of the invention or the method that provides of second aspect, the first half of described interrupt processing task is embodied as work queue or kernel thread.
According to a first aspect of the invention or the method that provides of second aspect, the first half of described interrupt processing task is used for save register state and removes interruption, according to preserved buffer status, the latter half of described interrupt processing task determines that the first I/O request process completes.
According to a third aspect of the invention we, provide a kind of method of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processing unit and multiple PCIE slot, described memory device is coupled to one in described multiple processing unit by the PCIE slot in described multiple PCIE slot, described method comprises:
Identify the first processing unit that a described PCIE slot is coupled;
Determine that described memory device is coupled to described first processing unit;
The first half of the interrupt processing task be associated with described memory device is set to perform on described first processing unit;
The latter half of described interrupt processing task is set to perform on the second processing unit;
Wherein said first processing unit and described second processing unit share cache memory.
According to a forth aspect of the invention, provide a kind of method of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processor slot and multiple PCIE slot, each processor slot is furnished with polycaryon processor, described polycaryon processor has multiple processor core, described memory device is coupled to one in described multiple processor slot by the PCIE slot in described multiple PCIE slot, described method comprises:
Identify the first processor slot that a described PCIE slot is coupled;
Determine that described memory device is coupled to described first processor slot;
The first half of the interrupt processing task be associated with described memory device is set to perform on the first processor core being arranged in the polycaryon processor on described first processor slot;
The latter half of described interrupt processing task is set to perform on the second processor core being arranged in the polycaryon processor on described first processor slot;
Wherein, described first processor core and described second processor core share cache memory.
According to a first aspect of the invention or the method for the third aspect, also comprise and the application program of the described memory device of access be set to perform on described first processing unit, described second processing unit or the 3rd processing unit, described first processing unit, described second processing unit and described 3rd processing unit share described cache memory.
According to a second aspect of the invention or the method for fourth aspect, also comprise and the application program of the described memory device of access be set to perform on the 3rd processor core being arranged in the polycaryon processor on described first processor slot, described first processor core, described second processor core and described 3rd processor core share cache memory.
According to a first aspect of the invention, the method for second aspect, the third aspect or fourth aspect, comprise further, the data of described computer system request are transferred to described cache memory with dma mode by described memory device.
According to a fifth aspect of the invention, provide a kind of device of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processing unit, and described memory device is by PCIE bus coupling in described multiple processing unit, and described device comprises:
For obtaining the module of the identifier of described memory device;
For the identifier according to described memory device, determine the module of the first processing unit that described memory device is coupled;
For the module that the first half of the interrupt processing task be associated with described memory device is set to perform on described first processing unit;
For the module that the latter half of described interrupt processing task is set to perform on the second processing unit;
Wherein said first processing unit and described second processing unit share cache memory.
According to a sixth aspect of the invention, provide a kind of device of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processor slot, each processor slot is furnished with polycaryon processor, described polycaryon processor has multiple processor core, described memory device is by PCIE bus coupling in described multiple processor slot, and described device comprises:
For obtaining the module of the identifier of described memory device;
For the identifier according to described memory device, determine the module of the first processor slot that described memory device is coupled;
For the first half of the interrupt processing task be associated with described memory device being set to be arranged in the module that the first processor core of the polycaryon processor on described first processor slot performs;
For the latter half of described interrupt processing task being set to be arranged in the module that the second processor core of the polycaryon processor on described first processor slot performs;
Wherein said first processor core and described second processor core share cache memory.
According to a seventh aspect of the invention, provide a kind of device of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processing unit and multiple PCIE slot, described memory device is coupled to one in described multiple processing unit by the PCIE slot in described multiple PCIE slot, described device comprises:
For identifying the module of the first processing unit that a described PCIE slot is coupled;
For determining that described memory device is coupled to the module of described first processing unit;
For the module that the first half of the interrupt processing task be associated with described memory device is set to perform on described first processing unit;
For the module that the latter half of described interrupt processing task is set to perform on the second processing unit;
Wherein said first processing unit and described second processing unit share cache memory.
According to an eighth aspect of the invention, provide a kind of device of the interrupt request for the treatment of the memory device in computer system, described computer system comprises multiple processor slot and multiple PCIE slot, each processor slot is furnished with polycaryon processor, described polycaryon processor has multiple processor core, described memory device is coupled to one in described multiple processor slot by the PCIE slot in described multiple PCIE slot, described device comprises:
For identifying the module of the first processor slot that a described PCIE slot is coupled;
For determining that described memory device is coupled to the module of described first processor slot;
For the first half of the interrupt processing task be associated with described memory device being set to be arranged in the module that the first processor core of the polycaryon processor on described first processor slot performs;
For the latter half of described interrupt processing task being set to be arranged in the module that the second processor core of the polycaryon processor on described first processor slot performs;
Wherein, described first processor core and described second processor core share cache memory.
According to a ninth aspect of the invention, provide a kind of method of the interrupt request for the treatment of the memory device in computer system, described computer system comprises the first processing unit and the second processing unit, and described computer system is coupled with memory device; Described method comprises:
The interrupt processing task be associated with the I/O request of described memory device is set to perform on described first processing unit;
Receive the first I/O request for accessing described memory device;
Record the start time of described first I/O request;
In response to receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, perform described interrupt processing task, to record the end time of described first I/O request;
The very first time calculating the end time of described first I/O request and the start time of described first I/O request is poor;
Calculate the distribution of described very first time difference in a predetermined time period; And
Based on the distribution of described very first time difference, interrupt processing task is set to perform on described second processing unit.
Method according to a ninth aspect of the invention, also comprises:
Receive the second I/O request for accessing described memory device;
Record the start time of described second I/O request;
In response to receiving the interrupt request completed from the described second I/O request process of instruction of described memory device, perform described interrupt processing task, to record the end time of described second I/O request;
Calculate second mistiming of the end time of described second I/O request and the start time of described second I/O request;
Calculate the distribution of described second mistiming in a predetermined time period; And
Based on distribution and the distribution of described second mistiming of described very first time difference, interrupt processing task is set to perform on one of described first processing unit or described second processing unit.
Method according to a ninth aspect of the invention, also comprises and sends the distribution of described very first time difference to application program.
Method according to a ninth aspect of the invention, also comprises and sends the distribution of described very first time difference and/or the distribution of described second mistiming to application program.
According to the tenth aspect of the invention, provide a kind of device of the interrupt request for the treatment of the memory device in computer system, described computer system comprises the first processing unit and the second processing unit, and described computer system is coupled with memory device; Described device comprises:
For the module that the interrupt processing task be associated with the I/O request of described memory device is set to perform on described first processing unit;
For receiving the module of the first I/O request for accessing described memory device;
For recording the module of the start time of described first I/O request;
For in response to receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, perform described interrupt processing task, to record the module of the end time of described first I/O request;
For the module of the very first time difference of the start time of end time and described first I/O request of calculating described first I/O request;
For calculating the module of the distribution of described very first time difference in a predetermined time period; And
For the distribution based on described very first time difference, interrupt processing task is set to the module performed on described second processing unit.
Device according to the tenth aspect of the invention, also comprises:
For receiving the module of the second I/O request for accessing described memory device;
For recording the module of the start time of described second I/O request;
For in response to receiving the interrupt request completed from the described second I/O request process of instruction of described memory device, perform described interrupt processing task, to record the module of the end time of described second I/O request;
For the module of second mistiming of the start time of end time and described second I/O request of calculating described second I/O request;
For calculating the module of the distribution of described second mistiming in a predetermined time period; And
For based on the distribution of described very first time difference and the distribution of described second mistiming, interrupt processing task is set to the module of execution on one of described first processing unit or described second processing unit.
Accompanying drawing explanation
When reading together with accompanying drawing, by reference to below to the detailed description of the embodiment of illustrating property, will understand the present invention and preferred using forestland and its further object and advantage best, wherein accompanying drawing comprises:
Fig. 1 is the structured flowchart of the storage system according to prior art;
Fig. 2 is the structured flowchart of the NUMA system according to prior art;
Fig. 3 A is the structured flowchart of the NUIO system of the interruption optimization method implementing the embodiment of the present invention;
Fig. 3 B is the process flow diagram interrupting optimization method according to an embodiment of the invention;
Fig. 4 A is the structured flowchart of the interruption optimization method NUIO system implementing another embodiment of the present invention;
Fig. 4 B is the process flow diagram of interruption optimization method according to another embodiment of the present invention; And
Fig. 5 is the process flow diagram of interruption optimization method according to still another embodiment of the invention.
Embodiment
Fig. 3 A is the structured flowchart of the NUIO system of the interruption optimization method implementing the embodiment of the present invention.The computer system of embodiment as shown in Figure 3A has NUIO structure, because memory device 321 is coupled to CPU301, and memory device 322 is coupled to CPU302, the IO between CPU301 from memory device 321 is made to be different with the IO between CPU322 and memory device 321.CPU301 and CPU302 is all coupled to high-speed cache 311, and CPU301 and CPU302 shares high-speed cache 311.
CPU301 by such as PCIE bus/interface coupling to memory device 321.Illustrate that a memory device 321 is coupled to CPU301 at Fig. 3 A, but in one embodiment, multiple memory device can be coupled to CPU301.Memory device 301 can be solid storage device (SSD), also can be other memory devices with PCIE interface.Can also by such as DRAM(DynamicRandomAccessMemory) storer be coupled to CPU301 and 302 respectively, and thus form the computer system of NUMA structure.Also by such as PCIE bus/interface coupling between CPU302 and memory device 322.CPU301 and 302 can have this locality and unshared high-speed cache (not shown) separately, is such as integrated in the L1 high-speed cache of cpu chip inside.And high-speed cache 311 can be the L2 high-speed cache being arranged on mainboard or being integrated on chipset.
Because memory device 321 is coupled to CPU301, the interrupt processing task be thus associated by same memory device 321 arranges or is bundled in and CPU301 performs is favourable.That is, when memory device 321 produces interrupt request, interrupt request is forwarded to CPU301, by CPU301 by performing this interrupt request of ISR process.ISR is configured to the first half processing the interrupt processing task be associated with memory device 321.Such as, in ISR, to inquire about and the PCI/PCIE equipment preserved corresponding to memory device 321 is deposited buffer status and removes interruption.Similarly, the interrupt processing task be associated by same memory device 322 arranges or is bundled on CPU302 and performs.
The process flow diagram interrupting optimization method according to an embodiment of the invention referring to Fig. 3 B, Fig. 3 B.In step 360, obtain the identifier of memory device 321.For PCI/PCIE equipment, obtain unique identifier describing memory device 321 by pci configuration space.Based on the identifier of memory device 321, in step 370, determine that memory device is coupled to the CPU301 among CPU301 and 302.In (SuSE) Linux OS, obtain the APICID corresponding to CPU301 by "/proc/cpuinfo ", then access APIC table and obtain the CPU301 belonging to identical NUMA territory with memory device 321.In (SuSE) Linux OS, can also obtain by "/sys/devices/pci<pci-bus>/LEss the T.LTssT.LTpci-dev>/numa_node ", "/sys/devices/pci<pci-bus>/LEss T.LTssT.LTpci-dev>/local_cpus " or "/sys/devices/pci<pci-bus>/LEss T.LTssT.LTpci-dev>/local_cpulist " of user's space the CPU301 belonging to identical NUMA territory with memory device 321.Also in the driver of the memory device 321 of Linux, the CPU301 belonging to identical NUMA territory with memory device 321 can be obtained by " dev-numa_node ".In Windows operating system, also may have access to APIC table, and obtain the CPU301 belonging to identical NUMA territory with memory device 321.By checking memory device 301 and the circuit board at memory device 321 place, the coupled relation of CPU301 and memory device 321 can also be determined.Preferably, on the slot of coupled storage equipment 321, mark at circuit board the CPU301 that this slot is coupled, and by configuration file, command line interface or graphic user interface, memory device 321 and the incidence relation of CPU301 are supplied to driver.
In step 380, the first half of the interrupt processing task be associated with memory device 321 is set to perform on CPU301.The ISR being used in memory device 321 performs on CPU301.In one embodiment, when memory device 321 produces interrupt request, interrupt request is forwarded to CPU301.In ISR, to inquire about and the PCI/PCIE equipment preserved corresponding to memory device 321 is deposited buffer status and removes interruption.Alternatively, in ISR, also data are copied to DRAM storer and/or high-speed cache 311 from the buffer zone of memory device 321.In step 390, the latter half of the interrupt processing task be associated with memory device 321 is set to perform on CPU302.Because CPU301 and 302 shares high-speed cache 311, the data handled by the first half of interrupt processing task are made to be interrupted the latter half access of Processing tasks by high-speed cache 311, thus the swapping data decreased at DRAM storer and high-speed cache 311 and the expense that produces.Further, the first half of handoff Processing tasks on CPU301 and the latter half is decreased and the expense produced.Especially, when memory device 321 frequently produces interrupt request, the first half of interrupt processing task will frequently perform, and the latter half of interrupt processing task is arranged on be different from CPU301 CPU302 on perform, also reduce the burden of CPU301, and the latter half reducing interrupt processing task is by the probability frequently interrupted.In one embodiment, the latter half of interrupt processing task is embodied as the work queue (Wordqueue) that (SuSE) Linux OS provides.In one embodiment, the latter half of interrupt processing task is embodied as kernel thread, the operating system of such as Linux or Windows etc. all supports kernel thread.For memory device 322, also by step 360-390, be arranged on CPU302 with the first half of being interrupted Processing tasks and perform, and the latter half of being interrupted Processing tasks is arranged on execution on CPU301.The latter half of interrupt processing task for memory device 322 can also be arranged on be different from CPU301 and 302 other CPU on perform.
In a further embodiment, also the coupled relation of memory device 321 with CPU301 preserved and be supplied to application program, making user obtain the coupled relation of memory device 321 and CPU301 by application program.And, still further in embodiment, according to the coupled relation of memory device 321 with CPU301, the application program of accessing storage device 321 is set to perform on CPU301 or 302.
Fig. 4 A is the structured flowchart of the NUIO system of the interruption optimization method implementing the embodiment of the present invention.The computer system of embodiment as shown in Figure 4 A has NUIO structure, because memory device 431 is coupled to CPU slot 405, and memory device 441 is coupled to CPU slot 415, the IO between CPU slot 405 from memory device 431 is made to be different with the IO between CPU slot 415 and memory device 441.CPU400 is arranged in CPU slot 405, and CPU410 is arranged in CPU slot 415.CPU400,410 can be multi-core CPU, and CPU400 comprises CPU core 401,402, and CPU410 comprises CPU core 411 and 412.In CPU400, be furnished with the high-speed cache 403 shared by CPU core 401,402, and in CPU410, be furnished with the high-speed cache 413 shared by CPU core 411,412.CPU core 401,402,411 and 412 can be configured with exclusive high-speed cache (not shown) respectively.
By such as QPI(QuickPathInterconnect between CPU400 and CPU410) or high speed FSB(Front-SideBus) connect 450 and be coupled.PCIE slot 435,445 is coupled to CPU slot 405,415 respectively, and and then is coupled to CPU400,410 respectively.PCIE slot 435,445 is coupled with respectively memory device 431,441.Memory device 431,441 can be solid storage device (SSD), also can be other memory devices with PCIE interface.Also by such as DRAM(DynamicRandomAccessMemory) primary memory 421,422 be coupled to CPU400 and 410 respectively, and thus form the computer system of NUMA structure.Further, one of ordinary skill in the art will recognize, CPU400 can access the primary memory 421 and memory device 431 that are coupled to CPU slot 405, and also can access the primary memory 422 and memory device 441 that are coupled to CPU slot 415.Similar, CPU410 can access the primary memory 422 and memory device 441 that are coupled to CPU slot 415, and also can access the primary memory 421 and memory device 431 that are coupled to CPU slot 405.
Because memory device 431 is coupled to CPU400, the interrupt processing task be thus associated by same memory device 431 arranges or is bundled in and CPU400 performs is favourable.That is, when memory device 431 produces interrupt request, interrupt request is forwarded to CPU400, by CPU400 by performing this interrupt request of ISR process.ISR is configured to the first half processing the interrupt processing task be associated with memory device 431.Such as, in ISR, to inquire about and the PCI/PCIE equipment preserved corresponding to memory device 431 is deposited buffer status and removes interruption.Similarly, the interrupt processing task be associated by same memory device 441 arranges or is bundled in and CPU410 performs is favourable.
The process flow diagram interrupting optimization method according to an embodiment of the invention referring to Fig. 4 B, Fig. 4 B.In step 460, obtain the identifier of memory device 431.For PCI/PCIE equipment, obtain unique identifier describing memory device 431 by pci configuration space.In step 470, determine that memory device 431 is coupled to the CPU slot 405 among CPU slot 405 and CPU slot 415, and and then determine that memory device 431 is coupled to CPU400.Show to obtain CPU400 and the CPU core 401,402 belonging to identical NUMA territory with memory device 431 by the APIC of access based on ACPI specification (AdvancedConfigurationandPowerInterfaceSpecification).Also by checking PCIE slot 435 and the circuit board at CPU slot 405 place, the coupled relation between CPU slot 405 and PCIE slot 435 can be determined.Preferably, at circuit board for the PCIE slot 435 of coupled storage equipment 431 marking the CPU slot 405 that this slot is coupled, or be coupled to CPU400 and/or the CPU core 401,402 of CPU slot 405, and by configuration file, command line interface or graphic user interface memory device 431 and the incidence relation of CPU400 and/or CPU core 401,402 be supplied to the application program of 431 of driver and/or accessing storage device.Similarly, at circuit board for the PCIE slot 445 of coupled storage equipment 441 marking the CPU slot 415 that this slot is coupled, or be coupled to CPU410 and/or the CPU core 411,412 of CPU slot 415.And memory device 441 and the incidence relation of CPU410 and/or CPU core 411,412 are supplied to the application program of 441 of driver and/or accessing storage device.
In step 480, the first half of the interrupt processing task be associated is set to perform on the CPU core 401 of CPU400 with memory device 431.The ISR being used in memory device 431 performs on CPU core 401.In one embodiment, when memory device 431 produces interrupt request, interrupt request is forwarded to CPU core 401.In ISR, to inquire about and the PCI/PCIE equipment preserved corresponding to memory device 431 is deposited buffer status and removes interruption.Alternatively, in ISR, also data are copied to primary memory 421 and/or high-speed cache 403 from the buffer zone of memory device 431.
In step 490, the latter half of the interrupt processing task be associated is set to perform on the CPU core 402 of CPU400 with memory device 431.In the latter half of interrupt processing task, determine which I/O request completes according to the buffer status that the first half of interrupt processing task stores, complete corresponding IO and upgrade statistical information.Because CPU core 401,402 shares high-speed cache 403, the data handled by the first half of interrupt processing task are made to be interrupted the latter half access of Processing tasks by high-speed cache 403, thus the swapping data decreased at primary memory 421 and high-speed cache 403 and the expense produced, decrease the swapping data at CPU400 and CPU410 and the expense produced.Further, the first half of handoff Processing tasks on CPU core 401 and the latter half is decreased and the expense produced.
For memory device 441, also by step 460-490, the CPU core 411 being arranged on CPU410 with the first half of being interrupted Processing tasks performs, and the latter half of being interrupted Processing tasks be arranged on CPU410 CPU core 412 on execution.One of ordinary skill in the art will recognize, when the load of CPU410 is higher, also the latter half of the interrupt processing task for memory device 441 can be arranged on CPU400 or other CPU and perform.
In still another embodiment, memory device 431, by the DMA transmission on ground for the purpose of high-speed cache 403, writes data in high-speed cache 403, and sends the interrupt request that the request of instruction corresponding I/O completes.CPU core 401 performs the first half of interrupt processing task, inquires about and the PCI/PCIE equipment preserved corresponding to memory device 431 is deposited buffer status and removes interruption.And CPU core 402 performs the latter half of interrupt processing task, determine which I/O request completes according to the buffer status that the first half of interrupt processing task stores, utilize the data in high-speed cache 403 to complete corresponding I/O operation.The application program sending I/O request also can also be set to perform on CPU core 402, or be set to perform on other CPU core (not shown) sharing high-speed cache 403 with CPU core 401,402, transmit from memory device 431 the data of coming by I/O request directly to use in high-speed cache 403.
One of ordinary skill in the art also will recognize, also the ISR being used for memory device 431 can be performed on CPU core 402.And the latter half of the interrupt processing task being used for memory device 431 is arranged on execution on CPU core 401.And, still further in embodiment, according to the coupled relation of memory device 431 with CPU400, the application program of accessing storage device 431 is set to perform on CPU core 401 or 402.
In a further embodiment, also the coupled relation of memory device 431 with CPU slot 405 and/or CPU400 and/or CPU core 401,402 preserved and be supplied to application program, making user obtain the coupled relation of memory device 431 and CPU slot 405 and/or CPU400 and/or CPU core 401,402 by application program.And similarly, the coupled relation of memory device 441 with CPU slot 415 and/or CPU410 and/or CPU core 411,412 is preserved and be supplied to application program.
Fig. 5 is the process flow diagram of optimization method according to still another embodiment of the invention.Based on the distribution of the time delay of I/O request, adjust CPU or the CPU core for performing interrupt processing task.The optimization method of the embodiment according to Fig. 5 is described below in conjunction with the NUIO system of Fig. 4 A.One of ordinary skill in the art will recognize, also can composition graphs 3A NUIO system and implement the optimization method of the embodiment according to Fig. 5.
In step 501, the interrupt processing task be associated with memory device 431 is set to perform on CPU core 401.This interrupt processing task is for the treatment of the I/O request from memory device 431.I/O request is initiated by application program.In step 520, the driver for memory device 431 receives the I/O request for accessing storage device 431.In step 530, record the start time of this I/O request.In step 540, when receiving the interrupt request that completes from this I/O request process of instruction of memory device 431 and after processing this interrupt request, recording the end time of this I/O request.In step 550, calculate the mistiming of end time and start time, to obtain the time delay of this I/O request.In this way, for the multiple or each I/O request be associated with memory device 431, the time delay of its I/O request can all be obtained.In step 560, calculate the distribution of the time delay of multiple I/O request of in a predetermined time period (such as, 0.5 second or 1 second).Such as, in one embodiment, as shown in the table, time delay accounts for 64.3% of the I/O request total amount in the schedule time in the I/O request of 90-400 microsecond, and time delay accounts for 33.4% of the I/O request total amount in the schedule time in the I/O request of 400-1.5 millisecond.And the distribution of this time delay can be supplied to application program.
In step 570, based on the distribution of the time delay of multiple I/O request, determine whether the execution needing to optimize interrupt processing task.In one embodiment, if the distribution of the time delay of multiple I/O request is obviously inferior to expection or predetermined distribution, then in step 580, the interrupt processing task be associated with memory device 431 is set to above perform at the CPU core being different from CPU core 401 (such as CPU core 402,411 or 412).In one embodiment, the time delay distribution before if the distribution of the time delay of multiple I/O request is inferior to, then in step 580, the interrupt processing task be associated with memory device 431 is set to above perform at the CPU core being different from CPU core 401 (such as CPU core 402,411 or 412).If judge not need in step 570 execution optimizing interrupt processing task, then get back to step 520, to monitor and to calculate time delay and the distribution thereof of other I/O request.
In one embodiment, IO delay measurements process is provided.The interrupt processing task be associated with memory device 431 is separately positioned on CPU core 401,402,411 or 412, measures the distribution of the time delay of respective I/O request.And the time delay parameter interrupt processing task be associated with memory device 431 being arranged on I/O request is preferably on CPU core.In another embodiment, the time delay or its distribution of measuring the I/O request obtaining each CPU core corresponding to outage Processing tasks are supplied to application program, and then show user.Make user can select to perform the CPU core (preferably, CPU core 402) of the interrupt processing task be associated with memory device 431 according to the needs of its application.
Interrupt processing task involved in step 510 and/or 580 can be the first half of interrupt processing task and/or the latter half of interrupt processing task.As already mentioned, for the first half of interrupt processing task, by arranging the compatibility of interrupt request and CPU and/or CPU core, so that the interrupt request of memory device 431 is forwarded to specific CPU and/or CPU core, such as CPU core 401.For the latter half of interrupt processing task, schedule it to the upper execution of specific CPU and/or CPU core (such as CPU core 402).
In a further embodiment, except the time delay distribution of I/O request during except measuring memory device 431 disconnected Processing tasks being separately positioned on CPU core 401,402,411 or 412 wherein, the time delay distribution of I/O request when also measuring memory device 441 disconnected Processing tasks is separately positioned on CPU core 401,402,411 or 412 wherein, and for scheduling association in the interrupt processing task of memory device 431 and/or the execution of interrupt processing task on multiple CPU and/or CPU core being associated with memory device 441.
Still further in embodiment, also obtain the quantity of the passage (Lane) of the PCIE interface of coupled storage equipment 431,441.The number of channels of PCIE interface means data transfer bandwidth.For the memory device with larger data transmission demand, it is favourable for selecting to be coupled to the PCIE interface had compared with major path quantity.Also the quantity of the passage (Lane) of the PCIE interface of coupled storage equipment 431,441 can be supplied to user, the PCIE interface making user can select whether to adjust memory device 431,441 to be coupled, and/or select the CPU core performing the interrupt processing task be associated with memory device 431, and/or select the CPU core performing the application program be associated with memory device 431.
Present the description of this invention in order to the object illustrated and describe, and be not intended to disclosed form limit or restriction the present invention.To one of ordinary skill in the art, many adjustment and change are apparent.
Claims (6)
1., for the treatment of a method for the interrupt request of the memory device in computer system, described computer system comprises the first processing unit and the second processing unit, and described computer system is coupled with memory device; Described method comprises:
Receive the first I/O request for accessing described memory device;
Record the start time of described first I/O request;
In response to receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, record the end time of described first I/O request;
The very first time calculating the end time of described first I/O request and the start time of described first I/O request is poor;
Calculate the distribution of described very first time difference in a predetermined time period; And
Based on the distribution of described very first time difference, optimize the execution of the interrupt processing task be associated with the I/O request of described memory device;
Receive the second I/O request for accessing described memory device;
Record the start time of described second I/O request;
In response to receiving the interrupt request completed from the described second I/O request process of instruction of described memory device, performing described interrupt processing task, recording the end time of described second I/O request;
Calculate second mistiming of the end time of described second I/O request and the start time of described second I/O request;
Calculate the distribution of described second mistiming in a predetermined time period; And
Based on distribution and the distribution of described second mistiming of described very first time difference, interrupt processing task is set to perform on one of described first processing unit or described second processing unit, wherein, the distribution of described very first time difference corresponds to multiple described first I/O request, and described second mistiming distribution corresponds to multiple described second I/O request.
2. method according to claim 1, wherein said method also comprises, before receiving the first I/O request for accessing described memory device, the interrupt processing task be associated is set to perform on described first processing unit with the I/O request of described memory device;
After receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, perform described interrupt processing task; And
Based on the distribution of described very first time difference, interrupt processing task is set to perform on described second processing unit.
3., according to the described method of claim 1 or 2, also comprise and send the distribution of described very first time difference to application program.
4. described method according to claim 1, also comprises and sends the distribution of described very first time difference and/or the distribution of described second mistiming to application program.
5., for the treatment of a device for the interrupt request of the memory device in computer system, described computer system comprises the first processing unit and the second processing unit, and described computer system is coupled with memory device; Described device comprises:
For receiving the module of the first I/O request for accessing described memory device;
For recording the module of the start time of described first I/O request;
For in response to receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, record the module of the end time of described first I/O request;
For the module of the very first time difference of the start time of end time and described first I/O request of calculating described first I/O request;
For calculating the module of the distribution of described very first time difference in a predetermined time period; And
For the distribution based on described very first time difference, optimize the module of the execution of the interrupt processing task be associated with the I/O request of described memory device;
For receiving the module of the second I/O request for accessing described memory device;
For recording the module of the start time of described second I/O request;
For in response to receiving the interrupt request completed from the described second I/O request process of instruction of described memory device, performing described interrupt processing task, recording the module of the end time of described second I/O request;
For the module of second mistiming of the start time of end time and described second I/O request of calculating described second I/O request; For calculating the module of the distribution of described second mistiming in a predetermined time period; And
For based on the distribution of described very first time difference and the distribution of described second mistiming, interrupt processing task is set to the module performed on one of described first processing unit or described second processing unit, wherein, the distribution of described very first time difference corresponds to multiple described first I/O request, and described second mistiming distribution corresponds to multiple described second I/O request.
6. device according to claim 5, also comprises:
For before receiving the first I/O request for accessing described memory device, the interrupt processing task be associated is set to the module performed on described first processing unit with the I/O request of described memory device;
For after receiving the interrupt request completed from the described first I/O request process of instruction of described memory device, perform the module of described interrupt processing task; And
For the distribution based on described very first time difference, interrupt processing task is set to the module performed on described second processing unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210401747.9A CN102945214B (en) | 2012-10-19 | 2012-10-19 | Based on the method for IO distribution optimization time delay interrupt processing task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210401747.9A CN102945214B (en) | 2012-10-19 | 2012-10-19 | Based on the method for IO distribution optimization time delay interrupt processing task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102945214A CN102945214A (en) | 2013-02-27 |
| CN102945214B true CN102945214B (en) | 2016-02-10 |
Family
ID=47728162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210401747.9A Active CN102945214B (en) | 2012-10-19 | 2012-10-19 | Based on the method for IO distribution optimization time delay interrupt processing task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102945214B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104050071B (en) * | 2014-06-30 | 2017-02-08 | 苏州科达科技股份有限公司 | Storage device failure early warning method and system |
| CN107315700B (en) * | 2016-04-27 | 2020-12-08 | 华为技术有限公司 | Interrupt processing method and related device |
| KR102649324B1 (en) | 2016-05-19 | 2024-03-20 | 삼성전자주식회사 | Computer system for performing adaptive interrupt control and method for controlling interrupt thereof |
| CN110933179A (en) * | 2019-12-10 | 2020-03-27 | 国家电网有限公司信息通信分公司 | Asynchronous request monitoring method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1372659A (en) * | 2000-05-08 | 2002-10-02 | 三菱电机株式会社 | Computer system and computer-readable recording medium |
| CN1535427A (en) * | 2001-07-30 | 2004-10-06 | 皇家菲利浦电子有限公司 | Efficient interrupt system for system on chip design |
| CN101706742A (en) * | 2009-11-20 | 2010-05-12 | 北京航空航天大学 | Method for dispatching I/O of asymmetry virtual machine based on multi-core dynamic partitioning |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6847619B2 (en) * | 2001-05-11 | 2005-01-25 | Northrop Grumman Corporation | Dual mode of operation multiple access system for data link communication |
| US7529875B2 (en) * | 2003-08-20 | 2009-05-05 | International Business Machines Corporation | Assigning interrupts for input/output (I/O) devices among nodes of a non-uniform memory access (NUMA) system |
| US8635387B2 (en) * | 2009-10-09 | 2014-01-21 | Emulex Design & Manufacturing Corporation | Enhanced I/O performance in a multi-processor system via interrupt affinity schemes |
-
2012
- 2012-10-19 CN CN201210401747.9A patent/CN102945214B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1372659A (en) * | 2000-05-08 | 2002-10-02 | 三菱电机株式会社 | Computer system and computer-readable recording medium |
| CN1535427A (en) * | 2001-07-30 | 2004-10-06 | 皇家菲利浦电子有限公司 | Efficient interrupt system for system on chip design |
| CN101706742A (en) * | 2009-11-20 | 2010-05-12 | 北京航空航天大学 | Method for dispatching I/O of asymmetry virtual machine based on multi-core dynamic partitioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102945214A (en) | 2013-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11036650B2 (en) | System, apparatus and method for processing remote direct memory access operations with a device-attached memory | |
| KR102371916B1 (en) | Storage device for supporting virtual machines, storage system including the storage device, and method of the same | |
| EP3272072B1 (en) | Method, apparatus and system for dynamic bandwidth management in systems | |
| CN102567109B (en) | Interrupt distribution scheme | |
| US10585709B2 (en) | Job scheduling optimization based on ratio of stall to active cycles | |
| US8484495B2 (en) | Power management in a multi-processor computer system | |
| KR20210011451A (en) | Embedded scheduling of hardware resources for hardware acceleration | |
| US9600618B2 (en) | Implementing system irritator accelerator FPGA unit (AFU) residing behind a coherent attached processors interface (CAPI) unit | |
| KR101661259B1 (en) | Fast deskew when exiting low-power partial-width high speed link state | |
| US9460032B2 (en) | Apparatus and method for processing an interrupt | |
| KR20070024670A (en) | An apparatus and method for high performance volatile disk drive memory access using an integrated dma engine | |
| US20130086315A1 (en) | Direct memory access without main memory in a semiconductor storage device-based system | |
| CN102521179A (en) | Achieving device and achieving method of direct memory access (DMA) reading operation | |
| CN102945214B (en) | Based on the method for IO distribution optimization time delay interrupt processing task | |
| WO2016041191A1 (en) | Method and apparatus for reading and writing data, storage device and computer system | |
| CN104965678A (en) | Solid-state storage control method and apparatus and solid-state storage device | |
| US11734067B2 (en) | Multi-core system and controlling operation of the same | |
| KR20160128751A (en) | APPLICATION PROCESSOR, SYSTEM ON CHIP (SoC), AND COMPUTING DEVICE INCLUDING THE SoC | |
| EP3716084A1 (en) | Apparatus and method for sharing a flash device among multiple masters of a computing platform | |
| CN102929819B (en) | For the treatment of the method for the interrupt request of the memory device in computer system | |
| JP2013025794A (en) | Effective utilization of flash interface | |
| US9047264B2 (en) | Low pin count controller | |
| US9690619B2 (en) | Thread processing method and thread processing system for setting for each thread priority level of access right to access shared memory | |
| US10649943B2 (en) | System and method for I/O aware processor configuration | |
| CN102789384B (en) | Method for implementing server operating system applied to Loongson 3B processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 100192 room A302, building B-2, Dongsheng Science Park, Zhongguancun, 66 xixiaokou Road, Haidian District, Beijing Patentee after: Beijing yihengchuangyuan Technology Co.,Ltd. Address before: 312, building D, entrepreneurship Park, No. 2, Shangdi Information Road, Haidian District, Beijing 100085 Patentee before: MEMBLAZE TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| CP03 | Change of name, title or address |