CN111737176A - PCIE data-based synchronization device and driving method - Google Patents
PCIE data-based synchronization device and driving method Download PDFInfo
- Publication number
- CN111737176A CN111737176A CN202010392108.5A CN202010392108A CN111737176A CN 111737176 A CN111737176 A CN 111737176A CN 202010392108 A CN202010392108 A CN 202010392108A CN 111737176 A CN111737176 A CN 111737176A
- Authority
- CN
- China
- Prior art keywords
- virtual channel
- unit
- operation request
- steps
- following
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/382—Information transfer, e.g. on bus using universal interface adapter
- G06F13/385—Information transfer, e.g. on bus using universal interface adapter for adaptation of a particular data processing system to different peripheral devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bus Control (AREA)
Abstract
The invention provides a PCIE data-based synchronization device and a driving method, wherein the driving method comprises the following steps: the method comprises the following steps: an MMU unit acquires an operation request to an IOVA; the MMU unit converts the operation request of the IOVA into a virtual channel in the VF unit; the PF unit judges whether a responding virtual channel with higher priority than the virtual channel exists, if so, the PF unit suspends responding to the virtual channel, and enables the virtual channel to enter a waiting queue, and after the response of the virtual channel with the higher priority is finished, the PF unit responds to the operation request of the virtual channel, and if not, the PF unit responds to the operation request of the virtual channel. The technical scheme can ensure the data orderliness, avoid consuming excessive bandwidth of the flow and realize the flow control of each thread.
Description
Technical Field
The invention relates to the field of computer communication, in particular to a PCIE data-based synchronization device and a driving method.
Background
PCIe (peripheral component interconnect express) may map a PCIe domain address to a chip storage domain address through an ATS (address translation service), and a Memory Management Unit (MMU) of the chip may map a segment of the storage domain address in a kernel space and a user state space of an operating system to form a corresponding IOVA segment. Because of the data management for the address mapping mode, if different software and threads need to access the region at the same time, a signal synchronization mechanism, such as a software lock, is inevitably required to be added. Such a mechanism, while guaranteeing data ordering, may consume CPU bandwidth unnecessarily, and lock contention may actually reduce the bandwidth of the entire flow. And flow control cannot be performed on each thread, and hard real-time indexes of partial access and the like cannot be guaranteed.
Disclosure of Invention
Therefore, a PCIE data-based synchronization apparatus and a driving method are needed to solve the problem of the utilization rate of the CPU bandwidth.
In order to achieve the above object, the inventor provides a method for driving a synchronization apparatus based on PCIE data, including the following steps:
an MMU unit acquires an operation request to an IOVA;
the MMU unit converts the operation request of the IOVA into a virtual channel in the VF unit;
the PF unit judges whether a responding virtual channel with higher priority than the virtual channel exists, if so, the PF unit suspends responding to the virtual channel, and enables the virtual channel to enter a waiting queue, and after the response of the virtual channel with the higher priority is finished, the PF unit responds to the operation request of the virtual channel, and if not, the PF unit responds to the operation request of the virtual channel.
Further, when the virtual channel is entered into the waiting queue, the method further comprises the following steps:
and the PF unit judges whether the virtual channels in the waiting queue exceed preset time, and if so, the PF unit promotes the priority level of the virtual channels to the highest priority level.
Further, if yes, the PF unit raises the priority of the virtual channel to the highest priority, further including the steps of:
and the VF unit controls the virtual channel which is lifted to the highest priority level to carry out single-stroke transmission.
Further, the method also comprises the following steps:
after the virtual channel with the highest priority is promoted to perform single transmission, the VF unit restores the original priority level of the virtual channel.
Further, the method also comprises the following steps:
the PF unit judges whether a responding virtual channel with lower priority than the virtual channel exists, if so, the PF unit responds to the operation request of the virtual channel, the virtual channel with lower priority enters a waiting queue, after the operation request response of the virtual channel is finished, the PF unit responds to the suspended operation request with lower priority, and if not, the PF unit responds to the operation request of the virtual channel.
Further, when the virtual channel is entered into the waiting queue, the method further comprises the following steps:
the PF unit controls the virtual channel to enter the buffer of the VF unit.
Further, when the virtual channel is entered into the waiting queue, the method further comprises the following steps:
and the real-time control module of the MMU unit closes the translation action of the access request of the corresponding IOVA of the virtual channel.
Further, the method also comprises the following steps:
a page fault message is generated, and the real-time control module shelves the page fault message.
Further, the method also comprises the following steps:
and the real-time control module responds to the translation action of the access request of the corresponding IOVA of the virtual channel and eliminates page fault messages.
The inventor provides a synchronization device based on PCIE data, including an MMU unit, a VF unit, and a PF unit, where the MMU unit, the VF unit, and the PF unit are configured to execute a synchronization management method based on PCIE data according to any one of the embodiments.
Different from the prior art, the technical scheme enables the high-priority virtual channel to operate firstly, and the low-priority virtual channel enters the waiting queue when the high-priority virtual channel responds. The virtual channels with low priority in the waiting queue also determine the order of response according to the order of the priority from high to low. The method can ensure the orderliness of data, avoid consuming excessive bandwidth of the flow and realize the flow control of each thread.
Drawings
Fig. 1 is an architecture diagram of a synchronization management apparatus based on PCIE data according to this embodiment.
Detailed Description
To explain technical contents, structural features, and objects and effects of the technical solutions in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
Referring to fig. 1, the present embodiment is a method for synchronous management based on PCIE data, including the following steps: a Memory Management Unit (MMU) Unit obtains an operation request for an IOVA (input/output virtual address), and the MMU Unit converts the operation request for the IOVA into a virtual channel in a virtual-Function (VF) Unit. A Physical channel (PF) unit determines whether there is a responding virtual channel with a higher priority than the virtual channel. If yes, the PF unit suspends responding to the virtual channel, enables the virtual channel to enter a waiting queue, and responds to the operation request of the virtual channel after the response of the high-priority virtual channel is finished. And if not, the PF unit responds to the operation request of the virtual channel. It should be noted that the configuration priority may be performed by a Quality of service (QOS) module in the PF unit.
According to the technical scheme, the PF unit configures the priority for each virtual channel, and runs the operation requests (such as writing or reading) of the application or the thread according to the sequence of the priority from high to low. And the high-priority virtual channel is enabled to operate firstly, and the low-priority virtual channel enters a waiting queue when the high-priority virtual channel responds. The virtual channels with low priority in the waiting queue also determine the order of response according to the order of the priority from high to low. The method can ensure the orderliness of data, avoid consuming excessive bandwidth of the flow and realize the flow control of each thread.
An MMU unit of a chip maps a Physical Address (PA) of a storage domain to a plurality of virtual addresses (IOVA) of the storage domain, and then allocates a virtual-Function (VF) to each application or thread that needs to access the Physical Address. Referring to fig. 1, two physical addresses, PA1 and PA2, two IOVA1 and IOVA2, and two virtual channels, VF1 and VF2 are shown. And one virtual channel corresponds to one IOVA through the ATS unit.
The virtual channel has the same address configuration for the PCIe domain, so that the virtual channel corresponds to the physical address PA after the ATS, but the IOVA is different after the virtual channel passes through the MMU, so that the addresses seen from different applications or threads are different, the virtual channel is considered to be different devices, and software does not need to additionally increase a locking mechanism. The software efficiency can be improved, and the bandwidth utilization rate can be fully improved.
In this embodiment, a high priority virtual channel may interrupt the operation of a low priority virtual channel. If a virtual channel is configured with the highest priority (also referred to as real-time transmission mode), other ongoing transmissions with lower priority are stopped. Specifically, the PF unit determines whether there is a responding virtual channel with a lower priority than the virtual channel, and if so, the PF unit responds to the operation request of the virtual channel, and allows the virtual channel with the lower priority to enter a waiting queue, and after the operation request response of the virtual channel is finished, responds to the suspended operation request with the lower priority, and if not, the PF unit responds to the operation request of the virtual channel.
In this embodiment, in order to prohibit the low-priority virtual channel from further receiving the transmission data sent by the software, when the low-priority virtual channel is buffered in the buffer of the VF unit to form the wait queue, the real-time control unit (thread unit) of the memory management unit closes the translation operation of the access request of the virtual channel corresponding to the IOVA. It should be noted that the translation action refers to a process from the correspondence between the physical address and the IOVA to the virtual channel, and by limiting the process, the virtual channel with low priority can be inhibited from receiving the transmission data sent by the software.
In a further embodiment, if a page fault message (page fault) is generated after shutdown, the page fault message is not sent to upper layer software. And the real-time control module responds to the translation action of the access request of the corresponding IOVA of the virtual channel and eliminates page fault messages.
In this embodiment, after the low-priority virtual channel is preempted for a long time by other high-priority virtual channels or virtual channels requiring real-time performance, in order to prevent the user status from being abnormal, a maximum timeout transmission time (preset time) is configured for the virtual channels in the wait queue. The method can be carried out by depending on a quality management unit of a memory management unit, and a timer can be used for exhausting the maximum overtime transmission time and sending out a corresponding request after the maximum overtime transmission time is exhausted. And the PF unit judges whether the virtual channels in the waiting queue exceed preset time, and if so, the PF unit promotes the priority level of the virtual channels to the highest priority level. Then automatically reducing the priority again, waiting for queuing and then allowing the transmission of the queue or waiting for the next maximum timeout time to be exhausted again and then being temporarily allowed to insert a transmission. If not, the queue is continuously arranged in the waiting queue. The method is an emergency measure for the virtual channel with the low priority level, so that the abnormal user state caused by long-time non-running of the process or thread with the low priority level is avoided.
In this embodiment, in order to maintain the ordering of the same priority level, the quality management unit of the memory management unit uses a rotation method to allocate the time slices to a plurality of virtual channels at the same priority level. High priority applications may interrupt low priority in the quality management unit and also occupy a larger transmission time slice. The same-level priority is subjected to a round robin strategy by allocating time slices by the quality management unit. Let a virtual channel run on the CPU for a time slice, such as 100ms (milliseconds) of time, this 100ms interval is called a time slice. When a process runs out of time slices allocated to it, the scheduler stops the process and puts it at the end of the ready queue, letting the next virtual channel also execute a time slice. Of course, the time slice for each virtual channel may be different and set according to the actual requirements of the program or process. Therefore, a plurality of processes or programs can be responded by the system in time, and the running efficiency is improved.
It should be noted that, only two connections between the physical addresses, the IOVA, and the virtual channels are illustrated herein, for example, three, four, or even more connections between the physical addresses, the IOVA, and the virtual channels are still a connection relationship where one physical address corresponds to one IOVA, and one IOVA corresponds to one virtual channel.
The present embodiment provides a synchronization management apparatus based on PCIE data, including an MMU unit, a VF unit, and a PF unit, where the MMU unit, the VF unit, and the PF unit are configured to execute any one of the synchronization management methods based on PCIE data in this embodiment.
It should be noted that, although the above embodiments have been described herein, the invention is not limited thereto. Therefore, based on the innovative concepts of the present invention, the technical solutions of the present invention can be directly or indirectly applied to other related technical fields by making changes and modifications to the embodiments described herein, or by using equivalent structures or equivalent processes performed in the content of the present specification and the attached drawings, which are included in the scope of the present invention.
Claims (10)
1. A driving method of a synchronization device based on PCIE data is characterized by comprising the following steps:
an MMU unit acquires an operation request to an IOVA;
the MMU unit converts the operation request of the IOVA into a virtual channel in the VF unit;
the PF unit judges whether a responding virtual channel with higher priority than the virtual channel exists, if so, the PF unit suspends responding to the virtual channel, and enables the virtual channel to enter a waiting queue, and after the response of the virtual channel with the higher priority is finished, the PF unit responds to the operation request of the virtual channel, and if not, the PF unit responds to the operation request of the virtual channel.
2. The method according to claim 1, wherein when the virtual channel is allowed to enter the wait queue, the method further comprises the following steps:
and the PF unit judges whether the virtual channels in the waiting queue exceed preset time, and if so, the PF unit promotes the priority level of the virtual channels to the highest priority level.
3. The method according to claim 2, wherein if the PF unit has raised the priority level of the virtual channel to the highest priority level, the method further comprises the following steps:
and the VF unit controls the virtual channel which is lifted to the highest priority level to carry out single-stroke transmission.
4. The method according to claim 3, further comprising the following steps:
after the virtual channel with the highest priority is promoted to perform single transmission, the VF unit restores the original priority level of the virtual channel.
5. The method according to claim 1, further comprising the following steps:
the PF unit judges whether a responding virtual channel with lower priority than the virtual channel exists, if so, the PF unit responds to the operation request of the virtual channel, the virtual channel with lower priority enters a waiting queue, after the operation request response of the virtual channel is finished, the PF unit responds to the suspended operation request with lower priority, and if not, the PF unit responds to the operation request of the virtual channel.
6. The method according to claim 1, wherein when the virtual channel is allowed to enter the wait queue, the method further comprises the following steps:
the PF unit controls the virtual channel to enter the buffer of the VF unit.
7. The method according to claim 1, wherein when the virtual channel is allowed to enter the wait queue, the method further comprises the following steps:
and the real-time control module of the MMU unit closes the translation action of the access request of the corresponding IOVA of the virtual channel.
8. The method according to claim 6, further comprising the following steps:
a page fault message is generated, and the real-time control module shelves the page fault message.
9. The method according to claim 8, further comprising the following steps:
and the real-time control module responds to the translation action of the access request of the corresponding IOVA of the virtual channel and eliminates page fault messages.
10. A synchronization apparatus based on PCIE data, comprising an MMU unit, a VF unit, and a PF unit, where the MMU unit, the VF unit, and the PF unit are configured to execute the synchronization management method based on PCIE data according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010392108.5A CN111737176B (en) | 2020-05-11 | 2020-05-11 | PCIE data-based synchronization device and driving method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010392108.5A CN111737176B (en) | 2020-05-11 | 2020-05-11 | PCIE data-based synchronization device and driving method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111737176A true CN111737176A (en) | 2020-10-02 |
| CN111737176B CN111737176B (en) | 2022-07-15 |
Family
ID=72647013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010392108.5A Active CN111737176B (en) | 2020-05-11 | 2020-05-11 | PCIE data-based synchronization device and driving method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111737176B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080109565A1 (en) * | 2006-11-02 | 2008-05-08 | Jasmin Ajanovic | PCI express enhancements and extensions |
| CN102650976A (en) * | 2012-04-01 | 2012-08-29 | 中国科学院计算技术研究所 | Control device and method supporting single IO (Input/Output) virtual user level interface |
| CN102662763A (en) * | 2012-04-11 | 2012-09-12 | 华中科技大学 | Virtual machine resource scheduling method based on service quality |
| CN103049331A (en) * | 2012-12-06 | 2013-04-17 | 华中科技大学 | Dynamic scheduling method with virtual function |
| CN103647807A (en) * | 2013-11-27 | 2014-03-19 | 华为技术有限公司 | Information caching method, device and communication apparatus |
| CN104685479A (en) * | 2012-08-29 | 2015-06-03 | 超威半导体公司 | Virtual input/output memory management unit wihtin a guest virtual machine |
| CN105487990A (en) * | 2014-09-19 | 2016-04-13 | 中兴通讯股份有限公司 | Method and device for transmitting information messages between CPU and chip |
| CN106664242A (en) * | 2015-07-03 | 2017-05-10 | 华为技术有限公司 | Network configuration method, network system and device |
| CN107209681A (en) * | 2015-10-21 | 2017-09-26 | 华为技术有限公司 | A kind of storage device access methods, devices and systems |
| CN107889529A (en) * | 2015-08-07 | 2018-04-06 | 高通股份有限公司 | Share the dynamic data link selection on physical interface |
| CN108027642A (en) * | 2015-06-24 | 2018-05-11 | 英特尔公司 | System and method for isolation input/output computing resource |
| CN108958884A (en) * | 2018-06-22 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method and relevant apparatus of Virtual Machine Manager |
| CN109196486A (en) * | 2016-05-24 | 2019-01-11 | 赛灵思公司 | Memory pre-fetch for virtual memory |
| CN109496296A (en) * | 2016-07-26 | 2019-03-19 | 微软技术许可有限责任公司 | Remote metering system is set to be shown as local hardware in virtualized environment |
| CN110121698A (en) * | 2016-12-31 | 2019-08-13 | 英特尔公司 | System, method and apparatus for Heterogeneous Computing |
| CN110275774A (en) * | 2018-03-13 | 2019-09-24 | 三星电子株式会社 | The mechanism of physical storage device resource is dynamically distributed in virtualized environment |
| CN110830392A (en) * | 2018-08-07 | 2020-02-21 | 马维尔国际贸易有限公司 | Enabling virtual functions on a storage medium |
-
2020
- 2020-05-11 CN CN202010392108.5A patent/CN111737176B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080109565A1 (en) * | 2006-11-02 | 2008-05-08 | Jasmin Ajanovic | PCI express enhancements and extensions |
| CN102650976A (en) * | 2012-04-01 | 2012-08-29 | 中国科学院计算技术研究所 | Control device and method supporting single IO (Input/Output) virtual user level interface |
| CN102662763A (en) * | 2012-04-11 | 2012-09-12 | 华中科技大学 | Virtual machine resource scheduling method based on service quality |
| CN104685479A (en) * | 2012-08-29 | 2015-06-03 | 超威半导体公司 | Virtual input/output memory management unit wihtin a guest virtual machine |
| CN103049331A (en) * | 2012-12-06 | 2013-04-17 | 华中科技大学 | Dynamic scheduling method with virtual function |
| CN103647807A (en) * | 2013-11-27 | 2014-03-19 | 华为技术有限公司 | Information caching method, device and communication apparatus |
| CN105487990A (en) * | 2014-09-19 | 2016-04-13 | 中兴通讯股份有限公司 | Method and device for transmitting information messages between CPU and chip |
| CN108027642A (en) * | 2015-06-24 | 2018-05-11 | 英特尔公司 | System and method for isolation input/output computing resource |
| CN106664242A (en) * | 2015-07-03 | 2017-05-10 | 华为技术有限公司 | Network configuration method, network system and device |
| CN107889529A (en) * | 2015-08-07 | 2018-04-06 | 高通股份有限公司 | Share the dynamic data link selection on physical interface |
| CN107209681A (en) * | 2015-10-21 | 2017-09-26 | 华为技术有限公司 | A kind of storage device access methods, devices and systems |
| CN109196486A (en) * | 2016-05-24 | 2019-01-11 | 赛灵思公司 | Memory pre-fetch for virtual memory |
| CN109496296A (en) * | 2016-07-26 | 2019-03-19 | 微软技术许可有限责任公司 | Remote metering system is set to be shown as local hardware in virtualized environment |
| CN110121698A (en) * | 2016-12-31 | 2019-08-13 | 英特尔公司 | System, method and apparatus for Heterogeneous Computing |
| CN110275774A (en) * | 2018-03-13 | 2019-09-24 | 三星电子株式会社 | The mechanism of physical storage device resource is dynamically distributed in virtualized environment |
| CN108958884A (en) * | 2018-06-22 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method and relevant apparatus of Virtual Machine Manager |
| CN110830392A (en) * | 2018-08-07 | 2020-02-21 | 马维尔国际贸易有限公司 | Enabling virtual functions on a storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111737176B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10897428B2 (en) | Method, server system and computer program product for managing resources | |
| US20230128059A1 (en) | Dynamic resources allocation method and system for guaranteeing tail latency slo of latency-sensitive application | |
| EP2506147B1 (en) | Epoll optimisations | |
| US8478926B1 (en) | Co-processing acceleration method, apparatus, and system | |
| WO2019075980A1 (en) | Thread adjustment method and terminal thereof | |
| US8627325B2 (en) | Scheduling memory usage of a workload | |
| US9389921B2 (en) | System and method for flexible device driver resource allocation | |
| US11899939B2 (en) | Read/write request processing method and apparatus, electronic device, and storage medium | |
| US20220222111A1 (en) | Deep learning framework scheduling | |
| CN109284192B (en) | Parameter configuration method and electronic equipment | |
| WO2024088268A1 (en) | Rdma event management methods, and device and storage medium | |
| CN112783659A (en) | Resource allocation method and device, computer equipment and storage medium | |
| WO2022057718A1 (en) | Coding scheduling method, server and client terminal, and system for acquiring remote desktop | |
| EP1029284B1 (en) | Shared memory access controller | |
| CN111737176B (en) | PCIE data-based synchronization device and driving method | |
| CN115378885B (en) | Virtual machine service network bandwidth management method and device under super fusion architecture | |
| US12019909B2 (en) | IO request pipeline processing device, method and system, and storage medium | |
| US8869171B2 (en) | Low-latency communications | |
| CN115344350A (en) | Node equipment of cloud service system and resource processing method | |
| KR20120067865A (en) | Channel multiplexing method and apparatus in shared memory | |
| CN112328387A (en) | Multi-tenant-oriented micro-service resource scheduling method | |
| US9330036B2 (en) | Interrupt reduction by dynamic application buffering | |
| US11921648B1 (en) | Statistic-based adaptive polling driver | |
| CN118138668A (en) | Method, device and equipment for adjusting time slices of protocol stack process | |
| JP4877317B2 (en) | Information processing apparatus and interrupt control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 350003 building 18, No.89, software Avenue, Gulou District, Fuzhou City, Fujian Province Applicant after: Ruixin Microelectronics Co.,Ltd. Address before: 350003 building 18, No.89, software Avenue, Gulou District, Fuzhou City, Fujian Province Applicant before: FUZHOU ROCKCHIP ELECTRONICS Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |