WO2018113701A1 - 资源调度 - Google Patents

资源调度 Download PDF

Info

Publication number
WO2018113701A1
WO2018113701A1 PCT/CN2017/117507 CN2017117507W WO2018113701A1 WO 2018113701 A1 WO2018113701 A1 WO 2018113701A1 CN 2017117507 W CN2017117507 W CN 2017117507W WO 2018113701 A1 WO2018113701 A1 WO 2018113701A1
Authority
WO
WIPO (PCT)
Prior art keywords
service board
user
entry
data plane
slot
Prior art date
Application number
PCT/CN2017/117507
Other languages
English (en)
French (fr)
Inventor
廖以顺
章靠
寇远芳
Original Assignee
新华三技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 新华三技术有限公司 filed Critical 新华三技术有限公司
Priority to EP17882416.5A priority Critical patent/EP3544251B1/en
Priority to JP2019533519A priority patent/JP6741872B2/ja
Priority to US16/472,861 priority patent/US11184291B2/en
Publication of WO2018113701A1 publication Critical patent/WO2018113701A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2854—Wide area networks, e.g. public data networks
    • H04L12/2856—Access arrangements, e.g. Internet access
    • H04L12/2869—Operational details of access network equipments
    • H04L12/287—Remote access server, e.g. BRAS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/50—Address allocation
    • H04L61/5007—Internet protocol [IP] addresses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/029—Firewall traversal, e.g. tunnelling or, creating pinholes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/0892—Network architectures or network communication protocols for network security for authentication of entities by using authentication-authorization-accounting [AAA] servers or protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Definitions

  • the BRAS (Broadband Remote Access Server) service system is mainly composed of a user host, a BRAS device, and an AAA (Authentication Authorization and Accounting) server.
  • the BRAS device sends the user information sent by the user host to the AAA server, thereby completing authentication, authorization, and accounting for the user.
  • the user host can access the external network, such as the Internet, through the BRAS device.
  • FIG. 1 is a schematic structural diagram of a BRAS device according to an exemplary embodiment of the present application.
  • FIG. 2 is a flowchart of a resource scheduling method according to an exemplary embodiment of the present application
  • FIG. 3 is a flowchart of forwarding data packets sent to a user host according to an exemplary embodiment of the present application
  • FIG. 4 is another flowchart of forwarding a data packet sent to a user host according to an exemplary embodiment of the present application
  • FIG. 5 is a schematic structural diagram of a hardware of a distributed access gateway device according to an exemplary embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of resource scheduling logic according to an exemplary embodiment of the present application.
  • FIG. 7 is another schematic structural diagram of resource scheduling logic according to an exemplary embodiment of the present application.
  • FIG. 8 is still another schematic structural diagram of resource scheduling logic according to an exemplary embodiment of the present application.
  • FIG. 9 is still another schematic structural diagram of resource scheduling logic according to an exemplary embodiment of the present application.
  • the BRAS device adopting a distributed hardware architecture includes: a main control board 100 and a plurality of service boards 1_1 to 1_5, and the main control board 100 is responsible for controlling and managing all service boards and synchronizing global information; Responsible for access authentication and packet forwarding processing.
  • the process of access authentication and packet forwarding processing on the service board is as follows:
  • the data plane receives the access authentication request packet sent by the user host, and identifies that the packet is a protocol packet, and sends the packet to the control plane for processing;
  • control plane After receiving the access authentication request packet, the control plane completes the authentication of the user by interacting with the AAA server. After the user authentication is passed, the control plane establishes a software user entry corresponding to the user, and the entry may be recorded in the memory; The control plane notifies the data plane to create a user entry.
  • the data plane After receiving the notification, the data plane creates a software user entry and a hardware user entry corresponding to the user, and the hardware user entry is usually recorded on a TCAM (Ternary Content Addressable Memory) device;
  • TCAM Ternary Content Addressable Memory
  • the data plane receives the data packet sent by the user host after the authentication is passed. If the packet is a non-protocol packet, the data plane is searched for the corresponding hardware user entry. After the hardware user entry is found, the hardware user entry is obtained according to the hardware user entry. Perform corresponding processing, for example, statistics traffic and duration, user policy control, etc., and then forward the data packet through routing.
  • the control plane may be a CPU (Central Processing Unit), and the data plane may be a driver.
  • CPU Central Processing Unit
  • data plane may be a driver.
  • the hardware resource specifications of each service board are fixed. For example, 16K user access can be supported. In this case, the total number of configurable hardware user entries is 16K, if hardware resources on a service board. The newly added user's hardware user entry cannot be delivered due to insufficient hardware resources, which may result in the new user being unable to go online.
  • the method may be performed by a first service board in a distributed access gateway device as shown in FIG. 1, for example, a data plane of a first service board, where the method includes: after the first user authentication is passed, creating The first software user entry corresponding to the first user determines whether there is an idle hardware resource on the first service board; if not, the slot ID (Identity) of the second service board with the idle hardware resource is obtained.
  • the second service board is selected from the resource group, and the resource group is composed of two or more service boards; the first software user entry is sent to the second service board according to the obtained slot ID, so that The second service board creates a hardware user entry corresponding to the first user.
  • the data plane when the data plane creates a software user entry corresponding to a certain user, and needs to create a hardware user entry corresponding to the user, if it is determined that the first service board has been If there is no idle hardware resource, the slot ID of the second service board that has the idle hardware resource in the resource group is obtained, and the created software user entry is sent to the second service board according to the obtained slot ID.
  • the second service board creates a hardware user entry corresponding to the user. Therefore, when the hardware resources of the first service board are insufficient, the hardware user entry of the newly added user may be configured on the second service board, and the second service board shares the hardware resource to the first service with insufficient hardware resources. The board ensures that the new user can go online and realizes the rational utilization and scheduling of the hardware resources of the distributed gateway device.
  • the distributed BRAS device shown in FIG. 1 is used as an example.
  • the distributed BRAS device includes: a main control board 100 and multiple service boards.
  • the service board that supports the BRAS function exists in the distributed BRAS device, and the service board that does not support the BRAS function exists.
  • the embodiment of the present application only considers the service board that supports the BRAS function, that is, the service board mentioned below. Both are service boards that support the BRAS function.
  • the main control board maintains a resource group consisting of two or more service boards that support the BRAS function in the distributed BRAS device. Specifically, the main control board records the resource group information, including: the slot ID, the hardware resource specification, and the hardware resource usage of each service board in the resource group.
  • the hardware resource usage can be represented by the number of idle hardware resources.
  • the resource group includes the five service boards in Figure 1.
  • the service board 1_1 with the slot ID Slot2_1, the service board 1_2 with the slot ID Slot2_2, the service board 1_3 with the slot ID Slot2_3, and the slot ID are The service board 1_4 of the Slot 2_4 and the service board 1_5 of the slot ID of Slot 2_5.
  • the resource group information recorded on the main control board is shown in Table 1:
  • Each service board in the resource group notifies the main control board when the usage of the hardware resources of the service board changes, so that the main control board can update the idle hardware resources of the service board in Table 1 in real time.
  • the main control board is updated in Table 1, for example, the service board is added or removed, the number of idle hardware resources of a service board changes, and the updated information is synchronized to each service board in the resource group.
  • resource group information as shown in Table 1 is also maintained on the control plane of each service board in the resource group.
  • the control plane of the first service board acquires the resource group information from the main control board, and selects the resource group according to the obtained resource group information.
  • a second service board with idle hardware resources sends the slot ID of the second service board to the data plane.
  • the control plane of the first service board after receiving the notification message sent by the data plane to indicate that the hardware resource is insufficient, notifies the main control board to select a second service that has idle hardware resources from the resource group according to the resource group information.
  • the board returns the slot ID of the second service board.
  • the control plane After obtaining the slot ID of the second service board, the control plane sends the slot ID to the data plane.
  • the service board can be added to the resource group by manual static configuration.
  • the service board can be selectively added to the resource group.
  • the hardware resource specifications of the service board are very low, and the service board is low. It is mainly used to implement other functions. You can not add the service board to the resource group.
  • the service board can also be added to the resource group in a dynamic manner. Specifically, after the newly inserted service board is started, the main control board identifies the type of the service board. When it is determined that the service board supports the BRAS function, the service board is added to the resource group, and the service board is added in the table 1. Slot ID, hardware resource specifications, and idle hardware resources of the service board. In this way, the management board can be automatically added to the resource group by the main control board without manual configuration by the administrator.
  • the resource scheduling method in the embodiment of the present application includes the following content:
  • the data plane of the service board 1_1 After receiving the access authentication request packet sent by the user host, the data plane of the service board 1_1 identifies the packet as a protocol packet, and sends the packet to the control plane for processing; the control plane receives the access.
  • the user After the authentication request is received, the user is authenticated with the AAA server. After the user passes the authentication, the control plane establishes a software user entry corresponding to the user and notifies the data plane to create a user entry.
  • the data plane of the service board 1_1 After receiving the notification of creating a user entry sent by the control plane, the data plane of the service board 1_1 performs the steps shown in Figure 2:
  • Step S201 creating a software user entry corresponding to the user, for the convenience of description, the entry is recorded as entry3_1;
  • the entry3_1 can be stored in the memory of the data plane. As shown in Table 2-1, the entry3_1 can include the user name, the MAC address and IP address of the user host, the rate limit information, and the slot of the original service board accessed by the user host. ID, the port (that is, the original port) ID of the user host connected to the service board.
  • Step S202 it is determined whether there is idle hardware resources on the service board, and if so, step S203 is performed, otherwise, step S204 is performed;
  • Step S203 creating a hardware user entry corresponding to the user, and then exiting the process
  • Step S204 sending a notification message indicating that the hardware resource is insufficient to the control plane
  • a flag may be added to the software user entry entry 3_1 as shown in Table 2-1 to indicate that there is no idle hardware resource on the service board. Redirect to other business boards, for example, the tag is represented by NoResRedi.
  • entry3_1 as shown in Table 2-1 is updated as shown in Table 2-2:
  • the control plane of the service board 1_1 selects a service board with idle hardware resources from the resource group according to the resource group information, for example, selecting hardware.
  • the slot ID with the highest resource idle rate is Slot 2_3, and then Slot 2_3 is sent to the data plane.
  • the control plane of the service board 1_1 obtains the resource group information from the main control board after receiving the notification message, and obtains the resource group information according to the obtained resource group information as shown in Table 1.
  • Select a service board with idle hardware resources For example, select the service board 1_3 with the slot ID with the highest hardware resource idle rate as Slot2_3, and then send Slot2_3 to the data plane.
  • the control plane of the service board 1_1 notifies the main control board to select one of the resource groups from the resource group according to the resource group information as shown in Table 1.
  • the service board selects the service board 1_3 with the slot ID of the slot with the highest hardware resource idle rate as Slot 2_3, and returns the Slot 2_3 to the control plane of the service board 1_1.
  • the control plane of the service board 1_1 sends the Slot 2_3 to the data. flat.
  • Step S205 receiving the Slot2_3 sent by the control plane, and sending the created software user entry entry3_1 to the service board 1_3;
  • the data plane of the service board 1_1 may send the content of the entry 3_1 to the service board 1_3 through an inter-board communication message, where the inter-board communication message may be, for example, an Inter-Process Communication (IPC) message.
  • IPC Inter-Process Communication
  • the data plane of the service board 1_3 After receiving the inter-board communication message, the data plane of the service board 1_3 creates the software user entry entry3_2 and the hardware user entry entry3_3 corresponding to the user according to the content of the entry 3_1 of the software user entry carried in the message, and enters entry3_2
  • the slot ID Slot2_1 of the service board 1_1 is used to indicate that the original service board accessed by the user is the service board 1_1.
  • the contents of entry3_2 and entry3_3 are shown in Table 3 and Table 4, respectively:
  • the entry 3_2 of the software user entry as shown in Table 3 is usually stored in the memory of the data plane.
  • the entry 3_2 mainly includes: the user name, the MAC address and IP address of the user host, the rate limit information, and the corresponding hardware user entry entry3_3. Index, and the slot ID of the original service board accessed by the user; the hardware user entry entry3_3 shown in Table 4 can be stored in devices such as TCAM.
  • Entry3_3 mainly includes: user name, user host MAC address, and IP address. Rate limit information, the group ID of the user, the domain name to which it belongs, the corresponding user policy ID, and so on.
  • the data plane of the service board 1_3 may reply to the service board 1_1 with an inter-board communication message indicating that the hardware user entry is successfully created.
  • Step S206 after receiving the inter-board communication message sent by the service board 1_3 to indicate that the hardware user entry is successfully created, record the slot of the service board 1_3 in the software user entry entry 3_1 as shown in Table 2-2. ID Slot2_3, indicating that the user is redirected to the service board 1_3. At this point, entry3_1 is updated as shown in Table 2-3:
  • the data plane of the service board 1_3 also creates a routing entry containing the host host IP address. For the routing entry, see Table 5:
  • the destination device ID in the routing entry is the ID of the resource group, which is RG, and the destination port ID is the slot ID Slot2_3 of the service board 1_3.
  • the service board 1_3 also synchronizes the routing entries shown in Table 5 to other service boards through the main control board, including service board 1_1, service board 1_2, service board 1_4, and service board 1_5.
  • each of the service boards 1_1, the service board 1_2, the service board 1_4, and the service board 1_5 will follow any of the following two methods.
  • a routing entry is delivered:
  • the routing entries including the PC-IP on the data planes of the service board 1_1, the service board 1_2, the service board 1_4, and the service board 1_5 are as shown in Table 5.
  • Method 2 Find the corresponding software user entry based on the IP address PC-IP included in the routing entry. If the user host whose IP address is PC-IP is not the user host connected to the service board, Save the routing entry directly on the data plane. If the user host whose IP address is PC-IP is the user host that accesses the service board, save the routing entry and save the saved routing entry. DDev is modified to be the slot ID of the service board, and the DPort is modified to the original port ID recorded in the found software user entry.
  • the service board 1_1 finds the software user entry as shown in Table 2-3. Therefore, the routing table entry of the PC-IP on the data plane of the service board 1_1 is as shown in Table 6. 1_2, the service board 1_4, and the service board 1_5 have not found the corresponding software user entry. Therefore, the routing entries of the PC-IP on the data plane of these service boards are shown in Table 5.
  • the routing entries including the PC-IP on the data planes of the service board 1_2, the service board 1_4, and the service board 1_5 are as shown in Table 5, regardless of the mode 1 or mode 2.
  • the routing table entry including the PC-IP on the data plane is as shown in Table 5
  • the routing entry containing the PC-IP on the data plane is shown in Table 6.
  • the routing entries in the different manners are different. Therefore, the forwarding processing of the data packets sent to the user host is different.
  • the method of forwarding when the mode 2 is adopted can be referred to the method shown in FIG. 4 later.
  • the user host After the authentication is passed, the user host sends a data packet whose source IP address is PC-IP to access the Internet.
  • the data plane of the service board 1_1 identifies the packet as a non-protocol packet.
  • the device searches for the corresponding routing entry based on the destination IP address of the data packet, and then searches for the corresponding software user entry and hardware user entry based on the source IP address PC-IP of the data packet.
  • NoResRedi is marked in entry3_1, it can be determined that the user is redirected to another service board, and the data packet is sent to the service board 1_3 through the switching network according to Slot2_3 recorded in entry3_1.
  • the data plane of the service board 1_3 After receiving the data packet, the data plane of the service board 1_3 first searches for the corresponding routing entry according to the destination IP address of the data packet, and then searches for the corresponding software according to the source IP address PC-IP of the data packet.
  • the user entry and the hardware user entry find the software user entry entry3_2 as shown in Table 3 and the hardware user entry entry3_3 as shown in Table 4.
  • the entry of the hardware user entry entry3_3 for example, statistics traffic and duration, user policy control, etc., and then forwarding the data packet according to the found routing entry, thereby finally reaching the Internet, for example, the data
  • the packet is forwarded to the egress service board 1_4 and forwarded to the Internet by the egress service board 1_4.
  • the data packet forwarding process of data packets from the Internet to the user host is as follows:
  • the service board 1_4 receives the data packet sent to the user host, and then searches for the corresponding routing entry according to the destination IP address PC-IP of the data packet, and finds the routing entry as shown in Table 5, and the routing entry.
  • the DDev is the RG, which is the resource group.
  • the DPort in the routing entry is Slot2_3. Then, according to the destination IP address PC-IP of the data packet, the corresponding software user entry and hardware user entry are searched. If the data packet is not found, the data packet is forwarded to the service board 1_3 through the switching network according to the Slot 2_3 of the DPort.
  • the service board 1_3 After receiving the data packet, the service board 1_3 searches for the corresponding routing entry according to the destination IP address PC-IP of the data packet, and finds the routing entry as shown in Table 5, in the routing entry.
  • DDev is the RG, which is the resource group.
  • the DPort in the routing entry is Slot2_3. Then, according to the destination IP address PC-IP of the data packet, the corresponding software user entry and hardware user entry are searched.
  • the user entry entry3_2 and the hardware user entry entry3_3 are processed according to the hardware user entry entry entry3_3, for example, statistics traffic and duration, user policy control, etc., and then the data is according to Slot2_1 recorded in the software user entry entry entry3_2.
  • the packet is forwarded to the service board 1_1 through the switching network.
  • the service board 1_1 may forward the data packet according to the method shown in FIG. 3, or forward the data packet according to the method shown in FIG. The data packet is forwarded to the user host.
  • the routing table entry including the PC-IP on the data plane of the service board 1_1 is as shown in Table 5, so that the service board 1_1 receives the packet addressed to the user host. After the data packet, you need to perform the steps shown in Figure 3:
  • Step S301 Searching for the corresponding routing entry according to the destination IP address PC-IP of the data packet, and searching for the routing entry as shown in Table 5, where the DDev in the routing entry is RG, that is, the resource group, The DPort in the routing entry is Slot2_3.
  • Step S302 Search for the corresponding software user entry and hardware user entry according to the destination IP address PC-IP of the data packet, and find the software user entry entry3_1 as shown in Table 2-3, but the search is not found.
  • Step S303 Forward the data packet to the user host according to the original port ID PortA recorded in the entry 3_1 of the software user entry.
  • the routing table entry containing the PC-IP on the data plane of the service board 1_1 is as shown in Table 6 when the routing entry is sent in the second mode. Therefore, the service board 1_1 receives the message to the user. After the data packet of the host, you need to perform the steps shown in Figure 4:
  • Step S401 Find the routing entry according to the destination IP address PC-IP of the data packet, and find the routing entry as shown in Table 6.
  • the DDev in the routing entry is the slot ID of the service board.
  • Slot2_1 the DPort in the routing entry is PortA.
  • Step S402 Search for the corresponding software user entry and hardware user entry according to the destination IP address PC-IP of the data packet, and find the software user entry entry3_1 as shown in Table 2-3, but the search is not found.
  • Step S403 Forward the data packet to the user host according to the PortA that is the DPort in the found routing entry.
  • the service board 1_3 Since the hardware user entry corresponding to the user accessing the service board 1_1 is on the service board 1_3, the service board 1_3 is issued in order to perform traffic statistics and the like on the data packet sent to the user host according to the hardware user entry. You can set the destination device ID to the ID of the resource group, set the destination port ID to the slot ID of the service board 1_3, and synchronize the routing entry to other service boards to make the other host IP routing entries.
  • the service board sends the routing entry. In this way, the service board that first receives the data packet sent from the Internet to the user host can forward the data packet to the service board 1_3 through the lookup route, and the service board 1_3 sends the data to the user host according to the hardware user entry. The packet is forwarded to the service board 1_1 and then forwarded to the user host by the service board 1_1.
  • the present application also provides an embodiment of the apparatus.
  • FIG. 5 is a structural diagram of a distributed access gateway device according to an embodiment of the present application.
  • the gateway device includes multiple service boards, a first service board to an Nth service board, and the first service board 510 is any one of the service boards.
  • the first business board 510 can include a processor 511 and a machine readable storage medium 512.
  • processor 511 and machine readable storage medium 62 can communicate via system bus 513.
  • the processor 511 can perform the method of address allocation described above.
  • the machine-readable storage medium 512 referred to herein can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like.
  • the machine readable storage medium 512 may be a RAM (Random Access Memory), a volatile memory, a nonvolatile memory, a flash memory, a storage drive (such as a hard disk drive), a solid state drive, any type of A storage disk (such as a compact disc, a DVD, etc.), or a similar storage medium, or a combination thereof.
  • the resource scheduling logic 60 of the embodiment of the present application includes the following units: a determining unit 601, an obtaining unit 602, and a sending unit 603, where:
  • the determining unit 601 is configured to: after the first user is authenticated, create a first software user entry corresponding to the first user on a data plane of the first service board 510 in the distributed access gateway device, and determine the location Whether there is idle hardware resources on the first service board 510;
  • the obtaining unit 602 is configured to: obtain, if not existing, the slot identifier ID of the second service board 520 that has the idle hardware resource on the data plane, where the second service board 520 is selected from the resource group.
  • the resource group includes two or more service boards on the distributed access gateway device;
  • the sending unit 603 sends the first software user entry to the second service board 520 according to the obtained slot ID of the second service board 520 on the data plane, so that the second service board 520: Create a first hardware user entry corresponding to the first user on a data plane.
  • the resource scheduling logic 60 may further include: an updating unit 604 and a first receiving unit 605, where:
  • the updating unit 604 is configured to record, in the first software user entry, on the data plane when determining that the second service board 520 successfully creates the first hardware user entry corresponding to the first user The slot ID of the second service board 520;
  • the first receiving unit 605 is configured to receive a data packet from the user host.
  • the sending unit 603 is further configured to: if the received data packet matches the first software user entry, the first service board is configured on the data plane according to the record in the first software user entry.
  • the slot ID of the second service board 520 forwards the data packet to the second service board 520.
  • the resource scheduling logic 60 may further include: a second receiving unit 606 and a creating unit 607, where:
  • the second receiving unit 606 is configured to receive a software user entry corresponding to the second user sent by the third service board 530.
  • a creating unit 607 configured to create a second software user entry and a second hardware user entry corresponding to the second user according to the received software user entry, and record the second software user entry in the second software user entry
  • the slot ID of the third service board 530 configured to create a second software user entry and a second hardware user entry corresponding to the second user according to the received software user entry, and record the second software user entry in the second software user entry The slot ID of the third service board 530.
  • the creating unit 607 is further configured to: after the second software user entry and the second hardware user entry are created, create a first route that includes the host IP address of the second user on the data plane.
  • the sending unit 603 is further configured to synchronize the first routing entry to another service board in the resource group.
  • the resource scheduling logic 60 may also include a saving unit 608. among them:
  • the second receiving unit 606 is further configured to receive a second routing entry that is synchronized by the fourth service board.
  • the saving unit 608 is configured to save the routing entry received by the second receiving unit 606.
  • the port ID of the host connected to the first user on the service board is also recorded in the first software user entry.
  • the resource scheduling apparatus 60 may further include: a saving unit 608. among them:
  • the second receiving unit 606 is further configured to receive a second routing entry that is synchronized by the fourth service board.
  • the saving unit 608 is configured to search for a corresponding software user entry according to the IP address included in the second routing entry on the data plane; if the software user entry corresponding to the IP address is not found, the data plane is saved on the data plane.
  • the slot ID of the first service board is changed to the port ID of the host connected to the first user on the first service board recorded in the first software user entry corresponding to the IP address.
  • the resource scheduling logic 60 may further include: a searching unit 609, where:
  • the second receiving unit 606 is further configured to receive a data packet sent to the user host.
  • the searching unit 609 is configured to search for a corresponding third routing entry according to the destination IP address of the received data packet, where the destination device ID in the third routing entry is the ID of the resource group and the destination port ID is a slot. a bit ID; searching for a corresponding software user entry and a hardware user entry according to the destination IP address of the data packet on the data plane;
  • the sending unit 603 is further configured to: if not found, the data packet is forwarded to the corresponding slot according to the slot ID of the destination port ID in the third routing entry on the data plane of the first service board. If the second software user entry and the second hardware user entry are found, the data is processed according to the second hardware user entry on the data plane, and then according to the second software.
  • the slot ID of the third service board 520 recorded in the user entry is forwarded to the third service board; if the first software user entry is found, the hardware user is not found.
  • the data item is forwarded to the first user on the data plane according to the port ID of the host connected to the first user on the first service board recorded in the first software user entry. Host.
  • the first receiving unit 605 and the second receiving unit 606 may be two independent units, which may be implemented by being integrated into one unit, which is not limited in this embodiment of the present application.
  • the device embodiment since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without any creative effort.

Abstract

本申请提供资源调度方法、分布式接入网关设备和机器可读存储介质，其中，分布式接入网关设备中的第一业务板在数据平面上创建与第一用户对应的第一软件用户表项，判断第一业务板上是否存在空闲硬件资源；若不存在，则获取存在空闲硬件资源的第二业务板的槽位标识ID；根据获取到的第二业务板的槽位ID将第一软件用户表项发送给第二业务板，以使第二业务板在数据平面上创建与第一用户对应的硬件用户表项。

Description

资源调度
相关申请的交叉引用
本专利申请要求于2016年12月21日提交的、申请号为201611192753.2、发明名称为“资源调度方法及装置”的中国专利申请的优先权，该申请的全文以引用的方式并入本文中。
背景技术
BRAS(Broadband Remote Access Server，宽带远程接入服务器)业务系统主要由用户主机、BRAS设备、以及AAA(Authentication Authorization and Accounting，认证授权计费)服务器组成。BRAS设备将用户主机发来的用户信息发送给AAA服务器，从而完成对用户的认证、授权和计费。在用户认证通过后，用户主机即可通过BRAS设备访问外部网络，例如互联网。
附图说明
图1是本申请一示例性实施例示出的BRAS设备的架构示意图；
图2是本申请一示例性实施例示出的资源调度方法的流程图；
图3是本申请一示例性实施例示出的对发往用户主机的数据报文进行转发处理的一种流程图；
图4是本申请一示例性实施例示出的发往用户主机的数据报文进行转发处理的另一种流程图；
图5是本申请一示例性实施例示出的分布式接入网关设备的硬件结构示意图；
图6是本申请一示例性实施例示出的资源调度逻辑的一种结构示意图；
图7是本申请一示例性实施例示出的资源调度逻辑的另一种结构示意图；
图8是本申请一示例性实施例示出的资源调度逻辑的又一种结构示意图；
图9是本申请一示例性实施例示出的资源调度逻辑的又一种结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
如图1所示，采用分布式硬件架构的BRAS设备中包括：主控板100和多个业务板1_1～1_5，主控板100负责控制和管理所有业务板，以及全局信息的同步；业务板负责接入认证和报文转发处理。其中，业务板进行接入认证和报文转发处理的过程如下：
数据平面接收用户主机发出的接入认证请求报文，识别出该报文为协议报文，则将该报文上送给控制平面进行处理；
控制平面接收到该接入认证请求报文后，通过与AAA服务器交互完成对用户的认证，用户认证通过后，控制平面建立该用户对应的软件用户表项，该表项可记录在内存中；控制平面通知数据平面创建用户表项；
数据平面接收到该通知后，创建该用户对应的软件用户表项和硬件用户表项，该硬件用户表项通常是记录在TCAM(Ternary Content Addressable Memory，三态内容寻址存储器)等器件上；
数据平面接收用户主机在认证通过后发出的数据报文，识别出该报文为非协议报文，则查找对应的硬件用户表项，在查找到了硬件用户表项后，根据该硬件用户表项进行相应处理，例如，统计流量和时长、进行用户策略控制等，然后，通过路由转发该数据报文。
其中，上述控制平面具体可以是CPU(Central Processing Unit，中央处理单元)，上述数据平面具体可以是驱动程序。
在一示例中，每个业务板的硬件资源规格是固定的，例如，可支持16K个用户接入，则此时可配置的硬件用户表项总数就是16K，如果某一业务板上的硬件资源已经全部被占用，那么新增用户的硬件用户表项就会因为硬件资源不足而无法下发，从而导致该新增用户无法上线。
为了解决上述问题，本申请以下实施例中提供了一种资源调度方法和分布式接入网关设备。该方法可以由如图1所示的分布式接入网关设备中的第一业务板来执行，例如，第一业 务板的数据平面来执行，该方法包括：在第一用户认证通过后，创建与第一用户对应的第一软件用户表项，判断本第一业务板上是否存在空闲硬件资源；若不存在，则获取存在空闲硬件资源的第二业务板的槽位ID(Identity，标识)，其中，第二业务板是从资源组中选择出来的，该资源组由两个以上业务板组成；根据获取到的槽位ID将第一软件用户表项发送给第二业务板，以使第二业务板创建与第一用户对应的硬件用户表项。
上述方法中，在第一业务板上，当数据平面创建了与某一用户对应的软件用户表项，需要创建与该用户对应的硬件用户表项时，如果判断出本第一业务板上已经没有空闲的硬件资源，则获取资源组中存在空闲的硬件资源的第二业务板的槽位ID，根据获取到的槽位ID将创建的软件用户表项发送给该第二业务板，由该第二业务板创建与该用户对应的硬件用户表项。从而，在第一业务板上的硬件资源不足时，可以将新增用户的硬件用户表项配置到第二业务板上，由该第二业务板共享硬件资源给该硬件资源不足的第一业务板，从而确保该新增用户可以上线，实现了分布式网关设备的硬件资源的合理利用和调度。
以如图1所示的分布式BRAS设备为例进行说明，该分布式BRAS设备中包括：主控板100和多个业务板。在一示例中，分布式BRAS设备中存在支持BRAS功能的业务板，也存在不支持BRAS功能的业务板，本申请实施例仅考虑支持BRAS功能的业务板，即，下文中所说的业务板均为支持BRAS功能的业务板。
主控板上维护一个资源组，该资源组由分布式BRAS设备中的两个以上支持BRAS功能的业务板组成。具体的，主控板记录资源组信息，包括：该资源组中每一个业务板的槽位ID、硬件资源规格和硬件资源使用情况。其中，硬件资源使用情况可以通过空闲硬件资源的数量来表示。
假设，该资源组中包括图1中的5个业务板：槽位ID为Slot2_1的业务板1_1、槽位ID为Slot2_2的业务板1_2、槽位ID为Slot2_3的业务板1_3、槽位ID为Slot2_4的业务板1_4、以及槽位ID为Slot2_5的业务板1_5。则，主控板上记录的资源组信息如表1所示：
表1
槽位ID 硬件资源规格(K) 空闲硬件资源(K)
Slot2_1 32 20
Slot2_2 16 0.1
Slot2_3 16 15
Slot2_4 32 10
Slot2_5 16 7
资源组中的每一个业务板会在本业务板的硬件资源使用情况发生变化时通知主控板，以使主控板实时更新表1中该业务板的空闲硬件资源。
主控板会在表1发生更新时，例如，新增或减少了业务板，某一业务板的空闲硬件资源的数量发生变化，将更新后的信息同步给资源组中的每一个业务板，以确保业务板和主控板上的资源组信息的一致性。此时，资源组中各个业务板的控制平面上也维护了如表1所示的资源组信息。第一业务板的控制平面在接收到数据平面发来的用于指示硬件资源不足的通知消息后，根据该资源信息，从资源组中选择一个存在空闲硬件资源的第二业务板，将该第二业务板的槽位ID发送给数据平面。
或者，第一业务板的控制平面在接收到数据平面发来的用于指示硬件资源不足的通知消息后，向主控板获取资源组信息，根据获取到的资源组信息，从资源组中选择一个存在空闲硬件资源的第二业务板，将该第二业务板的槽位ID发送给数据平面。
或者，第一业务板的控制平面在接收到数据平面发来的用于指示硬件资源不足的通知消息后，通知主控板根据资源组信息从资源组中选择一个存在空闲硬件资源的第二业务板，并回复该第二业务板的槽位ID，控制平面获取到该第二业务板的槽位ID后，将该槽位ID发送给数据平面。
其中，可以采用手动静态配置的方式将业务板加入到资源组中。采用手动静态配置的方式时，可以由管理人员有选择性的将业务板加入到资源组中，例如，某业务板虽然支持BRAS功能，但是该业务板的硬件资源规格很低，而且该业务板主要应用于实现其他功能，则可以不将该业务板加入到资源组中。
另外，也可以采用动态方式，将业务板加入到资源组中。具体的，当新插入的业务板启动完成后，主控板识别该业务板的类型，在确定出该业务板支持BRAS功能时，将该业务板加入到资源组中，在表1中添加该业务板的槽位ID、硬件资源规格和空闲硬件资源。采用此种方式，无需管理人员手动配置，可以由主控板自动将业务板加入资源组。
基于此，本申请实施例的资源调度方法包括以下内容：
业务板1_1的数据平面接收到用户主机发出的接入认证请求报文之后，识别出该报文为协议报文，则将该报文上送给控制平面进行处理；控制平面接收到该接入认证请求报文后，通过与AAA服务器交互完成对用户的认证，用户认证通过后，控制平面建立与该用户对应的软件用户表项，并通知数据平面创建用户表项。
业务板1_1的数据平面接收到控制平面发来的创建用户表项通知后，会执行如图2所示的步骤：
步骤S201，创建与该用户对应的软件用户表项，为了描述方便，将该表项记为entry3_1；
entry3_1可以存放在数据平面的内存中，如表2-1所示，entry3_1中可以包括：用户名、用户主机的MAC地址和IP地址、限速信息、用户主机接入的原始业务板的槽位ID、该业务板上连接该用户主机的端口(即原始端口)ID等。
表2-1
用户名 MAC地址 IP地址 限速信息 原始业务板 原始端口 …
User PC-MAC PC-IP XXX Slot2_1 PortA …
步骤S202，判断本业务板上是否存在空闲硬件资源，若是，则执行步骤S203，否则，执行步骤S204；
步骤S203，创建与该用户对应的硬件用户表项，之后退出本流程；
步骤S204，向控制平面发送用于指示硬件资源不足的通知消息；
在判断出本业务板1_1上不存在空闲硬件资源时，还可以在如表2-1所示的软件用户表项entry3_1中添加一个标记，用于指示本业务板上不存在空闲硬件资源、需要重定向到其它业务板，例如，该标记以NoResRedi表示。此时，如表2-1所示的entry3_1更新为如表2-2所示：
表2-2
用户名 MAC地址 IP地址 限速信息 原始业务板 原始端口 … 标记
User PC-MAC PC-IP XXX Slot2_1 PortA … NoResRedi
在本地保存有如表1所示的资源组信息时，业务板1_1的控制平面接收到该通知消息后，根据该资源组信息从资源组中选择一个存在空闲硬件资源的业务板，例如，选择硬件资源空闲率最高的槽位ID为Slot2_3的业务板1_3，然后，将Slot2_3发送给数据平面。
或者，在本地未保存资源组信息时，业务板1_1的控制平面接收到该通知消息后，向主控板获取资源组信息，根据获取到的如表1所示的资源组信息，从资源组中选择一个存在空 闲硬件资源的业务板，例如，选择硬件资源空闲率最高的槽位ID为Slot2_3的业务板1_3，然后，将Slot2_3发送给数据平面。
或者，在本地未保存资源组信息时，业务板1_1的控制平面接收到该通知消息后，通知主控板根据如表1所示的资源组信息，从资源组中选择一个存在空闲硬件资源的业务板，例如，选择硬件资源空闲率最高的槽位ID为Slot2_3的业务板1_3，并将Slot2_3回复给业务板1_1的控制平面，业务板1_1的控制平面接收到Slot2_3后，将Slot2_3发送给数据平面。
步骤S205，接收控制平面发来的Slot2_3，将创建的软件用户表项entry3_1发送给业务板1_3；
具体的，业务板1_1的数据平面可以通过板间通信消息将entry3_1的内容发送给业务板1_3，其中，该板间通信消息例如可以是IPC(Inter-Process Communication，进程间通信)消息。
业务板1_3的数据平面接收到该板间通信消息后，根据该消息中携带的软件用户表项entry3_1的内容，创建与该用户对应的软件用户表项entry3_2和硬件用户表项entry3_3，并在entry3_2中记录业务板1_1的槽位ID Slot2_1，用于表示该用户接入的原始业务板为业务板1_1。entry3_2和entry3_3的内容分别如表3和表4所示：
表3
Figure PCTCN2017117507-appb-000001
表4
用户名 MAC地址 IP地址 限速信息 …
User PC-MAC PC-IP XXX …
其中，如表3所示的软件用户表项entry3_2通常存放在数据平面的内存中，entry3_2中主要包括：用户名、用户主机MAC地址和IP地址、限速信息、对应的硬件用户表项entry3_3的索引、以及用户接入的原始业务板的槽位ID等；如表4所示的硬件用户表项entry3_3可存放在TCAM等器件中，entry3_3中主要包括：用户名、用户主机MAC地址和IP地址、限速 信息、用户所属分组ID和所属域名、对应的用户策略ID等。
业务板1_3的数据平面在创建了entry3_2和entry3_3之后，可以向业务板1_1回复一个用于指示硬件用户表项创建成功的板间通信消息。
步骤S206，在接收到业务板1_3发来的用于指示硬件用户表项创建成功的板间通信消息之后，在如表2-2所示的软件用户表项entry3_1中记录业务板1_3的槽位ID Slot2_3，用于表示该用户被重定向到了业务板1_3。此时，entry3_1更新为如表2-3所示：
表2-3
Figure PCTCN2017117507-appb-000002
至此，在业务板1_1上不存在空闲硬件资源时，接入业务板1_1的用户被重定向到了业务板1_3上，由业务板1_3创建与该用户对应的硬件用户表项，保证了该用户的上线，实现了资源组中各个业务板的硬件资源的整合、共享和调度。
另外，业务板1_3的数据平面在创建了软件用户表项entry3_2和硬件用户表项entry3_3之后，还会创建包含用户主机IP地址的路由表项，该路由表项可以参见表5所示：
表5
IP地址 DDev(目的设备) DPort(目的端口)
PC-IP RG Slot2_3
由表5可以看出，该路由表项中的目的设备ID为资源组的ID，记为RG，目的端口ID为业务板1_3的槽位ID Slot2_3。业务板1_3还会将如表5所示的路由表项通过主控板同步给其它业务板，包括业务板1_1、业务板1_2、业务板1_4、业务板1_5。
业务板1_1、业务板1_2、业务板1_4、业务板1_5中的每一个业务板在接收到业务板1_3同步来的如表5所示的路由表项后，会按照以下两种方式中的任一种下发路由表项：
方式一：直接在数据平面上保存该路由表项。
如果采用方式一，则业务板1_1、业务板1_2、业务板1_4、业务板1_5的数据平面上的包含PC-IP的路由表项均如表5所示。
方式二：根据该路由表项中包含的IP地址PC-IP查找对应的软件用户表项；若没有查找到，说明IP地址为PC-IP的用户主机不是接入本业务板的用户主机，则直接在数据平面上保存该路由表项；若查找到了，说明IP地址为PC-IP的用户主机是接入本业务板的用户主机，则保存该路由表项，并将保存的路由表项的DDev修改为本业务板的槽位ID，将DPort修改为查找到的软件用户表项中记录的原始端口ID。
采用方式二时，业务板1_1由于查找到了如表2-3所示的软件用户表项，因此，业务板1_1的数据平面上包含PC-IP的路由表项如表6所示；而业务板1_2、业务板1_4、业务板1_5由于没有查找到对应的软件用户表项，因此，这些业务板的数据平面上包含PC-IP的路由表项均如表5所示。
表6
IP地址 DDev DPort
PC-IP Slot2_1 PortA
由此可见，无论采用方式一还是方式二，业务板1_2、业务板1_4、业务板1_5的数据平面上的包含PC-IP的路由表项均如表5所示。然而，对于业务板1_1来说，采用方式一时，数据平面上包含PC-IP的路由表项如表5所示，而采用方式二时，数据平面上包含PC-IP的路由表项如表6所示，即，采用不同方式时的路由表项不同，这样，对于发往用户主机的数据报文的转发处理也不同，其中，当采用方式一时的转发处理可以参见后文中如图3所示的方法，当采用方式二时的转发处理可以参见后文中如图4所示的方法。
后续，用户从主机到互联网方向的数据报文转发过程如下：
在认证通过后，用户主机会发出源IP地址为PC-IP的数据报文以访问互联网，业务板1_1的数据平面接收到该数据报文后，识别出该报文为非协议报文，则先根据该数据报文的目的IP地址，查找对应的路由表项，然后，根据该数据报文的源IP地址PC-IP查找对应的软件用户表项和硬件用户表项，结果查找到了如表2-3所示的软件用户表项entry3_1，但是没有查找到对应的硬件用户表项。此时，由于entry3_1中标记了NoResRedi，因此，可以确定该用户被重定向到了其它业务板，根据entry3_1中记录的Slot2_3，将该数据报文通过交换网发送给业务板1_3。
业务板1_3的数据平面接收到该数据报文后，先根据该数据报文的目的IP地址，查找对应的路由表项，然后，根据该数据报文的源IP地址PC-IP查找对应的软件用户表项和硬件用 户表项，结果查找到了如表3所示的软件用户表项entry3_2和如表4所示的硬件用户表项entry3_3。根据硬件用户表项entry3_3进行相应操作，例如，统计流量和时长、进行用户策略控制等，然后，根据查找到的路由表项将该数据报文转发出去，从而最终到达互联网，例如，将该数据报文转发给出口业务板1_4，由出口业务板1_4转发到互联网。
从互联网到用户主机方向的数据报文的数据报文转发过程如下：
业务板1_4接收发往用户主机的数据报文，先根据该数据报文的目的IP地址PC-IP查找对应的路由表项，结果查找到如表5所示的路由表项，该路由表项中的DDev为RG，即资源组，该路由表项中的DPort为Slot2_3；然后，根据该数据报文的目的IP地址PC-IP，查找对应的软件用户表项和硬件用户表项，结果均没有查找到，则根据查找到的路由表项中作为DPort的Slot2_3，将该数据报文通过交换网转发给业务板1_3。
业务板1_3接收到该数据报文后，先根据该数据报文的目的IP地址PC-IP查找对应的路由表项，结果查找到如表5所示的路由表项，该路由表项中的DDev为RG，即资源组，该路由表项中的DPort为Slot2_3；然后，根据该数据报文的目的IP地址PC-IP，查找对应的软件用户表项和硬件用户表项，结果查找到软件用户表项entry3_2和硬件用户表项entry3_3，则根据硬件用户表项entry3_3进行相应处理，例如，统计流量和时长、进行用户策略控制等，之后根据软件用户表项entry3_2中记录的Slot2_1，将该数据报文通过交换网转发给业务板1_1。
业务板1_1接收到该数据报文后，可以按照如图3所示的方法对该数据报文进行转发，也可以按照如图4所示的方法对该数据报文进行转发，从而最终将该数据报文转发给用户主机。
一种情况下，由于在采用方式一下发路由表项时，业务板1_1的数据平面上包含PC-IP的路由表项如表5所示，因此，业务板1_1在接收到发往用户主机的数据报文后，需要执行如图3所示的步骤：
步骤S301，根据该数据报文的目的IP地址PC-IP查找对应的路由表项，结果查找到如表5所示的路由表项，该路由表项中的DDev为RG，即资源组，该路由表项中的DPort为Slot2_3；
步骤S302，根据该数据报文的目的IP地址PC-IP，查找对应的软件用户表项和硬件用户表项，结果查找到如表2-3所示的软件用户表项entry3_1，但是没有查找到对应的硬件用户表项；
步骤S303，根据软件用户表项entry3_1中记录的原始端口ID PortA，将该数据报文转发 给用户主机。
另一种情况下，由于在采用方式二下发路由表项时，业务板1_1的数据平面上包含PC-IP的路由表项如表6所示，因此，业务板1_1在接收到发往用户主机的数据报文后，需要执行如图4所示的步骤：
步骤S401，根据该数据报文的目的IP地址PC-IP查找对应的路由表项，结果查找到如表6所示的路由表项，该路由表项中的DDev为本业务板的槽位ID Slot2_1，该路由表项中的DPort为PortA；
步骤S402，根据该数据报文的目的IP地址PC-IP，查找对应的软件用户表项和硬件用户表项，结果查找到如表2-3所示的软件用户表项entry3_1，但是没有查找到对应的硬件用户表项；
步骤S403，根据查找到的路由表项中作为DPort的PortA，将该数据报文转发给用户主机。
由于接入业务板1_1的用户对应的硬件用户表项在业务板1_3上，因此，为了能够根据该硬件用户表项对发往用户主机的数据报文进行流量统计等处理，业务板1_3在下发用户主机IP路由表项时，可将目的设备ID设置为资源组的ID、将目的端口ID设置为本业务板1_3的槽位ID，并将该路由表项同步给其它业务板，以使其它业务板下发该路由表项。这样，最先接收到互联网发往用户主机的数据报文的业务板，可以通过查找路由将该数据报文转发给业务板1_3，由业务板1_3根据硬件用户表项对发往用户主机的数据报文进行流量统计等处理后，转发给业务板1_1，最终由业务板1_1转发给用户主机。
与前述资源调度方法的实施例相对应，本申请还提供了装置的实施例。
参见图5，图5为本申请实施例提供的分布式接入网关设备的结构图。如图5所示，该网关设备包括多个业务板，第一业务板至第N业务板，其中，第一业务板510为其中任一业务板。第一业务板510可包括处理器511以及机器可读存储介质512。其中，处理器511和机器可读存储介质62可经由系统总线513通信。并且，通过读取并执行机器可读存储介质512中存储的与资源调度逻辑60对应的机器可执行指令，处理器511可执行上文所述的地址分配的方法。
本文提到的机器可读存储介质512可以是任何电子、磁性、光学或其他物理存储装置，可以包含或存储信息，如可执行指令、数据，等等。例如，所述机器可读存储介质512可以是RAM(Random Access Memory，随机存取存储器)、易失存储器、非易失性存储器、闪存、存储驱动器(如硬盘驱动器)、固态硬盘、任何类型的存储盘(如光盘、DVD等)，或 者类似的存储介质，或者它们的组合。
请参考图6，本申请实施例的资源调度逻辑60中包括以下单元：判断单元601、获取单元602和发送单元603，其中：
判断单元601，用于在第一用户认证通过后，在分布式接入网关设备中的第一业务板510的数据平面上创建与所述第一用户对应的第一软件用户表项，判断所述第一业务板510上是否存在空闲硬件资源；
获取单元602，用于若不存在，在数据平面上获取存在空闲硬件资源的第二业务板520的槽位标识ID，其中，所述第二业务板520是从资源组中选择出来的，所述资源组包括所述分布式接入网关设备上的两个以上业务板；
发送单元603，在数据平面上根据获取到的所述第二业务板520的槽位ID将所述第一软件用户表项发送给所述第二业务板520，以使所述第二业务板520在数据平面上创建与所述第一用户对应的第一硬件用户表项。
如图7所示，资源调度逻辑60中还可以包括：更新单元604和第一接收单元605，其中：
更新单元604，用于在确定所述第二业务板520成功创建了与所述第一用户对应的第一硬件用户表项时，在数据平面上在所述第一软件用户表项中记录所述第二业务板520的槽位ID；
第一接收单元605，用于接收来自用户主机的数据报文；
发送单元603，还用于若接收到的数据报文与所述第一软件用户表项匹配，则所述第一业务板在数据平面上根据所述第一软件用户表项中记录的所述第二业务板520的槽位ID，将所述数据报文转发给所述第二业务板520。
如图8所示，资源调度逻辑60中还可以包括：第二接收单元606和创建单元607，其中：
第二接收单元606，用于接收第三业务板530发来的与第二用户对应的软件用户表项；
创建单元607，用于根据接收到的软件用户表项，创建与所述第二用户对应的第二软件用户表项和第二硬件用户表项，并在所述第二软件用户表项中记录所述第三业务板530的槽位ID。
其中，创建单元607，还用于在创建了所述第二软件用户表项和所述第二硬件用户表项之后，在数据平面上创建包含所述第二用户的主机IP地址的第一路由表项，其中，所述第一路由表项中的目的设备ID为所述资源组的ID、目的端口ID为本第一业务板510的槽位ID；
发送单元603，还用于将所述第一路由表项同步给所述资源组中的其它业务板。
在一示例中，资源调度逻辑60中还可以包括保存单元608。其中：
第二接收单元606，还用于接收第四业务板同步过来的第二路由表项。
保存单元608，用于对第二接收单元606接收到的路由表项进行保存。
在一示例中，第一软件用户表项中还记录有本业务板上连接第一用户的主机的端口ID，则，如图9所示，资源调度装置60中还可以包括：保存单元608，其中：
第二接收单元606，还用于接收第四业务板同步过来的第二路由表项；
保存单元608，用于在数据平面上根据该第二路由表项中包含的IP地址查找对应的软件用户表项；若没有查找到该IP地址对应的软件用户表项，在数据平面上则保存所述第二路由表项；若查找到了所述第一软件用户表项，在数据平面上则保存所述第二路由表项，并所述第二路由表项中的目的设备ID修改为本第一业务板的槽位ID，将目的端口ID修改为该IP地址对应的所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID。
如图9所示，资源调度逻辑60中还可以包括：查找单元609，其中：
第二接收单元606，还用于接收发往用户主机的数据报文；
查找单元609，用于根据接收到的数据报文的目的IP地址查找对应的第三路由表项，该第三路由表项中的目的设备ID为所述资源组的ID、目的端口ID为槽位ID；在数据平面上根据该数据报文的目的IP地址查找对应的软件用户表项和硬件用户表项；
发送单元603，还用于若均没有查找到，在第一业务板的数据平面上根据该第三路由表项中作为目的端口ID的槽位ID，将该数据报文转发给对应槽位上的业务板；若查找到了所述第二软件用户表项和所述第二硬件用户表项，在数据平面上则根据所述第二硬件用户表项进行相应处理，之后根据所述第二软件用户表项中记录的所述第三业务板520的槽位ID，将该数据报文转发给所述第三业务板；若查找到了所述第一软件用户表项、但没有查找到硬件用户表项，在数据平面上则根据所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID，将该数据报文转发给所述第一用户的主机。
在上述装置实施例中，第一接收单元605和第二接收单元606可以是两个独立的单元，也可以集成于一个单元来实现，本申请实施例对此不做限定。
上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上对本发明实施例所提供的方法和装置进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (15)

  1. 一种资源调度方法，包括：
    在第一用户认证通过后，分布式接入网关设备中的第一业务板在数据平面上创建与所述第一用户对应的第一软件用户表项，判断所述第一业务板上是否存在空闲硬件资源；
    若不存在，所述第一业务板在数据平面上则获取存在空闲硬件资源的第二业务板的槽位标识ID，其中，所述第二业务板是从资源组中选择出来的，所述资源组包括所述分布式接入网关设备上的两个以上业务板；
    所述第一业务板在数据平面上根据获取到的所述第二业务板的槽位ID将所述第一软件用户表项发送给所述第二业务板，以使所述第二业务板在数据平面上创建与所述第一用户对应的第一硬件用户表项。
  2. 根据权利要求1所述的方法，还包括：
    在确定所述第二业务板成功创建了与所述第一用户对应的第一硬件用户表项时，所述第一业务板在数据平面上在所述第一软件用户表项中记录所述第二业务板的槽位ID；
    所述第一业务板在数据平面上接收来自用户主机的数据报文，若接收到的数据报文与所述第一软件用户表项匹配，则所述第一业务板在数据平面上根据所述第一软件用户表项中记录的所述第二业务板的槽位ID，将所述数据报文转发给所述第二业务板。
  3. 根据权利要求1所述的方法，还包括：
    所述第一业务板的数据平面接收第三业务板发来的与第二用户对应的软件用户表项，根据接收到的软件用户表项，创建与所述第二用户对应的第二软件用户表项和第二硬件用户表项，并在所述第二软件用户表项中记录所述第三业务板的槽位ID。
  4. 根据权利要求3所述的方法，还包括：
    在创建了所述第二软件用户表项和所述第二硬件用户表项之后，所述第一业务板在数据平面上创建包含所述第二用户的主机IP地址的第一路由表项，其中，所述第一路由表项中的目的设备ID为所述资源组的ID、目的端口ID为本第一业务板的槽位ID；
    所述第一业务板在数据平面上将所述第一路由表项同步给所述资源组中的其它业务板。
  5. 根据权利要求4所述的方法，还包括：
    所述第一业务板在数据平面上接收第四业务板同步过来的第二路由表项并进行保存。
  6. 根据权利要求4所述的方法，其中，所述第一软件用户表项中还记录有本第一业务板上连接所述第一用户的主机的端口ID，所述方法还包括：
    所述第一业务板在数据平面上接收第四业务板同步过来的第二路由表项；
    所述第一业务板在数据平面上根据该第二路由表项中包含的IP地址查找对应的软件用户表项；
    若没有查找到该IP地址对应的软件用户表项，所述第一业务板在数据平面上则保存所述第二路由表项；
    若查找到了所述第一软件用户表项，所述第一业务板在数据平面上则保存所述第二路由表项，并将所述第二路由表项中的目的设备ID修改为本第一业务板的槽位ID，将目的端口ID修改为该IP地址对应的所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID。
  7. 根据权利要求5所述的方法，还包括：
    所述第一业务板在数据平面上接收发往用户主机的数据报文，根据接收到的数据报文的目的IP地址查找对应的第三路由表项，该第三路由表项中的目的设备ID为所述资源组的ID、目的端口ID为槽位ID；
    所述第一业务板在数据平面上根据该数据报文的目的IP地址查找对应的软件用户表项和硬件用户表项；
    若均没有查找到，所述第一业务板在数据平面上则根据该第三路由表项中作为目的端口ID的槽位ID，将该数据报文转发给对应槽位上的业务板；
    若查找到了所述第二软件用户表项和所述第二硬件用户表项，所述第一业务板在数据平面上则根据所述第二硬件用户表项进行相应处理，之后根据所述第二软件用户表项中记录的所述第三业务板的槽位ID，将该数据报文转发给所述第三业务板；
    若查找到了所述第一软件用户表项、但没有查找到硬件用户表项，所述第一业务板在数据平面上则根据所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID，将该数据报文转发给所述第一用户的主机。
  8. 一种分布式接入网关设备，包括：
    多个业务板，其中，
    第一业务板，包括：处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使：
    在第一用户认证通过后，在数据平面上创建与所述第一用户对应的第一软件用户表项，判断所述第一业务板上是否存在空闲硬件资源；
    若不存在，在数据平面上则获取存在空闲硬件资源的第二业务板的槽位标识ID，其中，所述第二业务板是从资源组中选择出来的，所述资源组包括所述分布式接入网关设备上的两 个以上业务板；
    在数据平面上根据获取到的所述第二业务板的槽位ID将所述第一软件用户表项发送给所述第二业务板，以使所述第二业务板在数据平面上创建与所述第一用户对应的第一硬件用户表项。
  9. 根据权利要求8所述的网关设备，其中，所述处理器进一步被所述机器可执行指令促使：
    在确定所述第二业务板成功创建了与所述第一用户对应的第一硬件用户表项时，在数据平面上在所述第一软件用户表项中记录所述第二业务板的槽位ID；
    在数据平面上接收来自用户主机的数据报文，若接收到的数据报文与所述第一软件用户表项匹配，则所述第一业务板在数据平面上根据所述第一软件用户表项中记录的所述第二业务板的槽位ID，将所述数据报文转发给所述第二业务板。
  10. 根据权利要求8所述的网关设备，其中，所述处理器进一步被所述机器可执行指令促使：
    在数据平面上接收第三业务板发来的与第二用户对应的软件用户表项，根据接收到的软件用户表项，创建与所述第二用户对应的第二软件用户表项和第二硬件用户表项，并在所述第二软件用户表项中记录所述第三业务板的槽位ID。
  11. 根据权利要求10所述的网关设备，其中，所述处理器进一步被所述机器可执行指令促使：
    在创建了所述第二软件用户表项和所述第二硬件用户表项之后，在数据平面上创建包含所述第二用户的主机IP地址的第一路由表项，其中，所述第一路由表项中的目的设备ID为所述资源组的ID、目的端口ID为本第一业务板的槽位ID；
    在数据平面上将所述第一路由表项同步给所述资源组中的其它业务板。
  12. 根据权利要求11所述的网关设备，其中，所述处理器进一步被所述机器可执行指令促使：
    所述第一业务板在数据平面上接收第四业务板同步过来的第二路由表项并进行保存。
  13. 根据权利要求11所述的网关设备，其中，所述第一软件用户表项中还记录有本第一业务板上连接所述第一用户的主机的端口ID，
    所述处理器进一步被所述机器可执行指令促使：
    在数据平面上接收第四业务板同步过来的第二路由表项；
    在数据平面上根据该第二路由表项中包含的IP地址查找对应的软件用户表项；
    若没有查找到该IP地址对应的软件用户表项，在数据平面上则保存接收到的所述第二路由表项；
    若查找到了所述第一软件用户表项，在数据平面上则保存所述第二路由表项，并将所述第二路由表项中的目的设备ID修改为本第一业务板的槽位ID，将目的端口ID修改为该IP地址对应的所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID。
  14. 根据权利要求12所述的网关设备，其中，所述处理器进一步被所述机器可执行指令促使：
    在数据平面上接收发往用户主机的数据报文，根据接收到的数据报文的目的IP地址查找对应的第三路由表项，该第三路由表项中的目的设备ID为所述资源组的ID、目的端口ID为槽位ID；
    在数据平面上根据该数据报文的目的IP地址查找对应的软件用户表项和硬件用户表项；
    若均没有查找到，在数据平面上则根据该第三路由表项中作为目的端口ID的槽位ID，将该数据报文转发给对应槽位上的业务板；
    若查找到了所述第二软件用户表项和所述第二硬件用户表项，在数据平面上则根据所述第二硬件用户表项进行相应处理，之后根据所述第二软件用户表项中记录的所述第三业务板的槽位ID，将该数据报文转发给所述第三业务板；
    若查找到了所述第一软件用户表项、但没有查找到硬件用户表项，在数据平面上则根据所述第一软件用户表项中记录的本第一业务板上连接所述第一用户的主机的端口ID，将该数据报文转发给所述第一用户的主机。
  15. 一种机器可读存储介质，存储有机器可读指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行：
    在第一用户认证通过后，在分布式接入网关设备中的第一业务板的数据平面上创建与所述第一用户对应的第一软件用户表项，判断所述第一业务板上是否存在空闲硬件资源；
    若不存在，在所述数据平面上则获取存在空闲硬件资源的第二业务板的槽位标识ID，其中，所述第二业务板是从资源组中选择出来的，所述资源组包括所述分布式接入网关设备上的两个以上业务板；
    在数据平面上根据获取到的所述第二业务板的槽位ID将所述第一软件用户表项发送给所述第二业务板，以使所述第二业务板在数据平面上创建与所述第一用户对应的硬件用户表项。
PCT/CN2017/117507 2016-12-21 2017-12-20 资源调度 WO2018113701A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP17882416.5A EP3544251B1 (en) 2016-12-21 2017-12-20 Resource scheduling method
JP2019533519A JP6741872B2 (ja) 2016-12-21 2017-12-20 リソーススケジューリング
US16/472,861 US11184291B2 (en) 2016-12-21 2017-12-20 Scheduling resources

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611192753.2 2016-12-21
CN201611192753.2A CN108234422B (zh) 2016-12-21 2016-12-21 资源调度方法及装置

Publications (1)

Publication Number Publication Date
WO2018113701A1 true WO2018113701A1 (zh) 2018-06-28

Family

ID=62624756

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/117507 WO2018113701A1 (zh) 2016-12-21 2017-12-20 资源调度

Country Status (5)

Country Link
US (1) US11184291B2 (zh)
EP (1) EP3544251B1 (zh)
JP (1) JP6741872B2 (zh)
CN (1) CN108234422B (zh)
WO (1) WO2018113701A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111083059A (zh) * 2019-12-30 2020-04-28 杭州迪普科技股份有限公司 一种分布式设备中的流量转发方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11219037B2 (en) * 2017-05-09 2022-01-04 Intel Corporation Radio resource scheduling
CN109831326B (zh) * 2019-01-25 2021-08-06 新华三技术有限公司 网络设备控制方法及网络设备
CN110300068B (zh) * 2019-06-14 2023-04-07 杭州迪普科技股份有限公司 Arp资源管理方法、装置、电子设备
CN114979282B (zh) * 2022-07-28 2023-01-20 北京金山云网络技术有限公司 任务调度方法、装置、存储介质以及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101247421A (zh) * 2008-03-28 2008-08-20 杭州华三通信技术有限公司 分布式架构下nat地址池的自适应分配方法和系统
US20130019276A1 (en) * 2011-07-11 2013-01-17 International Business Machines Corporation Automatic Generation of User Account Policies Based on Configuration Management Database Information
CN103702366A (zh) * 2013-12-25 2014-04-02 上海寰创通信科技股份有限公司 一种处理报文信息的系统及方法
CN104320272A (zh) * 2014-10-21 2015-01-28 中国联合网络通信集团有限公司 一种传递设备信息的方法和网络设备
CN104852859A (zh) * 2015-04-30 2015-08-19 杭州华三通信技术有限公司 一种聚合接口业务处理方法和设备

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6735198B1 (en) * 1999-12-21 2004-05-11 Cisco Technology, Inc. Method and apparatus for updating and synchronizing forwarding tables in a distributed network switch
US6807172B1 (en) * 1999-12-21 2004-10-19 Cisco Technology, Inc. Method and apparatus for learning and switching frames in a distributed network switch
JP4594874B2 (ja) * 2005-02-28 2010-12-08 富士通株式会社 ネットワークエリアにおける資源割当て方法、割当てプログラム、およびネットワークシステム
EP1976195B1 (en) * 2007-03-30 2014-05-07 Alcatel-Lucent Method and apparatus for Mac address learning
US20080298230A1 (en) * 2007-05-30 2008-12-04 Luft Siegfried J Scheduling of workloads in a distributed compute environment
WO2011093288A1 (ja) * 2010-02-01 2011-08-04 日本電気株式会社 ネットワークシステム、コントローラ、ネットワーク制御方法
JP5741150B2 (ja) * 2011-04-04 2015-07-01 富士通株式会社 中継装置、中継プログラム、及び中継方法
EP2536068B1 (en) 2011-06-18 2014-08-13 Deutsche Telekom AG Virtual subscriber management
EP2806603A4 (en) * 2012-01-16 2015-09-30 Nec Corp NETWORK SYSTEM AND SYNCHRONIZATION PROCESS FOR WAY INFORMATION
US8917627B2 (en) * 2012-02-01 2014-12-23 International Business Machines Corporation Synchronizing routing tables in a distributed network switch
US9135040B2 (en) * 2012-08-03 2015-09-15 International Business Machines Corporation Selecting provisioning targets for new virtual machine instances
CN103227757B (zh) * 2012-08-31 2016-12-28 杭州华三通信技术有限公司 一种报文转发方法及设备
US9749173B2 (en) * 2012-09-11 2017-08-29 Ciena Corporation Systems and methods for synchronizing forwarding databases across multiple interconnected layer-2 switches
JP5860423B2 (ja) * 2013-02-18 2016-02-16 日本電信電話株式会社 キャリア網仮想化システム及び方法
CN104022960B (zh) * 2013-02-28 2017-05-31 新华三技术有限公司 基于OpenFlow协议实现PVLAN的方法和装置
ES2644311T3 (es) * 2013-09-20 2017-11-28 Nokia Solutions And Networks Gmbh & Co. Kg Manipulación de mensajes de señalización en el plano de datos en una arquitectura definida por software
US9658984B2 (en) * 2014-07-15 2017-05-23 Cisco Technology, Inc. Method and apparatus for synchronizing multiple MAC tables across multiple forwarding pipelines
US20160034548A1 (en) * 2014-07-31 2016-02-04 Dell Products, Lp System and Method for Obtaining Automated Scaling of a Virtual Desktop Environment
US10305796B2 (en) * 2015-06-01 2019-05-28 Ciena Corporation Enhanced forwarding database synchronization for media access control addresses learned in interconnected layer-2 architectures

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101247421A (zh) * 2008-03-28 2008-08-20 杭州华三通信技术有限公司 分布式架构下nat地址池的自适应分配方法和系统
US20130019276A1 (en) * 2011-07-11 2013-01-17 International Business Machines Corporation Automatic Generation of User Account Policies Based on Configuration Management Database Information
CN103702366A (zh) * 2013-12-25 2014-04-02 上海寰创通信科技股份有限公司 一种处理报文信息的系统及方法
CN104320272A (zh) * 2014-10-21 2015-01-28 中国联合网络通信集团有限公司 一种传递设备信息的方法和网络设备
CN104852859A (zh) * 2015-04-30 2015-08-19 杭州华三通信技术有限公司 一种聚合接口业务处理方法和设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3544251A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111083059A (zh) * 2019-12-30 2020-04-28 杭州迪普科技股份有限公司 一种分布式设备中的流量转发方法
CN111083059B (zh) * 2019-12-30 2023-04-25 杭州迪普科技股份有限公司 一种分布式设备中的流量转发方法

Also Published As

Publication number Publication date
CN108234422A (zh) 2018-06-29
CN108234422B (zh) 2020-03-06
EP3544251A4 (en) 2019-11-20
US11184291B2 (en) 2021-11-23
EP3544251A1 (en) 2019-09-25
EP3544251B1 (en) 2021-07-14
JP6741872B2 (ja) 2020-08-19
US20190372907A1 (en) 2019-12-05
JP2020502941A (ja) 2020-01-23

Similar Documents

Publication Publication Date Title
WO2018113701A1 (zh) 资源调度
WO2018028606A1 (zh) 转发策略配置
EP3461072B1 (en) Access control in a vxlan
US8566257B2 (en) Address data learning and registration within a distributed virtual bridge
US10749799B2 (en) Data routing of extranet flows in fabric networks
US20220070095A1 (en) Data transmission method and apparatus, network adapter, and storage medium
US20180139272A1 (en) Live migration of load balanced virtual machines via traffic bypass
US20140286336A1 (en) System and Method for Paging Flow Entries in a Flow-Based Switching Device
US10616175B2 (en) Forwarding information to forward data to proxy devices
WO2014101777A1 (zh) 流表匹配方法和装置以及交换机
EP3451592B1 (en) Packet transmission between vxlan domains
TW200820678A (en) Address resolution protocol cache management methods and devices
RU2540820C2 (ru) Способ и коммутатор для отправки пакета
US20150281078A1 (en) Method and Apparatus for Cross-Service-Zone Communication, and Data Center Network
CN107493222B (zh) 一种vxlan报文转发的方法及装置
WO2018099394A1 (zh) 报文传输
US20150026780A1 (en) Host providing system and communication control method
US11451466B2 (en) Controlling route
US20220210005A1 (en) Synchronizing communication channel state information for high flow availability
KR20140007363A (ko) 기업 네트워크 외부로부터의 사이트-인식 분산형 파일 시스템 액세스
US10764234B2 (en) Method and system for host discovery and tracking in a network using associations between hosts and tunnel end points
US11146464B2 (en) Systems and methods for enabling inter-agent multicast domain name system client roaming in wide area network
WO2018171722A1 (zh) Mac地址同步
US9203741B1 (en) Managing multi-customer network traffic using lower layer protocol attributes
CN106878052B (zh) 一种用户迁移方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17882416

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019533519

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017882416

Country of ref document: EP

Effective date: 20190621