EP3051773B1 - Multi-path auxiliary stream control method, control device, node and system - Google Patents

Multi-path auxiliary stream control method, control device, node and system Download PDF

Info

Publication number
EP3051773B1
EP3051773B1 EP13896107.3A EP13896107A EP3051773B1 EP 3051773 B1 EP3051773 B1 EP 3051773B1 EP 13896107 A EP13896107 A EP 13896107A EP 3051773 B1 EP3051773 B1 EP 3051773B1
Authority
EP
European Patent Office
Prior art keywords
token
node
auxiliary
auxiliary stream
child
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13896107.3A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP3051773A4 (en
EP3051773A1 (en
Inventor
Jing Xiao
Jing Wang
Jiaoli Wu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP3051773A1 publication Critical patent/EP3051773A1/en
Publication of EP3051773A4 publication Critical patent/EP3051773A4/en
Application granted granted Critical
Publication of EP3051773B1 publication Critical patent/EP3051773B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems
    • H04N7/152Multipoint control units therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066Session management
    • H04L65/1083In-session procedures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066Session management
    • H04L65/1083In-session procedures
    • H04L65/1094Inter-user-equipment sessions transfer or sharing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40Support for services or applications
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40Support for services or applications
    • H04L65/403Arrangements for multi-party communication, e.g. for conferences
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40Network security protocols

Definitions

  • the present invention relates to the communications field, and in particular, to a method and system for controlling multiple auxiliary streams, a control device, and a node.
  • a user conference site has developed from one video camera, single channel of live video, and a single channel of live image to multiple cameras, multiple channels of live videos, and multiple channels of live images.
  • the multiple video cameras, multiple channels of live videos, and multiple channels of live images at the same conference site are associated via a physical or logical relationship.
  • a video of conference content for example, PPT
  • PPT video of conference content
  • Implementation of the technology relies on a definition of a role of an auxiliary stream in the H.323 and H.239 protocols in the prior art.
  • the protocols specify that an auxiliary stream is used to transmit an image of conference content, such as a PPT slide and a DVD video.
  • An auxiliary stream form that is commonly used at present is a PRE (Presentation) form.
  • PRE Presentation
  • a token management mode is used.
  • a conference has only one presentation token, a conference site that obtains the token can send an auxiliary stream, and the auxiliary stream of the conference site that has the token is viewed at each conference site.
  • the conference site stops sending an auxiliary stream the token can be released.
  • the another conference site needs to send an auxiliary stream, the another conference site must obtain the token by requesting, and all applicants may request the token as long as the token is not locked.
  • a general scenario is that a projected physical object is being viewed and a slide is being presented at the same time.
  • a solution for controlling multiple auxiliary streams is proposed in the prior art.
  • an extra PREAnnex logic channel is extended to improve a single node's capability of sending an auxiliary stream, so that a node that obtains a presentation token by means of application can send an extra auxiliary stream through the PREAnnex logic channel.
  • Sending of multiple auxiliary streams may be implemented by using this method.
  • a node can send multiple auxiliary streams only after the node obtains a token by means of application. If an auxiliary stream needs to be simultaneously sent at multiple conference sites in a conference, the solution still cannot be used to solve the problem.
  • Embodiments of the present invention provide a method and system for controlling multiple auxiliary streams, a control device, and a node, which can implement sending and receiving of multiple auxiliary streams by multiple nodes.
  • the embodiments of the present invention provide a method and system for controlling multiple auxiliary streams, a control device, and a node.
  • the control device determines to allocate a token to m auxiliary streams, and allocates a token to the m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n; then, the control device sends a token allocated to an auxiliary stream of a first node to the first node, where the first node is one node of the n node, so that the first node can obtain the token; then, the control device sends a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, so that the first node can send the first auxiliary stream after receiving the first indication message, where the first auxiliary stream includes at least one auxiliary stream of the m auxiliary streams.
  • the determining to allocate a token to m auxiliary streams may be performed by the control device according to a delivered allocation policy, or may be performed by the control device after an auxiliary stream request that is sent by a node managed by the control device is received and when a token capability of the control device is satisfied, which is not specifically limited in this example.
  • n is a quantity of nodes.
  • the quantity of auxiliary streams is not less than the quantity of nodes, that is, m ⁇ n, where n ⁇ 1 is also satisfied so that management on auxiliary streams of multiple nodes can be implemented, and m ⁇ 2 is also satisfied so that sending of multiple auxiliary streams of one node can be implemented.
  • the node in this example specifically may be a conference terminal at each conference site, may be a media gateway, or may be a conference application server, which is not specifically limited in this example; one node may include multiple conference terminals, which is also not specifically limited in this example either.
  • the control device allocates the token to the m auxiliary streams.
  • control device may allocate the token to the m auxiliary streams.
  • the control device sends the token allocated to an auxiliary stream of a first node to the first node, where the first node is one node of the n node.
  • the control device After the control device allocates the token to the m auxiliary streams, the control device sends the token allocated to an auxiliary stream of the first node to the first node, so that the first node can obtain the token, and then, when an auxiliary stream needs to be sent, determine, according to the token, whether the auxiliary stream can be sent.
  • first node is one node of the n node, and in this example, one first node may be included, or multiple first nodes may be included, which is not specifically limited in this example.
  • the control device sends a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, and the first auxiliary stream includes at least one auxiliary stream of the m auxiliary streams.
  • the first indication message may be sent to one first node, or the first indication message may be sent to multiple first nodes, which is not specifically limited in this example. It is limited only to that the first node is one node of the n node.
  • the first auxiliary stream may include one auxiliary stream, or may include multiple auxiliary streams, which is also not specifically limited in this example.
  • a control device determines to allocate a token to m auxiliary streams, and allocates a token to the m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n; then, the control device sends a token allocated to an auxiliary stream of a first node to the first node, where the first node is one node of the n node, so that the first node can obtain the token; then, the control device sends a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, so that the first node can send the first auxiliary stream after receiving the first indication message, where the first auxiliary stream includes at least one auxiliary stream of the m auxiliary streams.
  • simultaneously sending auxiliary streams by multiple nodes can be implemented.
  • An example further provides a method for controlling multiple auxiliary streams.
  • the method is applied to a node, and the method includes: 201.
  • a node acquires, from a control device, a token required by at least one auxiliary stream.
  • the token required by the at least one auxiliary stream is allocated by the control device; the allocating may be performed by the control device according to a delivered allocation policy, or may be performed by the control device, after an auxiliary stream request that is sent by a node managed by the control device is received and when a token capability of the control device is satisfied, which is not specifically limited in this example. It is described only that the node acquires, from the control device, the token required by the at least one auxiliary stream.
  • tokens may be defined in multiple manners, and different control devices may manage different types of tokens. Therefore, after acquiring, from the control device, the token required by the at least one auxiliary stream channel, the node first needs to determine the type of the token.
  • the type of the token may include: a first token corresponding to a node, a second token corresponding to an auxiliary stream, or a child token that is corresponding to an auxiliary stream and has a corresponding parent token, where the parent token is a first token corresponding to a node.
  • the first token is a token corresponding to a node; after a node acquires the first token, the node is capable of sending all auxiliary streams included in the node, but specifically which auxiliary streams need to be sent needs to be determined according to an indication message received by the node, which is not specifically limited in this example.
  • the first token may be of two types, that is, a token for a single auxiliary stream and a token for multiple auxiliary streams.
  • the first token is the token for a single auxiliary stream
  • the multiple auxiliary streams need to be synthesized and then sent as one auxiliary stream
  • the first token is the token for multiple auxiliary streams
  • multiple auxiliary stream channels may be established for sending multiple auxiliary streams; where the synthesis includes image processing operations such as combination, splicing, and Picture in Picture.
  • a parent token is allocated to a node to which the auxiliary stream belongs, and a child token is allocated to the auxiliary stream.
  • the node can send the auxiliary stream after receiving an indication message for sending the auxiliary stream; otherwise, the auxiliary stream cannot be sent.
  • the node after receiving, from the control device, the token required by the at least one auxiliary stream, the node does not send the at least one auxiliary stream until receiving the first indication message sent by the control device, and determining, according to the first indication message, the first auxiliary stream that needs to be sent.
  • the "specific mechanism” may be a rule of "applied first", that is, an auxiliary stream that is first triggered is determined as an auxiliary stream that needs to be sent; or may be a rule of "importance", that is, if two auxiliary streams are triggered simultaneously at a moment, the more important auxiliary stream is preferentially selected and determined as an auxiliary stream that needs to be sent.
  • a rule of "applied first” that is, an auxiliary stream that is first triggered is determined as an auxiliary stream that needs to be sent
  • a rule of "importance" that is, if two auxiliary streams are triggered simultaneously at a moment, the more important auxiliary stream is preferentially selected and determined as an auxiliary stream that needs to be sent.
  • another possibility may exist, which is not specifically limited in this example.
  • the node sends the first auxiliary stream according to the determined type of the token when the token required by the at least one auxiliary stream includes a token required by the first auxiliary stream.
  • the sending the first auxiliary stream according to the determined type of the token if the token required by the at least one auxiliary stream includes a token required by the first auxiliary stream may include:
  • node 1 belongs to a second node
  • the token that is acquired by node 1 and is corresponding to the two auxiliary streams may be one token, or may be two tokens, which is specifically determined according to a type of the token and is not specifically limited in this embodiment of the present invention.
  • the node may receive the multiple tokens simultaneously, or may receive the multiple tokens one by one, and the multiple tokens may be received in a random order, which is not specifically limited in this embodiment of the present invention.
  • Node 1 determines a type of the token.
  • the first token may further be classified into a token for multiple auxiliary streams and a token for a single auxiliary stream.
  • the token for multiple auxiliary streams may be used to send multiple auxiliary streams
  • the token for a single auxiliary stream may be used to send one auxiliary stream that is obtained by performing image processing such as combination, splicing, and Picture in Picture on some or all of multiple auxiliary streams.
  • the control device sends a first indication message to node 1, where the first indication message is used to instruct the node 1 to send a first auxiliary stream according to the received token.
  • the control device after the control device sends, to node 1, the token that is required by the p auxiliary stream and is allocated to node 1, the control device further sends the first indication message to the first node, where the first indication message may be triggered after the control device receives a request message that is sent by another node to request to acquire the first auxiliary stream, or may be triggered by the control device according to a control policy, which is not specifically limited in this embodiment of the present invention.
  • node 1 after receiving the token required by the p auxiliary stream, node 1 does not necessarily send the p auxiliary stream. Instead, node 1 first determines, according to the first indication message, the first auxiliary stream that needs to be sent.
  • the first auxiliary stream may be one auxiliary stream, or may be multiple auxiliary streams, which is not specifically limited in this embodiment of the present invention and reference may be made to the description in step 203 in example 1, and which is determined only according to the first indication message.
  • the control device sends, to node 2, the token that is required by the q auxiliary stream and allocated to node 2.
  • Node 2 acquires the token required by the q auxiliary stream.
  • Node 2 determines a type of the token.
  • an embodiment of the present invention further provides a method for controlling multiple auxiliary streams.
  • the method includes: 401a.
  • Node 1 sends auxiliary stream resource information to a control device, where the auxiliary stream resource information is information that represents an auxiliary stream included in node 1.
  • the control device receives the auxiliary stream resource information reported by node 1 and node 2.
  • the control device determines, according to the auxiliary stream resource information reported by node 1 and node 2, an auxiliary stream to which a token may be allocated.
  • a control device may not match all auxiliary streams of the controlled node with managed tokens because of a limited quantity of tokens managed by the control device. Therefore, an auxiliary stream to which a token may be allocated should be determined first.
  • the control device may determine, according to information, such as a resolution, bandwidth, and a node, that is carried in the auxiliary stream resource information, the auxiliary stream to which a token may be allocated.
  • auxiliary stream resource information reported from conference site 1 includes information about three auxiliary streams: slide 1, picture 1, and video 1
  • auxiliary stream resource information reported from conference site 2 includes information about two auxiliary streams: slide 2 and picture 2.
  • a conference chairperson determines that auxiliary streams that can be sent are the auxiliary streams of conference site 1.
  • the control device determines, according to node information carried in the reported auxiliary stream resource information, that auxiliary streams to which a token may be allocated may be slide 1, picture 1, and video 1 of conference site 1.
  • auxiliary stream resource information there may be another method for determining, according to the auxiliary stream resource information, the auxiliary stream to which a token may be allocated, which is not specifically limited in this embodiment of the present invention.
  • the control device matches a reserved token with the auxiliary stream to which a token may be allocated, so as to acquire a correspondence between a token and an auxiliary stream.
  • the control device matches the reserved token with the auxiliary stream to which a token may be allocated, which specifically includes the following three cases: Case 1:
  • the auxiliary stream resource information includes: a node number corresponding to an auxiliary stream included in each controlled node.
  • the matching a reserved token with the auxiliary stream to which a token may be allocated specifically includes: matching, according to a first rule, the reserved token with the auxiliary stream to which a token may be allocated, where the first rule is as follows: A token is allocated by node, and auxiliary streams with a same node number are allocated a same token.
  • the auxiliary stream resource information includes: an identifier of an auxiliary stream included in each controlled node.
  • control device acquires a correspondence between a token and an auxiliary stream only means that an auxiliary stream may be allocated a token, but does not represent that an auxiliary stream is already allocated a token. Therefore, after matching the reserved token with the auxiliary stream to which a token may be allocated, the control device further determines, according to the allocation policy, to allocate the token to the m auxiliary streams.
  • auxiliary stream 1 of a first conference site there are three conference sites in a conference, where a second conference site and a third conference site want to receive auxiliary stream 1 of a first conference site, the second conference site and the first conference site wants to receive auxiliary stream 3 of the third conference site, and the first conference site further wants to receive auxiliary stream 2 of the second conference site.
  • the allocation policy is the policy of precedence based on selection by a majority of users, it should be determined that the token is to be allocated to auxiliary stream 1 and auxiliary stream 3.
  • the control device allocates a token required by the m auxiliary streams to the m auxiliary streams according to the acquired correspondence between a token and an auxiliary stream.
  • the control device sends, to node 1, a token that is required by the p auxiliary stream and allocated to node 1.
  • node 1 acquires tokens required by two auxiliary streams.
  • the token that is acquired by node 1 and is corresponding to the two auxiliary streams may be one token, or may be two tokens, which is specifically determined according to a type of the token and is not specifically limited in this embodiment of the present invention.
  • the node may receive the multiple tokens simultaneously, or may receive the multiple tokens one by one, and the multiple tokens may be received in a random order, which is not specifically limited in this embodiment of the present invention.
  • the first token may further be classified into a token for multiple auxiliary streams and a token for a single auxiliary stream.
  • the token for multiple auxiliary streams may be used to send multiple auxiliary streams
  • the token for a single auxiliary stream may be used to send one auxiliary stream that is obtained by performing image processing such as combination, splicing, and Picture in Picture on some or all of multiple auxiliary streams.
  • the determining a type of the token may specifically include: determining the type of the token according to an identifier carried in the token, or determining the type of the token by sending a confirmation message to the control device, which is not specifically limited in this embodiment of the present invention.
  • the control device sends a first indication message to node 1, where the first indication message is used to instruct the node 1 to send a first auxiliary stream according to the received token.
  • the control device after the control device sends, to node 1, the token that is required by the p auxiliary stream and is allocated to node 1, the control device further sends the first indication message to the first node, where the first indication message may be triggered after the control device receives a request message that is sent by another node to request to acquire the first auxiliary stream, or may be triggered by the control device according to a control policy, which is not specifically limited in this embodiment of the present invention.
  • Node 1 receives the first indication message.
  • node 1 after receiving the token required by the p auxiliary stream, node 1 does not necessarily send the p auxiliary stream. Instead, node 1 first determines, according to the first indication message, the first auxiliary stream that needs to be sent.
  • the first auxiliary stream may be one auxiliary stream, or may be multiple auxiliary streams, which is not specifically limited in this embodiment of the present invention and reference may be made to the description in step 203 in example 1, and which is determined only according to the first indication message.
  • the sending the first auxiliary stream according to the determined type of the token when the token required by the p auxiliary stream includes a token required by the first auxiliary stream may specifically include:
  • the node should re-apply or wait for the token required by the first auxiliary stream according to a specific policy, which is not discussed in this embodiment of the present invention.
  • the control device sends, to node 2, a token that is required by the q auxiliary stream and allocated to node 2.
  • Node 2 determines a type of the token.
  • the control device sends a second indication message to node 2, where the second indication message is used to instruct the node 2 to send a second auxiliary stream according to the received token.
  • first indication message or the second indication message is an indication message sent by the control device to a node to instruct the node to send an auxiliary stream according to a received token.
  • Node 2 receives the second indication message.
  • the embodiments shown in FIG. 3A and FIG. 3B , and FIG. 4A and FIG. 4B elaborate on a method for controlling multiple auxiliary streams according to an interaction between a node and a control device.
  • the control device determines to allocate a token to m auxiliary streams, and allocates a token to the m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n; then, the control device sends a token allocated to an auxiliary stream of a first node to the first node, where the first node is one node of the n node, so that the first node can acquire at least one token and therefore can have permission to send an auxiliary stream; then, the control device sends a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, so that the first node can send the first auxiliary stream after receiving the first indication message, where the first
  • the control device may further send an auxiliary stream to a managed node. That is, after the sending a first indication message to the first node, the method further includes:
  • the first indication message is sent by the control device, which may be triggered by a user of a receiving party, or may be triggered by a user of a sending party.
  • the control device may be triggered by a user of a receiving party, or may be triggered by a user of a sending party.
  • the method further includes:
  • a related user may trigger a control related to auxiliary stream 1 and auxiliary stream 2, so that the fourth node sends a first request message to the control device, where the first request message is used to request to acquire auxiliary stream 1 and auxiliary stream 2.
  • the control device After receiving the first request message, the control device sends, according to the first request message, a first indication message to the first node.
  • the first node After receiving the first indication message, the first node sends auxiliary stream 1 and auxiliary stream 2, so that the control device receives auxiliary stream 1 and auxiliary stream 2, and sends auxiliary stream 1 and auxiliary stream 2 to the fourth node.
  • the sending, by the control device, a first indication message to the first node includes: sending the first indication message to the first node according to a control policy.
  • a control related to auxiliary stream 1 and auxiliary stream 2 may be triggered.
  • the control device may determine, according to a mechanism, to receive only auxiliary stream 2 and send auxiliary stream 2 to a second node.
  • the control device delivers a first indication message to the first node.
  • the first node sends auxiliary stream 2, so that the control device receives auxiliary stream 2 and sends auxiliary stream 2 to the second node.
  • the control device sends, according to the control policy, the first indication message to the first node, and acquires the first auxiliary stream sent by the first node, before the sending the first auxiliary stream to a fourth node
  • the control device further sends a query message to the fourth node to negotiate, with the fourth node, whether the fourth node can receive the first auxiliary stream, so as to ensure a smooth process of sending and receiving the auxiliary stream and avoid a waste of a resource. That is, the method may further include:
  • control device determines to send auxiliary stream 1 and auxiliary stream 2 to the fourth node, and if the fourth node does not send a confirmation message to the control device after a control device sends a query message to the fourth node, the control device does not send auxiliary stream 1 and auxiliary stream 2 to the fourth node.
  • the token type is the first token
  • the following provides three system framework diagrams showing that different nodes receive and send an auxiliary stream after the nodes acquire a token, which are respectively shown in FIG. 5, FIG. 6 , and FIG. 7 and briefly described as follows.
  • node 1 and node 2 respectively obtain token 1 and token 2 from a control device, and therefore, node 1 and node 2 are respectively capable of sending an auxiliary stream; node 3 and node 4 do not obtain a token and therefore are incapable of sending an auxiliary stream.
  • Node 1 sends auxiliary stream 1, and node 2 sends auxiliary stream 2.
  • node 1 and node 2 cannot receive any other auxiliary streams, node 3 can receive only auxiliary stream 1, and node 4 can receive auxiliary stream 1 and auxiliary stream 2.
  • the policy may be as follows: A node determines an auxiliary stream that the node wants to receive, a control device determines an auxiliary stream that the control device wants to send, and the node and the control device negotiates transmission of an auxiliary stream.
  • node 1 can send multiple auxiliary streams, for example, auxiliary stream 1 and auxiliary stream 3 after acquiring token 1.
  • the control device can perform processing on any combination of the multiple auxiliary streams, so that a node controlled by the control device can receive any combination of these auxiliary streams, where the performing processing on any combination of the multiple auxiliary streams includes: synthesis for continuous presence, synthesis for Picture in Picture, resolution change, bit rate adjustment, and the like. As shown in FIG. 7 , continuous presence obtained by synthesizing auxiliary stream 1 and auxiliary stream 2 may be sent to node 4.
  • the node in this embodiment of the present invention specifically may be a conference terminal at each conference site, may be a media gateway, or may be a conference application server, which is not specifically limited in this embodiment of the present invention; one node may include multiple conference terminals, which is also not specifically limited in this embodiment of the present invention either.
  • a control device determines to allocate a token to m auxiliary streams, and allocates a token to the m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n; then, the control device sends a token allocated to an auxiliary stream of a first node to the first node, where the first node is one node of the n node, so that the first node can obtain the token; then, the control device sends a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, so that the first node can send the first auxiliary stream after receiving the first indication message, where the first auxiliary stream includes at least one auxiliary stream of the m auxiliary streams.
  • simultaneously sending auxiliary streams by multiple nodes can be implemented.
  • An embodiment of the present invention provides a control device 800.
  • the control device 800 includes a determining unit 801, an allocating unit 802, and a sending unit 803.
  • the determining unit 801 is configured to determine to allocate a token to m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n.
  • the allocating unit 802 is configured to allocate a token to the m auxiliary streams determined by the determining unit 801.
  • the sending unit 803 is configured to send a token allocated by the allocating unit 802 to an auxiliary stream of a first node to the first node, where the first node is one node of the n node.
  • control device 800 further includes a receiving unit 804 and an updating unit 805.
  • the receiving unit 804 is configured to: before the determining unit 801 determines to allocate the token to the m auxiliary streams, receive a request message from the n node, where the request message from the n node is used to request to acquire a token required for sending the m auxiliary streams.
  • the determining unit 801 determining to allocate a token to m auxiliary streams specifically includes:
  • the allocating unit 802 allocating a token to the m auxiliary streams includes: allocating, according to the token information, the token required by the m auxiliary streams to the m auxiliary streams.
  • the updating unit 805 is configured to update the token information after the allocating unit 802 allocates the token to the m auxiliary streams.
  • the token is a first token corresponding to a node.
  • the token information includes a quantity of idle tokens, and a correspondence between a used token and a token-allocated node.
  • the determining unit 801 determining to allocate the token to the m auxiliary streams if the token information represents that the token required by the m auxiliary streams is not occupied specifically includes: if the quantity of idle tokens is not less than n, determining to allocate the token to the m auxiliary streams.
  • the updating unit 805 updating the token information includes: updating, in the token information, the quantity of idle tokens, and information about the correspondence between a used token and a token-allocated node.
  • the token is a second token corresponding to an auxiliary stream.
  • the token information includes: a quantity of idle tokens, and a correspondence between a used token and a token-allocated auxiliary stream of a node.
  • the determining unit 801 determining to allocate the token to the m auxiliary streams if the token information represents that the token required by the m auxiliary streams is not occupied specifically includes: if the quantity of idle tokens is not less than m, determining to allocate the token to the m auxiliary streams.
  • the updating unit 805 updating the token information includes: updating, in the token information, the quantity of idle tokens, and information about the correspondence between a used token and a token-allocated auxiliary stream of a node.
  • the token is a child token corresponding to an auxiliary stream, where the child token has a corresponding parent token, and the parent token is a first token corresponding to a node.
  • the token information includes: a quantity of idle parent tokens, a quantity of idle child tokens included in each idle parent token, a correspondence between a used parent token and a token-allocated node, a correspondence between the used parent token and a quantity of idle child tokens included in the used parent token, and a correspondence between a used child token included in the used parent token and a token-allocated auxiliary stream.
  • the determining unit 801 determining to allocate the token to the m auxiliary streams if the token information represents that the token required by the m auxiliary streams is not occupied specifically includes: if a quantity of tokens requested by a second node in the n node is not greater than a quantity of idle child tokens included in a parent token corresponding to the second node, a quantity of third nodes in the n node is not greater than the quantity of idle parent tokens in the token information, and a quantity of tokens requested by each third node is not greater than a quantity of idle child tokens included in a parent token corresponding to the third node, determining to allocate the token to the m auxiliary streams, where the second node is a node to which a used parent token is allocated, and the third node is a node to which no parent token is allocated.
  • the updating unit 805 updating the token information includes: updating, in the token information, the quantity of idle parent tokens, the quantity of idle child tokens included in each idle parent token, the correspondence between a used parent token and a token-allocated node, the correspondence between a used parent token and a quantity of idle child tokens included in the used parent token, and the correspondence between a used child token included in a used parent token and a token-allocated auxiliary stream.
  • the sending unit 803 sending a first indication message to the first node includes: sending the first indication message to the first node according to a control policy.
  • the type of the token is a first token corresponding to the node and is a token for multiple auxiliary streams.
  • the processor 1401 is configured to determine to allocate a token to m auxiliary streams, where the m auxiliary streams belong to n node, n ⁇ 1, m ⁇ 2, and m ⁇ n.
  • the token is a second token corresponding to an auxiliary stream.
  • the processor 1401 is further configured to determine, according to the auxiliary stream resource information, an auxiliary stream to which a token may be allocated.
  • the sending, by the sender 1402, a first indication message to the first node includes: sending the first indication message to the first node according to the first request message.
  • the sender 1402 is further configured to send a query message before sending the first auxiliary stream to the fourth node, where the query message is used to query whether the fourth node can receive the first auxiliary stream.
  • An embodiment of the present invention provides a node 1600.
  • the node 1600 includes a processor 1601, a receiver 1602, and a sender 1603.
  • the processor 1601 is configured to acquire, from a control device, a token required by at least one auxiliary stream.
  • the receiver 1602 is configured to receive a first indication message sent by the control device, where the first indication message is used to instruct the node 1600 to send a first auxiliary stream.
  • the sender 1603 is further configured to: before the processor 1601 acquires, from the control device, the token required by the at least one auxiliary stream, send a request message to the control device, where the request message is used to request to acquire the token required by the at least one auxiliary stream.
  • the type of the token is a first token corresponding to the node and is a token for multiple auxiliary streams.
  • the sending, by the sender 1603, the first auxiliary stream according to the determined type of the token when the token required by the at least one auxiliary stream includes a token required by the first auxiliary stream specifically includes:
  • the token is a first token corresponding to the node and is a token for a single auxiliary stream.
  • the sending, by the sender 1603, the first auxiliary stream according to the determined type of the token when the token required by the at least one auxiliary stream includes a token required by the first auxiliary stream specifically includes:
  • the token is the child token that is corresponding to an auxiliary stream and has a corresponding parent token.
  • the processor 1601 is further configured to: after the sender 1603 sends the request message to the control device, and before the token required by the at least one auxiliary stream is acquired from the control device, acquire, from the control device, a parent token of a node corresponding to the at least one auxiliary stream, where the parent token includes at least one idle child token, so as to acquire the token required by the at least one auxiliary stream.
  • auxiliary stream resource information specifically includes: a quantity of auxiliary streams included in the node 1600, and/or a type of an auxiliary stream included in the node 1600, and/or a brief description of an auxiliary stream included in the node 1600.
  • the auxiliary stream resource information includes: a node number of the node 1600.
  • auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, an auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with a reserved token and performs management, and allocates, with reference to an allocation policy, the token to the at least one auxiliary stream specifically includes: sending the auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, the auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with the reserved token and performs management according to a first rule, and allocates, with reference to the allocation policy, the token to the at least one auxiliary stream, where the first rule is as follows: A token is allocated by node 1600, and auxiliary streams with a same node 1600 number are allocated a same token.
  • the auxiliary stream resource information includes: an identifier of an auxiliary stream included in the node 1600.
  • auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, an auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with a reserved token and performs management, and allocates, with reference to an allocation policy, the token to the at least one auxiliary stream specifically includes: sending the auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, the auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with the reserved token and performs management according to a second rule, and allocates, with reference to the allocation policy, the token to the at least one auxiliary stream, where the second rule is as follows: A token is allocated by auxiliary stream, and auxiliary streams with different auxiliary stream identifiers are allocated different tokens.
  • auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, an auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with a reserved token and performs management, and allocates, with reference to an allocation policy, the token to the at least one auxiliary stream specifically includes: sending the auxiliary stream resource information of the node 1600 to the control device, so that after determining, according to the auxiliary stream resource information, the auxiliary stream to which a token may be allocated, the control device matches the auxiliary stream to which a token may be allocated with the reserved token and performs management according to a third rule, and allocates, with reference to the allocation policy, the token to the at least one auxiliary stream, where the third rule is as follows: A parent token is allocated by node 1600, and auxiliary streams with a same node number are allocated a same parent token; for auxiliary streams that have a same parent token, a child token is allocated
  • An embodiment of the present invention provides a system 1700 for controlling multiple auxiliary streams.
  • the system 1700 includes a control device 1701 and a node 1702.
  • the control device 1701 is configured to determine to allocate a token to m auxiliary streams, and allocate a token to the m auxiliary streams, where the m auxiliary streams belong to n node 1702, n ⁇ 1, m ⁇ 2, and m ⁇ n.
  • the control device 1701 is further configured to send a token allocated to an auxiliary stream of a first node to the first node, where the first node is one node 1702 of the n node.
  • the control device 1701 is further configured to send a first indication message to the first node, where the first indication message is used to instruct the first node to send a first auxiliary stream according to the received token, and the first auxiliary stream includes at least one auxiliary stream of the m auxiliary streams.
  • nodes are listed in the system 1700 for controlling multiple auxiliary streams shown in FIG. 17 .
  • a quantity of nodes may be 1, 2, 3, 4....
  • This embodiment of the present invention provides only an exemplary system for controlling multiple auxiliary streams, and does not specifically limit a quantity of nodes included in the system.
  • the first node is one node of the n node 1702, and the system 1700 for controlling multiple auxiliary streams that is shown in FIG. 17 may include one first node, or may include multiple first nodes, which is not specifically limited in this embodiment of the present invention.
  • a control device allocates tokens for auxiliary streams of multiple nodes, after a node acquires a token required by at least one auxiliary stream, the node can send an auxiliary stream to the control device, and then the control device can send the auxiliary stream to another node. In this way, managing and controlling multiple auxiliary streams of multiple nodes can be implemented.
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the described apparatus embodiment is merely exemplary.
  • the module or unit division is merely logical function division and may be other division in actual implementation.
  • a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces.
  • the indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.
  • the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the integrated unit When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the present invention essentially, or the part contributing to the prior art, or all or some of the technical solutions may be implemented in the form of a software product.
  • the software product is stored in a storage medium and includes several indication messages for instructing a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to perform all or some of the steps of the methods described in the embodiments of the present invention.
  • the foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc.
  • program code such as a USB flash drive, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
EP13896107.3A 2013-10-25 2013-10-25 Multi-path auxiliary stream control method, control device, node and system Active EP3051773B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/085935 WO2015058394A1 (zh) 2013-10-25 2013-10-25 一种多路辅流控制的方法、控制设备、节点和系统

Publications (3)

Publication Number Publication Date
EP3051773A1 EP3051773A1 (en) 2016-08-03
EP3051773A4 EP3051773A4 (en) 2016-09-28
EP3051773B1 true EP3051773B1 (en) 2020-09-16

Family

ID=50531063

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13896107.3A Active EP3051773B1 (en) 2013-10-25 2013-10-25 Multi-path auxiliary stream control method, control device, node and system

Country Status (4)

Country Link
US (1) US10455195B2 (zh)
EP (1) EP3051773B1 (zh)
CN (1) CN103765859B (zh)
WO (1) WO2015058394A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10887235B2 (en) * 2017-08-24 2021-01-05 Google Llc Method of executing a tuple graph program across a network
CN108566415A (zh) * 2018-03-12 2018-09-21 广东睿江云计算股份有限公司 一种基于web的分布式并发控制方法
CN111866580B (zh) * 2020-06-09 2022-12-20 厦门亿联网络技术股份有限公司 一种基于h.239的辅流协商方法及终端设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100373323B1 (ko) * 2000-09-19 2003-02-25 한국전자통신연구원 영상회의 시스템에서의 다자간 영상회의 방법
US20070256124A1 (en) * 2006-04-13 2007-11-01 Go Play Network, Inc. Collectible token data management
CN101459816B (zh) 2007-12-14 2015-09-09 华为终端有限公司 一种多点双流会议中控制辅流令牌的方法、系统及设备
CN101753329B (zh) * 2009-12-25 2013-08-07 华为终端有限公司 呼叫建立的方法、装置和系统
CN102572367B (zh) * 2010-12-07 2014-09-03 华为终端有限公司 多路辅流控制方法、装置及网络系统
WO2012170913A1 (en) * 2011-06-08 2012-12-13 Vidyo, Inc. Systems and methods for improved interactive content sharing in video communication systems
CN102868880B (zh) 2011-07-08 2017-09-05 中兴通讯股份有限公司 一种基于远程呈现的媒体传输方法及系统
US9600543B1 (en) * 2012-09-28 2017-03-21 Google Inc. Query-based stream

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP3051773A4 (en) 2016-09-28
WO2015058394A1 (zh) 2015-04-30
CN103765859B (zh) 2017-04-12
US20160241813A1 (en) 2016-08-18
US10455195B2 (en) 2019-10-22
CN103765859A (zh) 2014-04-30
EP3051773A1 (en) 2016-08-03

Similar Documents

Publication Publication Date Title
US7656824B2 (en) Method and system for providing a private conversation channel in a video conference system
US6704769B1 (en) Media role management in a video conferencing network
EP2437491A1 (en) Method and system for video conference control, network equipment and meeting places for video conference
US20050044503A1 (en) Server invoked time scheduled videoconference
EP2306714A1 (en) Multipoint control unit cascaded system and communication method and device thereof
KR102238399B1 (ko) Wi­fi 네트워크에서 멀티미디어 콘텐츠를 스트리밍하기 위한 방법 및 시스템
US20080285532A1 (en) Floor Control for Multimedia Push-To-Talk Applications
WO2020024775A1 (zh) 数据传输控制方法及相关装置
US10455195B2 (en) Method and system for controlling multiple auxiliary streams, control device, and node
US20060253589A1 (en) Media burst control system for multimedia, program, recording medium, media server and terminal
US9264662B2 (en) Chat preauthorization
US20220322469A1 (en) Radio bearer establishment method and apparatus, and communication device
US8327418B2 (en) Presentation apparatus with broadcasting function and method using the same
US20130083733A1 (en) Method and system for transmitting and receiving media according to importance of media burst
US20080244007A1 (en) Electronic conference system, information processing apparatus, and program
US20050282555A1 (en) Control arrangement for point-to-multipoint user services in a mobile radio network
EP3012998B1 (en) Multi-content media communication method, device and system
WO2012146203A1 (zh) 数据内容的显示方法、装置和系统
KR100755100B1 (ko) 멀티캐스트 기반 다자간 협업 시스템에서의 영상 서비스장치 및 방법
KR20160016655A (ko) 다자간 멀티미디어 공유 서비스 제공 시스템 및 방법
CN118101630A (zh) 通信方法和通信装置
CN117880169A (zh) 数据转发、转发管控方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20160428

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

A4 Supplementary search report drawn up and despatched

Effective date: 20160830

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 7/15 20060101ALI20160824BHEP

Ipc: H04L 29/08 20060101AFI20160824BHEP

Ipc: H04L 29/06 20060101ALI20160824BHEP

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20190703

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20200507

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013072669

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1315192

Country of ref document: AT

Kind code of ref document: T

Effective date: 20201015

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201217

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1315192

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200916

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210118

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210116

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013072669

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201025

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20201031

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20210617

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201031

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201031

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201025

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602013072669

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0029080000

Ipc: H04L0065000000

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200916

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230831

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230911

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230830

Year of fee payment: 11