CN102546081B - Method for detecting packet loss, system and media client - Google Patents
Method for detecting packet loss, system and media client Download PDFInfo
- Publication number
- CN102546081B CN102546081B CN201010599130.3A CN201010599130A CN102546081B CN 102546081 B CN102546081 B CN 102546081B CN 201010599130 A CN201010599130 A CN 201010599130A CN 102546081 B CN102546081 B CN 102546081B
- Authority
- CN
- China
- Prior art keywords
- sequence number
- bag
- packet
- contract
- project
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1809—Selective-repeat protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1829—Arrangements specially adapted for the receiver end
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/26—Special purpose or proprietary protocols or architectures
Abstract
The invention provides a kind of method for detecting packet loss, system and media client, wherein, this method for detecting packet loss comprises: receiving terminal obtains the bag sequence number of the packet self received; Receiving terminal obtains the total number information of giving out a contract for a project that transmitting terminal sends; And receiving terminal detects the packet of loss according to above-mentioned bag sequence number and total number information of giving out a contract for a project.Method for detecting packet loss provided by the invention, system and media client, initial packet loss can be detected with low cost and in time and and then realize packet loss and retransmit, and packet loss re-transmission can be carried out when switching flow media server.
Description
Technical field
The present invention relates to stream media technology, particularly relate to a kind of method for detecting packet loss, system and media client.
Background technology
Along with popularizing of broadband access, the application of stream media technology is more and more extensive, and media code stream transmits on ip networks, inevitably suffers from packet loss, thus causes the screen pause of picture flower and audio discontinuity to modify tone, and has a strong impact on media playback quality.In engineering practice, often adopt packet loss retransmission technique to deal with Network Packet Loss problem, by the distinguishing hierarchy of specific implementation, existing packet loss retransmission technique can roughly be divided into two types: transport layer realizes and application layer realizes.
Wherein, transport layer realizes packet loss re-transmission and adopts transmission control protocol (TCP) etc. to have connection protocol transmission of media data, is detected packet loss by transport layer protocol itself, automatically initiates to retransmit when packet loss.What this mode utilized is the packet loss functions of retransmission that transport layer carries, and realizes cost lower, but this type of protocol resource expense is comparatively large, and packet loss functions of retransmission is uncontrolled, is easily absorbed in the vicious circle of " congested-packet loss-re-transmission-more congested " when network congestion.Application layer realizes packet loss and retransmits and adopt the User Datagram Protoco (UDP) connectionless protocol transmission of media data such as (UDP), by application layer to packet loss carry out detection concurrent repeat requests.This mode need application sequence participate in, realize more complicated, but the resource overhead of transport layer protocol is lower, and the transmission of repeat requests and response all realized by application layer, corresponding strategies can be adjusted flexibly according to network condition, be relatively applicable to streaming media service.
The trend of current industry adopts udp protocol carrying media data, correspondingly be then that application layer packet loss retransmission technique reaches its maturity, these technology are similar, basic thought is all transmit leg is that each media (data) bag adds the bag sequence number increased progressively continuously, whether recipient detects the media bag sequence number received continuous, if find intermittently just to think and there occurs packet loss, and the sequence number of lost package is fed back to transmit leg by independently passage, after transmit leg receives feedback information, just corresponding media bag is resend to recipient.
Such scheme is analyzed a little and just can find: the precondition that packet loss is detected is that the bag before it is not lost completely, one at least will be had to be received side in that is little than loss media bag sequence number all media bags and successfully receive.This just means: if first media packet loss sending at first of streaming media server or unwrap several bags continuous of beginning from first media and all lose, recipient can't detect; Namely packet loss retransmits the overall process that cannot cover session.
What application number CN101123584 was called " a kind of method and apparatus of the IP of measurement link loss situation " patent discloses a kind of method measuring IP layer packet loss: send separately and delimit bag, delimit in bag the quantity information comprising the IP bag sent in a period of time, receiving terminal is according to this information, the bag quantity received during combining again, correctly can calculate the packet drop on link.There is disorder phenomenon in the transmission of IP bag, and the demarcation bag in this invention only comprises the bag total number information that transmitting terminal sends, lack with specifically certain wraps the information carrying out associating, so although receiving terminal can know in a period of time to lost how many bags, but accurately cannot know which two IP bag these packet losses specifically occur between, also cannot accurately know from first bag bag sum that transmitting terminal sends to certain specific IP bag---this means that it cannot detect initial packet loss.
Application number is CN101616316, the patent application of by name " a kind of transmission of video data, receiving system and sending, receiving method " does not adopt receiving terminal to carry out the conventional method of packet loss detection, but the information of the bag received by receiving terminal gathers and feeds back to transmitting terminal, itself and the situation of bag that self sends are carried out contrast by transmitting terminal just can detect packet loss, and and then performs packet loss and retransmit.In theory, transmitting terminal can detect initial packet loss in this way, but packet loss is only a few event after all in an ip network, and this invention needs to continue to transmitting terminal feedback information, larger to network bandwidth consumption; In addition, the packet loss detection work being originally distributed in each client is concentrated on transmitting terminal, also will inevitably consume the disposal ability of transmitting terminal.In a word, the lower deployment cost of this way is relatively high, so apply less in practice.
In addition, in the real time streaming transport protocol (RTSP) that (RFC) 2326 that request for comments defines, server can carry RTP-Info header field in the response message of PLAY operation, by seq subdomain wherein, the sequence number that first RTP wraps is informed to client, the RTP bag sequence number received compares with this value by client, can detect initial packet loss.In engineering practice, all that the independent bottom passage of employing two carries RTSP and RTP data respectively (Typical Disposition adopts TCP to carry RTSP when major part, UDP is adopted to carry RTP), can not ensure that client gets the relevant information of RTP-Info header field in time, visible, this mode adopting RTSP to provide detects initial packet loss, cannot ensure promptness, less effective.
Summary of the invention
Though initial packet loss cannot be detected in order to overcome prior art or to detect but effect is bad or the higher shortcoming not easily promoted of cost, the present invention proposes a kind of method for detecting packet loss, system and media client, with can detect initial packet loss with low cost and and then realize packet loss and retransmit, and packet loss re-transmission can be carried out when switching flow media server.
The invention provides one and realize method for detecting packet loss, the method comprises:
Receiving terminal obtains the bag sequence number of the packet self received;
Receiving terminal obtains the total number information of giving out a contract for a project that transmitting terminal sends; And
Receiving terminal detects the packet of loss according to above-mentioned bag sequence number and total number information of giving out a contract for a project.
Preferably, above-mentioned method for detecting packet loss can have following features:
The total number information of giving out a contract for a project that above-mentioned receiving terminal obtains transmitting terminal transmission comprises:
In conversation procedure, when receiving terminal judges that the current data packet received is the packet received first time, from the IP bag carrying above-mentioned current data packet, extract total number information of giving out a contract for a project.
Preferably, above-mentioned method for detecting packet loss also can have following features:
According to above-mentioned bag sequence number and total number information of giving out a contract for a project, above-mentioned receiving terminal detects that the packet of loss comprises:
Above-mentioned receiving terminal is according to above-mentioned total number information generating reference sequence number of giving out a contract for a project;
Using the bag sequence number of current data packet as current sequence number; And
If current sequence number equals reference number add one, then determine packet loss does not occur, and using current sequence number as reference sequence number; If current sequence number is not equal to reference number and adds one, then determine all data-bag losts between reference number and current sequence number, and using current sequence number as reference sequence number.
Preferably, above-mentioned method for detecting packet loss also can have following features:
Extract total number information of giving out a contract for a project in wrapping from the IP carrying above-mentioned current data packet before, said method also comprises:
Transmitting terminal generates gives out a contract for a project total number information, and is carried at by above-mentioned total number information of giving out a contract for a project in the IP bag comprising packet and sends to receiving terminal.
Preferably, above-mentioned method for detecting packet loss also can have following features:
Generate at above-mentioned transmitting terminal total number information of giving out a contract for a project, and above-mentioned total number information of giving out a contract for a project is carried at sends in the process of receiving terminal in the IP bag comprising packet,
Transmitting terminal counts the packet sent, and is carried in above-mentioned IP bag by above-mentioned counting and sends to receiving terminal; Or
First packet sequence number is carried in above-mentioned IP bag and sends to receiving terminal by transmitting terminal.
Preferably, above-mentioned method for detecting packet loss also can have following features:
After the above-mentioned all data-bag losts detected between reference number and current sequence number, when there is multiple transmitting terminal and sending IP bag in turn, said method also comprises:
If during judging that data-bag lost occurs in session first, then extract the source IP address of current IP bag, and send repeat requests to the transmitting terminal of correspondence;
If judge data-bag lost be not occur in session first during, then extract the source IP address that current IP bag and last IP wrap; Judge that whether the two is from same transmitting terminal by the source IP address of more above-mentioned current IP bag and above-mentioned last IP bag, if, transmitting terminal then to correspondence sends repeat requests, if not, then extract the first packet sequence number of current data packet, and with above-mentioned first packet sequence number for boundary sends repeat requests respectively to the transmitting terminal of former and later two sessions.
The invention provides a kind of media client, above-mentioned media client comprises:
Sequence number acquisition module, for obtaining the bag sequence number of the packet self received;
Data obtaining module, for obtaining the total number information of giving out a contract for a project that transmitting terminal sends; And
Detection module, for detecting the packet of loss according to above-mentioned bag sequence number and total number information of giving out a contract for a project.
Preferably, above-mentioned media client can have following features:
Above-mentioned data obtaining module, is further used in conversation procedure, when the current data packet judging to receive is the packet received first time, from the IP bag carrying above-mentioned current data packet, extracts total number information of giving out a contract for a project;
Above-mentioned detection module, is further used for giving out a contract for a project described in basis total number information generating reference sequence number; Using the bag sequence number of current data packet as current sequence number; And if current sequence number equals reference number and adds one, then determine packet loss does not occur, and using current sequence number as with reference to sequence number; If current sequence number is not equal to reference number and adds one, then determine all data-bag losts between reference number and current sequence number; And using current sequence number as reference sequence number.
Preferably, above-mentioned media client also can have following features:
Above-mentioned media client also comprises:
Repeat request module, for when there is multiple media server and sending IP bag in turn, if during judging that data-bag lost occurs in session first, then extract the source IP address of current IP bag, and sends repeat requests to the transmitting terminal of correspondence; If judge data-bag lost be not occur in session first during, then extract the source IP address of current IP bag and last IP bag, judge that whether the two is from same transmitting terminal by the source IP address of more above-mentioned current IP bag and above-mentioned last IP bag, if, transmitting terminal then to correspondence sends repeat requests, if not, then extract the first packet sequence number of current data packet, and with above-mentioned first packet sequence number for boundary sends repeat requests respectively to the transmitting terminal of former and later two sessions.
Present invention also offers a kind of packet loss detection system, said system comprises above-mentioned media client and media server, wherein:
Above-mentioned media server, to give out a contract for a project total number information for generating, and is carried at by above-mentioned total number information of giving out a contract for a project in the IP bag comprising packet and sends to above-mentioned media client.
Method for detecting packet loss provided by the invention, system and media client, can detect initial packet loss and and then realize packet loss re-transmission with low cost and in time.
Accompanying drawing explanation
Fig. 1 is the flow chart that the present invention realizes packet loss re-transmission;
Fig. 2 is the flow chart of method for detecting packet loss of the present invention;
Fig. 3 is the structural representation of deployment model one of the present invention;
Fig. 4 is the RTP package head format of the embodiment of the present invention one;
Fig. 5 is RTP packet header of the present invention extended field general format schematic diagram;
Fig. 6 is the RTP packet header extended field form schematic diagram of the embodiment of the present invention one;
Fig. 7 is the structural representation of deployment model two of the present invention;
Fig. 8 is the RTP packet header extended field form of the embodiment of the present invention two;
Fig. 9 is media bag sequence number and the first packet sequence number field value of the embodiment of the present invention two;
Figure 10 is for transport layer session belonging to receiving terminal determination packet loss in the embodiment of the present invention two and send the flow chart of repeat requests;
Figure 11 is receiving terminal packet receiving situation schematic diagram in the embodiment of the present invention two;
Figure 12 is the structural representation of media server of the present invention;
Figure 13 is the structural representation of packet loss detection system of the present invention.
Embodiment
Be described in further detail below in conjunction with the enforcement of accompanying drawing to technical scheme:
For ease of hereinafter stating, be defined as follows term at this:
Media server and streaming media server, be equivalent to " media server (MediaServer) " in RFC 2326.
Application layer data is the data that application layer is transmitted by bottom IP link, and application layer data is the final purpose of this secondary data communication, and other supplementary increased for realizing this object does not belong to application layer data; Such as, in Stream Media Application, media data is application layer data, and the information for packet loss detection and re-transmission does not then belong to application layer data.
Transmitting terminal is the IP communication entity generating application layer data and externally send, and in this article in most cases, transmitting terminal is all a streaming media server.Transmitting terminal also will coordinate and realizes packet loss functions of retransmission.
Receiving terminal is the IP communication entity receiving application layer data, coordinates realize packet loss functions of retransmission with transmitting terminal.Transmitting terminal herein and receiving terminal distinguish according to the flow direction of application layer data, instead of the flow direction of certain concrete IP bag.
Transport layer session is the process of the bag that once transmits and receive data continuously in time occurred between specific senders and receiving terminal, the change of transmitting terminal or receiving terminal all can produce a new transport layer session, therefore in single application layer communication process, repeatedly transport layer session may occur.Herein subsequent section causes unlikely the occasion obscured, all by transport layer session referred to as " session ".
Starting stage packet loss refers to first packet loss that transmitting terminal sends in certain session or the event of all losing from first several bag continuous unwrapping the beginning, can referred to as " initial packet loss ".
The invention provides a kind of method for detecting packet loss, said method comprises:
Step one, receiving terminal obtain the bag sequence number of the packet self received;
Step 2, receiving terminal obtain the total number information of giving out a contract for a project that transmitting terminal sends; And
Step 3, receiving terminal detect the packet of loss according to above-mentioned bag sequence number and total number information of giving out a contract for a project.
After step 3, the method can also comprise: receiving terminal sends repeat requests to transmitting terminal, so that the packet of receiving terminal retransmission of lost.
As shown in Figure 1, for the present invention realizes the flow chart of packet loss re-transmission, the method comprises:
Step 101, transmitting terminal generate total number information of giving out a contract for a project;
" total number information of giving out a contract for a project " not corresponding specific concrete form, its aim is the bag sum that when receiving terminal can be allowed accurately to know that any one packet is sent out, transmitting terminal has sent, and includes but not limited to following form:
1) to give out a contract for a project counting: transmitting terminal counts the bag sent, and carries out initialization, after this monotonic increase during session initiation to this counting.According to the difference of specific implementation, counting of giving out a contract for a project can comprise the current packet that will send, and also can not comprise the current packet that will send.To give out a contract for a project counting as giving out a contract for a project total number information, and receiving terminal can directly obtain this information.
2) first packet sequence number: the sequence number of refering in particular to first bag that transmitting terminal sends, the sequence number of arbitrary bag is deducted first packet sequence number by receiving terminal, also can obtain corresponding total number information of giving out a contract for a project;
Step 102, transmitting terminal are given out a contract for a project total number information;
Adopt in-band method to give out a contract for a project total number information in the present embodiment, total number information of giving out a contract for a project directly adds in the IP bag carrying application layer data by transmitting terminal, sends to receiving terminal in the lump, to this, also has various mode, such as:
1) using the bag sent counting as bag sequence number, the bag sequence number that so receiving terminal receives is given out a contract for a project sum exactly;
2) in application layer data bag, increase a special field be used for transmission and give out a contract for a project total number information, now require that application layer data bag arranges sequence number field in addition;
Step 103, receiving terminal carry out packet loss detection according to the total number information of giving out a contract for a project received;
Receiving terminal knows the bag sum self received, and according to the total number information of giving out a contract for a project that transmitting terminal provides, just can calculate packet loss sum when each IP wraps received, then combine the bag sequence number received, can also obtain the accurate sequence number of each lost package; Although give out a contract for a project, the generation of total number information is different with send mode, and the specific implementation of said process is also different, and result can not change;
If there is starting stage packet loss, when receiving terminal successfully receives first IP bag, packet receiving adds up to 1, and the sum of giving out a contract for a project of correspondence is then greater than 1, both differences are the total quantity of initial packet loss, and the sequence number of this bag received being successively decreased is the sequence number of each lost package; As can be seen here, support of the present invention is to the detection of initial packet loss;
Step 104, receiving terminal send repeat requests;
Receiving terminal generates repeat requests bag according to above-mentioned packet loss detection result, and carry the serial number information of lost package in this repeat requests bag, receiving end is sent to transmitting terminal;
Step 105, transmitting terminal respond this repeat requests.
After transmitting terminal receives repeat requests, resend to receiving terminal according to the lost package of serial number information by correspondence.
Transmitting terminal of the present invention total number information of giving out a contract for a project is supplied to receiving terminal, the bag sum that when receiving terminal can accurately know that any one packet is sent out accordingly, transmitting terminal sends, thus can detect starting stage packet loss; Owing to adopting in-band method to carry total number information of giving out a contract for a project, give out a contract for a project total number information and application layer data together arrive receiving terminal, as long as have received bag just can detect initial packet loss, thus ensure that promptness.
As shown in Figure 2, be the flow chart of method for detecting packet loss of the present invention, this embodiment is described from reception side, wherein each packet carries bag sequence number field, for giving top priority to what is the most important, this embodiment does not consider the process of out of order bag and retransmission packet, and this packet loss detection process comprises:
Step 201, wait are until receive a packet;
Step 202, judge whether be first time receive packet, if so, then perform step 203, otherwise, turn to step 204;
Step 203, extract and to give out a contract for a project total number information, and generating reference sequence number accordingly;
Different according to the concrete form of total number information of giving out a contract for a project, the concrete process of this step is also different:
If using counting of giving out a contract for a project as total number information of giving out a contract for a project, then the sequence number of current bag (first bag namely received) is deducted corresponding counting of giving out a contract for a project (if transmitting terminal generates when giving out a contract for a project counting do not comprise current bag, then need to deduct 1 more on this basis), be reference number;
If using first packet sequence number as total number information of giving out a contract for a project, first packet sequence number is deducted 1 and can obtain reference number;
Step 204, extract the sequence number of current bag as current sequence number;
Step 205, judge whether current sequence number equals reference number+1, if so, then turns to step 207, otherwise, perform step 206;
Step 206, the current bag that receives and the previous sequence number receiving bag are discontinuous, are considered as losing by bag corresponding for middle vacancy sequence number, send repeat requests;
Step 207, using current sequence number as being preserved with reference to sequence number;
Step 208, judge whether media session terminates, if do not terminate, then turn to step 201, otherwise, terminate.
In above-mentioned steps 101, receiving terminal mainly uses total number information of giving out a contract for a project to carry out packet loss detection, and uses the reference number derived by total number information of giving out a contract for a project to carry out packet loss detection in Fig. 2, should be considered as a kind of equivalent form of value of technical solution of the present invention.
Visible, this embodiment by total number information of giving out a contract for a project to determine reference number and for the packet loss detection to this bag itself, namely to achieve the detection to starting stage packet loss.
As shown in Figure 3, for the structural representation of deployment model one of the present invention, in this embodiment, media server 31 provides unicast stream media services for media client 32, set up a signalling path 33 between the two, media server 31 sends media bag by RTP media channel 34 to media client 32, after media client 32 detects packet loss, sends repeat requests by repeat requests passage 35.
In practice, signalling path 33 and repeat requests passage 35 can adopt RTSP or RTCP Real-time Transport Control Protocol (RTCP), also any similar agreement based on TCP or UDP can be adopted, signalling path 33 also can serve as repeat requests passage 35 simultaneously, and now two channels merge into a channel physically.
This embodiment adopts Real-time Transport Protocol carrying application layer data, needs to expand the RTP packet format on RTP passage 34.As shown in Figure 4, be the RTP package head format of the embodiment of the present invention one, this RTP package head format is specified by RFC 3550, field X is extended field mark, and X=0 represents that this RTP packet header does not have extended field, and X=1 represents that this RTP packet header exists extended field, in the present embodiment, X field value is 1; The implication of all the other fields of RTP packet header can with reference to RFC 3550.
As shown in Figure 5, be RFC 3550 be the general format that RTP packet header extended field provides, when the extended field in RTP packet header is masked as 1, what closely follow at the afterbody of standard RTP packet headers is extended field, and each field meanings is as follows:
Determine that the value of (defined by profile) is determined by concrete application by application, RFC 3550 does not specify;
The length (length) of extended field, in units of 32 bits, does not comprise 32 bits at the place of length field own;
Because length field is in units of 32 bits, concrete extended head (header extension) length is necessary for the integral multiple of 32 bits.
Determine the RTP packet header extended field form for transmitting total number information of giving out a contract for a project according to the definition of Fig. 4 and Fig. 5, this RTP packet header extended field form can see Fig. 6, wherein:
Magic number (magic number) meets the extended field form that Fig. 5 specifies, value is 16 system number 0x12345678;
Length value is 1, has the expansion content of 32 bits after representing this field;
To give out a contract for a project counting (packet number), represent that transmitting terminal has comprised the packet sum that this RTP wraps in interior transmission since session start.
Streaming media server 31 is as transmitting terminal, the RTP bag of counting of giving out a contract for a project is carried according to the formatted output of Fig. 4 to Fig. 6, streaming media clients 32 performs the flow process shown in Fig. 2 as receiving terminal: after receiving first RTP bag, sequence number being deducted counting of giving out a contract for a project can to reference number initialize, thus the detection realized the whole packet loss events comprising initial packet loss, and initiate repeat requests when packet loss being detected.Streaming media server 31 response repeat requests, coordinates with streaming media clients 32 and realizes packet loss and retransmit.
As shown in Figure 7, be the structural representation of deployment model two of the present invention, wherein, control module 71 controls multiple streaming media server 72, sets up the communication of control channel 73 for control information between control module 71 and each streaming media server 72.Control module 71 is logically independent, but can merge with certain streaming media server 72 and dispose.Control module 71, streaming media server 72 and control channel 73 together form a virtual media server 74.
Set up signalling path 76 between media client 75 and control module 71, control module 71 controls each streaming media server 72 by control channel 73, and wheel flows through media channel 77 sends media code stream from RTP form to media client 75.Only there is a streaming media server 72 the same time at transmission code stream, and the code stream RTP bag sequence number that two streaming media servers 72 adjacent in transmitting time send keeps continuously.After media client 75 detects packet loss, send repeat requests by repeat requests passage 78 to media server 72.
This embodiment adopts Real-time Transport Protocol carrying application layer data, needs to expand the RTP bag on media channel 77, in this regulation still following RFC 3550 to expand RTP bag.As shown in Figure 8, be the RTP packet header extended field form of the embodiment of the present invention two, wherein:
First packet sequence number (first sequence number) represents the sequence number of first media bag, and Length represents length, and the length of first packet sequence number is zero; Particularly, often switch once, this field in all RTP bags that this all will send by the media server 72 being responsible for transmission code stream and the media server newly starting to give out a contract for a project be set to the sequence number of first media bag that this sends.
As shown in Figure 9, be media bag sequence number and the first packet sequence number field value of the embodiment of the present invention two, wherein:
91,92,93 and 94 streaming media server-A, streaming media server-B, streaming media server-C is represented until a series of participations of streaming media server-X media server of giving out a contract for a project in turn, corresponding to the streaming media server 72 in Fig. 7.The a series of blockage 95 in below represents the RTP bag that each streaming media server sends, and have two numerals in each RTP bag, numeral is above RTP sequence number, and numeral is below then first packet sequence number.To wrap 96, its bag sequence number is 152, and first packet sequence number is 151.Each RTP bag sequence number according to providing in Fig. 9 is analyzed known: it is first from top to bottom more from left to right that each RTP wraps in temporal sequencing.
In fig .9, the media that same streaming media server sends wrap in vertical direction is alignment (for ease of observing, having used double dot dash line differentiation in figure).Visible, at every turn when the media server being used for giving out a contract for a project and transmitting terminal switch, the first packet sequence number field value of RTP bag also changes thereupon---be set to the sequence number of first RTP bag that this media server this time sends.
Give out a contract for a project in turn owing to adopting multiple media server, therefore media services in embodiment two will be made up of multiple transport layer session, for Fig. 9, distinguish with RTP serial number range, sequence number belongs to respectively [1,100], [101,150], [151,280], [501,650], [651,701] bag, within the scope of [702,800] these closed intervals forms a session separately.This embodiment not only can detect initial packet loss, accurately can also determine the transport layer session belonging to packet loss, thus can send repeat requests to corresponding media server.
As shown in Figure 10, send the flow chart of repeat requests for transport layer session belonging to receiving terminal determination packet loss in the embodiment of the present invention two, the treatment mechanism being receiving terminal and finding after packet loss given here, for giving top priority to what is the most important, do not consider the process of out of order bag and retransmission packet in figure, this process comprises:
Step 1001, judge whether current packet loss is in the first time session in whole media services process, if so, then perform step 1002, otherwise, turn to step 1003;
Because the packet loss during first time session can not involve multiple transmitting terminal, therefore increase this determining step;
During step 1002, packet loss occur in first time session, take out the source IP address of current bag, the transmitting terminal to correspondence sends repeat requests, terminates;
Step 1003, between former and later two bags, take out source IP address, thus identify corresponding transmitting terminal;
Here " previous bag " refers to the last bag received of receiving terminal, and " a rear bag " is then the current bag received, and when packet loss occurs, the RTP sequence numbers of these two bags are discontinuous;
Step 1004, the source IP address wrapped by comparing former and later two judge that whether they are from same transmitting terminal, if so, then perform step 1005, otherwise turn to step 1006;
Step 1005, send repeat requests to transmitting terminal; Terminate;
When former and later two bags belong to same session, and during all from current transmitting terminal, then send repeat requests to this transmitting terminal;
Step 1006, take out the first packet sequence number field of current bag, the transmitting terminal as boundary respectively to former and later two sessions sends repeat requests; Terminate.
When former and later two bags are from different transmitting terminals, then belong to two different sessions, then need the first packet sequence number field taking out current bag, the transmitting terminal as boundary respectively to former and later two sessions sends repeat requests.
As shown in figure 11, be receiving terminal packet receiving situation schematic diagram in the embodiment of the present invention two, in description receiving terminal packet receiving process, the step 1006 in Figure 10 given and further illustrate:
Last media server 111 and current media server 112 represent former and later two transmitting terminals respectively, create two transport layer sessions, and corresponding RTP bag set is respectively 113 and 114, and the sequence number of 113 each RTP bags is respectively SN
pa ..., SN
pb ..., SN
pc, first packet sequence number is FSN
p, the sequence number of 114 each RTP bags is respectively SN
ca ..., SN
cb ..., SN
cc, first packet sequence number is FSN
ca, obviously, FSN
c=SN
ca=SN
pc+1.
The RTP bag set 113 and 114 that transmitting terminal 111 and 112 sends is through Internet Transmission, three kinds of different packet loss phenomenons may be there are in transmitting terminal when receiving, 115,116 and 117 (solid border represents that this bag is successfully received, and dotted border represents that this wraps in transmitting procedure and loses) in corresponding Figure 11.
Wherein, 115 sequence number is received at SN for receiving terminal
p(contain) and SN before C
call bags after B (containing), sequence number is at SN
pc and SN
csN is comprised between B
ca all loses at interior bag;
116 receive sequence number at SN for receiving terminal
p(contain) and SN before B
call bags after A (containing), sequence number is at SN
pb and SN
csN is comprised between A
pc all loses at interior bag;
117 receive sequence number at SN for receiving terminal
p(contain) and SN before B
call bags after B (containing), sequence number is at SN
pb and SN
csN is comprised between B
pc and SN
ca all loses at interior bag.
Table 1 gives the key variables value of step 1006 correspondence 115,116 and 117 3 kind of packet drop in Figure 10:
Key variables value table in table 1 different packet loss situation
| Situation is classified | " previous bag " sequence number | " current bag " sequence number | First packet sequence number |
| 115 | SN PC | SN CB | FSN C |
| 116 | SN PB | SN CA | FSN C |
| 117 | SN PB | SN CB | FSN C |
Corresponding processing mode (serial number range is closed interval) as shown in table 2:
Processing mode in table 2 different packet loss situation
| Situation is classified | Processing mode |
| 115 | To transmitting terminal 112 request retransmission sequence number [FSN C，SN CB-1] between bag |
| 116 | To transmitting terminal 111 request retransmission sequence number [SN PB+1，FSN C-1] bag between |
| 117 | To transmitting terminal 111 request retransmission sequence number [SN PB+1，FSN C-1] bag between is to transmitting terminal 112 request retransmission sequence number [FSN C，SN CB-1] between bag |
Method for detecting packet loss provided by the invention, initial packet loss can be detected with low cost and in time and and then realize packet loss and retransmit, and packet loss re-transmission can be carried out when switching flow media server.
As shown in figure 12, be the structural representation of media client of the present invention, above-mentioned media client comprises sequence number acquisition module 121, data obtaining module 122 and detection module 123; Wherein, sequence number acquisition module is for obtaining the bag sequence number of the packet self received; The total number information of giving out a contract for a project that data obtaining module sends for obtaining transmitting terminal; Detection module is used for the packet detecting loss according to above-mentioned bag sequence number and total number information of giving out a contract for a project.
Preferably, above-mentioned data obtaining module can be further used in conversation procedure, when the current data packet judging to receive is the packet received first time, from the IP bag carrying above-mentioned current data packet, extracts total number information of giving out a contract for a project; Above-mentioned detection module can be further used for according to above-mentioned total number information generating reference sequence number of giving out a contract for a project; Using the bag sequence number of current data packet as current sequence number; And if current sequence number equals reference number and adds one, then using current sequence number as with reference to sequence number; If current sequence number is not equal to reference number and adds one, then detect all data-bag losts between reference number and current sequence number.The process of the packet that the detection of this media client is lost can see Fig. 2, and the process realizing the data of retransmission of lost see Fig. 1, can repeat no more herein.
It should be noted that, for right open interval is closed on a left side between reference number and current sequence number, namely comprise reference number, but do not comprise the packet that current sequence number represents.
Preferably, above-mentioned media client can also comprise repeat request module 124, if this repeat request module then extracts the source IP address of current IP bag, and sends repeat requests to the transmitting terminal of correspondence during may be used for judging that data-bag lost occurs in session first; If judge data-bag lost be not occur in session first during, then extract the source IP address of current IP bag and last IP bag, judge that whether the two is from same transmitting terminal by the source IP address of more above-mentioned current IP bag and above-mentioned last IP bag, if, transmitting terminal then to correspondence sends repeat requests, if not, then extract the first packet sequence number of current data packet, and with above-mentioned first packet sequence number for boundary sends repeat requests respectively to the transmitting terminal of former and later two sessions.Specific implementation process see Figure 10, can repeat no more herein.
This media client can detect initial packet loss with low cost and in time and and then realize packet loss and retransmit, and packet loss re-transmission can be carried out when streaming media switch client.
As shown in figure 13, for the structural representation of packet loss detection system of the present invention, said system comprises media server 131 and media client 132, wherein, the structure of media client is identical with the structure of the media client shown in Figure 12 and have identical function, repeats no more herein.
Wherein, above-mentioned media server is used for generation and gives out a contract for a project total number information, and is carried at by above-mentioned total number information of giving out a contract for a project in the IP bag comprising packet and sends to above-mentioned media server.Particularly, the packet that above-mentioned media server can be further used for sending counts, and is carried in above-mentioned IP bag by above-mentioned counting and sends to receiving terminal; Or first packet sequence number is carried in above-mentioned IP bag sends to receiving terminal.
Mutual by between media server and media client of this packet loss detection system, initial packet loss can be detected with low cost and in time and and then realize packet loss and retransmit, and packet loss re-transmission can be carried out when switching flow media server.
The all or part of step that one of ordinary skill in the art will appreciate that in said method is carried out instruction related hardware by program and is completed, said procedure can be stored in computer-readable recording medium, as read-only memory, random access storage device, disk or CD etc.Alternatively, all or part of step of above-described embodiment also can use one or more integrated circuit to realize.Correspondingly, each module/unit in above-described embodiment can adopt the form of hardware to realize, and the form of software function module also can be adopted to realize.The present invention is not restricted to the combination of the hardware and software of any particular form.
Above embodiment only in order to technical scheme of the present invention and unrestricted to be described, only with reference to preferred embodiment to invention has been detailed description.Those of ordinary skill in the art should be appreciated that and can modify to technical scheme of the present invention or equivalent replacement, and does not depart from the spirit and scope of technical solution of the present invention, all should be encompassed in the middle of right of the present invention.
Claims (8)
1. a method for detecting packet loss, is characterized in that, described method comprises:
Receiving terminal obtains the bag sequence number of the packet self received;
Receiving terminal obtains the total number information of giving out a contract for a project that transmitting terminal sends; And
Receiving terminal detects the packet of loss according to described bag sequence number and total number information of giving out a contract for a project;
According to described bag sequence number and total number information of giving out a contract for a project, described receiving terminal detects that the packet of loss comprises:
Described receiving terminal according to described in give out a contract for a project total number information generating reference sequence number;
Using the bag sequence number of current data packet as current sequence number; And
If current sequence number equals reference number add one, then determine packet loss does not occur, and using current sequence number as reference sequence number; If current sequence number is not equal to reference number and adds one, then determine all data-bag losts between reference number and current sequence number, and using current sequence number as new reference number;
Described receiving terminal according to described in total number information generating reference sequence number of giving out a contract for a project comprise:
When using give out a contract for a project counting as give out a contract for a project total number information time, the sequence number of current bag is deducted corresponding counting of giving out a contract for a project as reference sequence number;
When using first packet sequence number as give out a contract for a project total number information time, first packet sequence number is deducted 1 as with reference to sequence number.
2. method for detecting packet loss according to claim 1, is characterized in that, the total number information of giving out a contract for a project that described receiving terminal obtains transmitting terminal transmission comprises:
In conversation procedure, when receiving terminal judges that the current data packet received is the packet received first time, from the IP bag carrying described current data packet, extract total number information of giving out a contract for a project.
3. method for detecting packet loss according to claim 2, is characterized in that, before described receiving terminal extracts total number information of giving out a contract for a project from the IP bag carrying described current data packet, described method also comprises:
Transmitting terminal generates gives out a contract for a project total number information, and is carried at by described total number information of giving out a contract for a project in the IP bag comprising packet and sends to receiving terminal.
4. method for detecting packet loss according to claim 3, is characterized in that, generates at described transmitting terminal total number information of giving out a contract for a project, and is carried at by described total number information of giving out a contract for a project and sends in the process of receiving terminal in the IP bag comprising packet,
Transmitting terminal counts the packet sent, and described counting is carried at described IP wrap in send to receiving terminal; Or
First packet sequence number is carried at during described IP wraps and sends to receiving terminal by transmitting terminal.
5. the method for detecting packet loss according to claim 3 or 4, is characterized in that, after the described all data-bag losts detected between reference number and current sequence number, if there is multiple transmitting terminal to send IP bag, then described method also comprises:
If during judging that data-bag lost occurs in session first, then extract the source IP address of current IP bag, and send repeat requests to the transmitting terminal of correspondence;
If judge data-bag lost be not occur in session first during, then extract the source IP address that current IP bag and last IP wrap; Judge that whether the two is from same transmitting terminal by the source IP address of more described current IP bag and described last IP bag, if, transmitting terminal then to correspondence sends repeat requests, if not, then extract the first packet sequence number of current data packet, and with described first packet sequence number for boundary sends repeat requests respectively to the transmitting terminal of former and later two sessions.
6. a media client, is characterized in that, described media client comprises:
Sequence number acquisition module, for obtaining the bag sequence number of the packet self received;
Data obtaining module, for obtaining the total number information of giving out a contract for a project that transmitting terminal sends; And
Detection module, for detecting the packet of loss according to described bag sequence number and total number information of giving out a contract for a project;
Described data obtaining module, is further used in conversation procedure, when the current data packet judging to receive is the packet received first time, from the IP bag carrying described current data packet, extracts total number information of giving out a contract for a project;
Described detection module, is further used for giving out a contract for a project described in basis total number information generating reference sequence number; Using the bag sequence number of current data packet as current sequence number; And if current sequence number equals reference number and adds one, then determine packet loss does not occur, and using current sequence number as with reference to sequence number; If current sequence number is not equal to reference number and adds one, then determine all data-bag losts between reference number and current sequence number, and using current sequence number as reference sequence number;
Described detection module be further used for according to described in total number information generating reference sequence number of giving out a contract for a project refer to:
When using give out a contract for a project counting as give out a contract for a project total number information time, the sequence number of current bag is deducted corresponding counting of giving out a contract for a project as reference sequence number;
When using first packet sequence number as give out a contract for a project total number information time, first packet sequence number is deducted 1 as with reference to sequence number.
7. media client according to claim 6, is characterized in that, described media client also comprises:
Repeat request module, for when there is multiple media server and sending IP bag in turn, if during judging that data-bag lost occurs in session first, then extract the source IP address of current IP bag, and sends repeat requests to the transmitting terminal of correspondence; If judge data-bag lost be not occur in session first during, then extract the source IP address of current IP bag and last IP bag, judge that whether the two is from same transmitting terminal by the source IP address of more described current IP bag and described last IP bag, if, transmitting terminal then to correspondence sends repeat requests, if not, then extract the first packet sequence number of current data packet, and with described first packet sequence number for boundary sends repeat requests respectively to the transmitting terminal of former and later two sessions.
8. comprise a packet loss detection system for the media client described in claim 6 or 7, it is characterized in that, described system also comprises media server, wherein:
Described media server, to give out a contract for a project total number information for generating, and is carried at by described total number information of giving out a contract for a project in the IP bag comprising packet and sends to described media client.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010599130.3A CN102546081B (en) | 2010-12-21 | 2010-12-21 | Method for detecting packet loss, system and media client |
| PCT/CN2011/079717 WO2012083737A1 (en) | 2010-12-21 | 2011-09-16 | Method, system and media client for detecting packet loss |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010599130.3A CN102546081B (en) | 2010-12-21 | 2010-12-21 | Method for detecting packet loss, system and media client |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102546081A CN102546081A (en) | 2012-07-04 |
| CN102546081B true CN102546081B (en) | 2015-09-16 |
Family
ID=46313116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010599130.3A Expired - Fee Related CN102546081B (en) | 2010-12-21 | 2010-12-21 | Method for detecting packet loss, system and media client |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102546081B (en) |
| WO (1) | WO2012083737A1 (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103684923B (en) * | 2013-12-24 | 2017-10-17 | 华为技术有限公司 | The method and the network equipment of a kind of packet loss measurement |
| CN105791154B (en) * | 2014-12-26 | 2019-05-24 | 浙江大华技术股份有限公司 | A kind of data transmission method and device based on UDP |
| WO2017133014A1 (en) * | 2016-02-06 | 2017-08-10 | 中国科学院计算技术研究所 | Method and system for network performance detection based on receiving end in tcp transmission stream |
| CN106209915A (en) * | 2016-08-31 | 2016-12-07 | 深圳聚点互动科技有限公司 | A kind of real time flow medium radio transmitting method and system thereof |
| CN109391605B (en) * | 2017-08-14 | 2021-01-26 | 杭州海康威视数字技术股份有限公司 | Data transmission method, device and system |
| CN109587096B (en) * | 2017-09-28 | 2021-09-14 | 中国移动通信集团浙江有限公司 | Method and device for identifying RTP tail packet loss |
| CN109842465A (en) * | 2017-11-24 | 2019-06-04 | 阿里巴巴集团控股有限公司 | Data transmission method, data end equipment |
| CN108173807B (en) * | 2017-11-28 | 2021-12-03 | 贵阳语玩科技有限公司 | Unified message sending and processing method and device |
| CN108377427B (en) * | 2018-01-29 | 2021-11-26 | 明博教育科技股份有限公司 | Real-time video transmission method and system |
| CN109586931B (en) * | 2018-10-18 | 2021-01-15 | 招商证券股份有限公司 | Multicast method and terminal equipment |
| CN109889314A (en) * | 2018-12-29 | 2019-06-14 | 视联动力信息技术股份有限公司 | Information processing method and device |
| CN109889913B (en) * | 2019-03-21 | 2021-07-20 | 南京威翔科技有限公司 | Method for analyzing data packet loss in network environment |
| CN110677314B (en) * | 2019-08-29 | 2021-09-21 | 视联动力信息技术股份有限公司 | Network interface testing method, system, electronic device and storage medium |
| CN110855626B (en) * | 2019-10-18 | 2022-03-11 | 北京字节跳动网络技术有限公司 | Electronic whiteboard packet loss processing method, system, medium and electronic equipment |
| CN111277576A (en) * | 2020-01-14 | 2020-06-12 | 广州华多网络科技有限公司 | Drawing data display method and device, computer equipment and storage medium |
| CN112637162A (en) * | 2020-12-14 | 2021-04-09 | 上海金仕达软件科技有限公司 | UDP (user Datagram protocol) data packet processing method and device |
| CN113473111A (en) * | 2021-05-29 | 2021-10-01 | 江苏网进科技股份有限公司 | Method for detecting video blocking phenomenon based on rtsp protocol |
| CN113259062B (en) * | 2021-05-31 | 2021-10-29 | 恒生电子股份有限公司 | Method, device, readable medium and equipment for packet loss retransmission |
| CN113542061B (en) * | 2021-07-08 | 2023-03-31 | 阳光电源股份有限公司 | Data transmission control method and related device |
| CN113364814B (en) * | 2021-08-10 | 2021-11-30 | 北京翔东智能科技有限公司 | Anti-weak network transmission method based on RTP |
| CN114422397A (en) * | 2021-12-27 | 2022-04-29 | 中国电信股份有限公司 | Streaming media channel monitoring method and device, electronic equipment and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1525693A (en) * | 2003-02-26 | 2004-09-01 | 华为技术有限公司 | Statistic method for packet dropping number of communication device |
| CN101662418A (en) * | 2008-08-26 | 2010-03-03 | 华为技术有限公司 | Detecting method and terminal of file transmission |
| CN101686184A (en) * | 2008-09-23 | 2010-03-31 | 中兴通讯股份有限公司 | Processing method of service data packet loss of synchronous network of multimedia broadcast multicast service |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100334867C (en) * | 2002-11-20 | 2007-08-29 | 华为技术有限公司 | Method for assuring reliability of data package transmission in simple network management protocol |
| CN101296166B (en) * | 2007-04-29 | 2012-05-09 | 中兴通讯股份有限公司 | Method for measuring multimedia data based on index |
| CN101123584B (en) * | 2007-05-21 | 2010-11-10 | 华为技术有限公司 | A method and device for measuring packet drop of IP link |
| CN101159520A (en) * | 2007-10-29 | 2008-04-09 | 中兴通讯股份有限公司 | Data transmission method |
-
2010
- 2010-12-21 CN CN201010599130.3A patent/CN102546081B/en not_active Expired - Fee Related
-
2011
- 2011-09-16 WO PCT/CN2011/079717 patent/WO2012083737A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1525693A (en) * | 2003-02-26 | 2004-09-01 | 华为技术有限公司 | Statistic method for packet dropping number of communication device |
| CN101662418A (en) * | 2008-08-26 | 2010-03-03 | 华为技术有限公司 | Detecting method and terminal of file transmission |
| CN101686184A (en) * | 2008-09-23 | 2010-03-31 | 中兴通讯股份有限公司 | Processing method of service data packet loss of synchronous network of multimedia broadcast multicast service |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2012083737A1 (en) | 2012-06-28 |
| CN102546081A (en) | 2012-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102546081B (en) | Method for detecting packet loss, system and media client | |
| JP4550883B2 (en) | Transmission of parameters for broadcast / multicast sessions via communication protocols | |
| EP2894831B1 (en) | Transport mechanisms for dynamic rich media scenes | |
| JP4000905B2 (en) | Information processing system and method, information processing apparatus and method, recording medium, and program | |
| EP3090523B1 (en) | Content delivery | |
| CN101656747A (en) | Method and system for transmitting streaming media data | |
| CN103944834B (en) | Audio and video transmission control method and system | |
| JP6523260B2 (en) | How to subscribe to streams from multicast clients | |
| TW201424417A (en) | Systems and methods for improved data throughput in communications networks | |
| US10027496B2 (en) | Method for distributing identifiers of multicast sources | |
| US10361937B2 (en) | Method and apparatus for detecting operating status of node | |
| CN103329558B (en) | The method and server of Fast Channel change are realized in unicast multicast IPTV networks | |
| EP1914933A1 (en) | Method and apparatus for retransmission request reduction in a network | |
| JP4600513B2 (en) | Data transmission apparatus, transmission rate control method, and program | |
| CN105657374A (en) | Self-adaptive multi-network link real-time video transmission control system | |
| CN103220585B (en) | A kind of network video transmission method supporting QoS | |
| CN102845008B (en) | Message redundancy method of adjustment, relevant device and network system | |
| Qi et al. | Soft handover mechanism based on RTP parallel transmission for mobile IPTV services | |
| CN111385241B (en) | Method, device and system for repairing lost packet of multimedia data and readable storage medium | |
| CN100499659C (en) | Receiving report transmission time setting method in the communication network | |
| KR101107325B1 (en) | Method and System for Measuring Quality by Network Interval for Real-time Multimedia Service | |
| Chayat et al. | Applying deterministic feedback suppression to reliable multicasting protocols | |
| Bonastre et al. | Specification and Implementation of Multicast Congestion Control Stair Protocol: Use Case with Multimedia Ubiquitous Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150916 Termination date: 20161221 |