CN1645944A - Method for testing audio and video frequency step falling out of audio and video frequency coding-decoding system - Google Patents

Method for testing audio and video frequency step falling out of audio and video frequency coding-decoding system Download PDF

Info

Publication number
CN1645944A
CN1645944A CN 200410061320 CN200410061320A CN1645944A CN 1645944 A CN1645944 A CN 1645944A CN 200410061320 CN200410061320 CN 200410061320 CN 200410061320 A CN200410061320 A CN 200410061320A CN 1645944 A CN1645944 A CN 1645944A
Authority
CN
China
Prior art keywords
video
audio
frame number
frame
audio signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200410061320
Other languages
Chinese (zh)
Other versions
CN100496133C (en
Inventor
胡瑞敏
韩镇
刘琼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CNB2004100613204A priority Critical patent/CN100496133C/en
Publication of CN1645944A publication Critical patent/CN1645944A/en
Application granted granted Critical
Publication of CN100496133C publication Critical patent/CN100496133C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

The invention includes following steps: 1) each frame of image in video sequential file whose video frame rate is given is made frame number identification; 2) the sound signal is generated in audio flow in video sequential file, and the image frame number at the time point generating sound signal is recorded; 3) the audio and video file are inputted into video and audio coding system under test, and are played back; 4) the image frame number at the playing time point of sound signal is recorded; 5) through the formula: delta Tn = delta n/F, the out of alignment time of sound is detected; delta Tn is out of alignment time of sound; delta n is the difference between two frame signals; F is video frame rate of the video sequential file.

Description

A kind of method of test tone video coding and decoding system audio frequency and video step-out
Technical field
The present invention relates to the multimedia messages process field, particularly the method for test tone video coding and decoding system audio frequency and video step-out.
Background technology
Studies show that, when the out of alignment time of audio frequency and video surpasses 80 milliseconds, will spectators' subjective sensation be impacted.Therefore, the test of audio frequency and video step-out is very necessary for the performance of estimating a multi-media decoding and encoding system.
The method of existing test tone video coding and decoding system audio frequency and video step-out mainly contains two kinds.A kind of objective examination's method that is based on internal indicator, another kind is the subjective testing method.Objective examination's method based on internal indicator is exactly to utilize the audio-visual synchronization model of formulating in the coding international standard (as system reference clock SCR, Presentation Time Stamp PTS, program clock reference PCR etc.) to calculate the audio frequency and video out of alignment time.For example in MPEG-2, set up system reference clock SCR in decoding end earlier exactly, its initial value is obtained by first effective Presentation Time Stamp PTS.When a video access unit VAU (audio access unit AAU) shows, with SCR and corresponding PTS relatively, if difference beyond certain limit, is then thought the audio/video decoding step-out.Objective examination's method all has introduction in " Shanghai Communications University's journal " the 32nd the 9th phase of volume " audio-visual synchronization and real-time implementation thereof in the MPEG-2 decoding ", " infrared and laser engineering " the 29th volume the 5th phase articles such as " temporal information in the mpeg 2 transport stream and audio-visual synchronization ".The subjective testing rule is under the synchronous prerequisite of the audio frequency and video inlet flow that guarantees multimedia system, selects in the audio frequency and video output stream several time point positions to carry out playback at random, utilizes visual and ear to listen the audio frequency and video out of alignment time of comprehensive estimating system.
Though can calculate the audio frequency and video out of alignment time of system based on objective examination's method of internal indicator, it need utilize the audio-visual synchronization model of formulating in the coding international standard, and these need the frame number of the every two field picture of video more accurately.Existing video frame number label method mainly is semantic label method, semantic label method is exactly to utilize semanteme in the video flowing of various coding international standard defineds directly to identify frame number (as the TR in H.263) or identify frame number indirectly (as the time_code among the MPEG-4, modulo_time_base, vop_time_increment), these semantemes are encoded with video data together as the header information of video flowing, decoding these semantemes afterwards will be lost and can't be obtained, therefore from outputting video streams, can't obtain frame number, just can obtain frame number after must obtaining these semantic informations to the encoded video streams decoding of internal system earlier, promptly must be by the internal system code stream is resolved just and can be obtained, therefore require test side to have tested audio/video encoding/decoding internal system code stream is carried out operation permission, that is to say that tested audio/video encoding/decoding system must be a white box system.And this condition all is difficult to satisfy under many circumstances, and for example when the multimedia system product was tested, system under test (SUT) was actually a black-box system, and its internal processes is maintained secrecy.This moment, the objective examination's method based on internal indicator just can not be suitable for.Though the subjective testing method do not require that test side has the internal system program is carried out operation permission, dialogue box system and black-box system all are suitable for, its test data inaccuracy, and randomness is big.
Summary of the invention
Technical problem to be solved by this invention is: the method that a kind of test tone video coding and decoding system audio frequency and video step-out is provided, it is test tone video out of alignment time accurately, and not requiring that test side has carries out operation permission to tested audio/video encoding/decoding internal system program, and dialogue box system and black-box system are all applicable.
The present invention solves the problems of the technologies described above the technical scheme that is adopted:
A kind of method of test tone video coding and decoding system audio frequency and video step-out, it may further comprise the steps:
(1) each two field picture in the video sequence file of one section known video frame per second is carried out the frame number sign;
(2) in the audio stream of the video sequence file that step (1) obtains, produce audio signal, and the image frame number on the time point of record generation audio signal;
(3) audio-video document that step (2) is obtained is input to audio/video encoding/decoding system to be measured, and plays;
(4) image frame number on the recorde audio signal play time;
(5) pass through formula: Δ Tn=Δ n/F records the audio frequency and video out of alignment time; In the formula: Δ Tn is the audio frequency and video out of alignment time, and Δ n is the difference between two image frame number, and F is the video frame rate of video sequence file.
In the such scheme, a plurality of audio signals that produce in audio stream obtain a plurality of audio frequency and video out of alignment times, pass through formula: Δ T=∑ Δ Tn/N records the average out of alignment time of audio frequency and video; In the formula: Δ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
In the such scheme, the method for the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
In the such scheme, the method for the image frame number on the recorde audio signal play time is: utilize picture pick-up device that image and the sound of playing is carried out record.
In the such scheme, the audio signal that produces on audio stream is a simple signal.
The inventive method is compared with the objective examination's method based on internal indicator, because tested audio/video encoding/decoding system is in video encoding-decoding process, can not change in frame number sign in each two field picture and the audio stream and produce audio signal, therefore, do not need tested audio/video encoding/decoding internal system code stream is resolved, need not utilize the audio-visual synchronization model of formulating in the coding international standard to calculate the audio frequency and video out of alignment time, not requiring that test side has carries out operation permission to tested audio/video encoding/decoding internal system program, and dialogue box system and black-box system are all applicable.
The inventive method is compared with the subjective testing method, and the present invention is the test tone video time accurately, not influenced by subjective factor.
Description of drawings
Fig. 1 is the annexation block diagram between each equipment of embodiment of the present invention
Fig. 2 is the image schematic diagram after the frame number sign
Fig. 3 is image frame number sign software flow pattern
Fig. 4 detects software flow pattern for image frame number
Fig. 5 is input audio/video flow schematic diagram
Fig. 6 is output audio/video flow schematic diagram
Embodiment
Embodiment of the present invention: annexation such as Fig. 1 between each testing equipment and the audio/video encoding/decoding system to be measured, method of testing may further comprise the steps:
(1) each two field picture in one section known video frame per second video sequence file that was 25 frame/seconds is carried out the frame number sign, concrete grammar is:
Utilize zone that computer forms the first row block of pixels of each two field picture in the video sequence (CIF form) as identified areas; In this identified areas, the brightness of interior all pixels of block of pixels and chromatic value all are changed to black, and (brightness value is 255, chromatic value is 0) or white (brightness value is 0, and chromatic value is 0), each block of pixels is combined into a binary number respectively according to each two field picture order in the video sequence.
Image as shown in Figure 2, its resolution are 352 * 288.The zone that 22 block of pixels of first row (each block of pixels is 16 * 16 picture element matrixs herein) on image top are formed is as the two-value identified areas, the brightness of 16 * 16 pixels of each block of pixels and chromatic value all are changed to black, and (brightness value is 255, chromatic value is 0) or white (brightness value is 0, chromatic value is 0), black expression 1 represents 0 in vain.22 block of pixels of such first row according to the left side for high-order the right for the order of low level just can be combined into the binary number of one 22 bit, be illustrated in figure 2 as 0000000000110110010100, decimalize is 3476.If frame number increases progressively from the 0th frame, then this two field picture frame number is exactly the 3476th frame.
The workflow (as Fig. 3) of image frame number sign software.At first import video sequence to be identified, the initial value with frame number F is changed to 0 then, shows that promptly the frame number counting is since 0.Then transfer frame number F to binary number, prepare for carrying out the two-value sign.Next begin to read the data of the 1st two field picture in the video sequence, and the initial value of block of pixels number is changed to 1.Because it is good that the two-value identified areas of each two field picture and block of pixels are predefineds, so can read the data of the 1st block of pixels in the present frame two-value identified areas with that.Get the 1st that has been converted into binary frame number then, if this position is 0, thus just the brightness and the chromatic value of pixel in the current pixel piece is changed to white (brightness value is 0, and chromatic value is 0), otherwise just be changed to black (brightness value is 255, and chromatic value is 0).It is to be noted: the 1st of binary frame number is meant highest order or lowest order, with the 1st block of pixels sign be that highest order or lowest order are consistent.After the 1st block of pixels finished sign, just judge that the current pixel piece is last block of pixels.Get next block of pixels data if not attending school, and block of pixels number is added up 1, read frame number then the 2nd, carry out new judgement and sign.The rest may be inferred, is last block of pixels up to the current pixel piece, also is that the two-value frame number of present frame has identified.Just judge that then present frame is a last frame.If not, attend school the data of taking off a two field picture, frame number adds 1, repeats the sign operation of the 1st two field picture.The rest may be inferred, is last frame up to present frame, also is that the frame number of video sequence has identified, just quit a program, output promptly be the video sequence that each frame has all identified.
Image frame number detects workflow such as Fig. 4 of software.At first import video sequence to be identified, next begin to read the data of the 1st two field picture in the video sequence.Because it is good that the two-value identified areas of each two field picture and block of pixels are predefineds, so can read the data of the 1st block of pixels in the present frame two-value identified areas with that.If the pixel value in the current pixel piece is white (brightness value is 0, and chromatic value is 0), just the 1st with binary system frame number F is designated as 0; Otherwise just the 1st with binary system frame number F is designated as 1.Judge that then the current pixel piece is last block of pixels.Whether, then according to this block of pixels pixel value be white (brightness value be 0, chromatic value be 0), be designated as 0 or 1 with the 2nd of binary system frame number F is corresponding if getting next block of pixels data if not attending school, and block of pixels number is added up 1.The rest may be inferred, is last block of pixels up to the current pixel piece, also is that the two-value frame number sign of present frame has detected.At this moment also just having obtained all value of binary system frame number F, transferred it to decimal number then, is exactly the frame number of present frame.After the frame number of present frame has detected, just judge that present frame is a last frame.If not, attend school the data of getting next frame, repeat the detecting operation of the 1st two field picture.The rest may be inferred, is last frame up to present frame, also is that the frame number of video sequence has detected, just quit a program, output promptly be each frame video frame number.
(2) as Fig. 5, in the audio stream of the video sequence file that step (1) obtains, produce 2 audio signals, audio signal is a simple signal, and writes down the image frame number (the 2nd frame and n frame) on each time point that produces audio signal.
So just set up a kind of relation one to one between simple signal and frame of video, this relation is exactly the foundation of the inventive method test tone video step-out.If coding/decoding system does not cause the audio frequency and video step-out, so this corresponding relation just can not change in the output audio/video flow; Otherwise this corresponding relation will change, and that is to say that the simple signal of output audio no longer is to import the simple signal of audio frequency at pairing that frame of input video at pairing that frame of output video, and both frame numbers can be different.
(3) audio-video document that step (2) is obtained is input to audio/video encoding/decoding system to be measured, and plays;
(4) image frame number on the recorde audio signal play time;
The method of the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
When the audio frequency monitoring equipment monitors a simple signal, send instruction to capture apparatus, take corresponding down frame of video, should guarantee during shooting that two-value video frame number is identified in the picture.Deposit the frame of video of taking in computer then, detect its frame number, compare, calculate the audio frequency and video out of alignment time on this time point with the pairing frame number of this single-frequency audio signal in the input video stream; The rest may be inferred, monitors other single-frequency audio signals, tries to achieve the mean value of the audio frequency and video out of alignment time on all N time point at last.
The simple signal of output audio is pairing video frame number in output video, then with input video in pairing frame number subtract each other that to obtain frame poor.If the frame difference is 0, represent that then corresponding relation does not change, the audio frequency and video out of alignment time is 0; If the frame difference is Δ n, then change has taken place in corresponding relation, step-out occurred, and the time of step-out is exactly the shared time of frame difference Δ n.The frame per second of supposing input video is F frame/second, and the time that each frame occupied is exactly 1/F second so, so out of alignment time Δ Tn=Δ n/F second.For example in the input audio/video flow of Fig. 5 the 1st single-frequency audio signal correspondence be the 2nd frame, the 2nd single-frequency audio signal correspondence be the n frame; And (as Fig. 6) detect the 1st single-frequency audio signal correspondence in the output audio/video flow is the 2nd frame, but the 2nd single-frequency audio signal correspondence is the n-1 frame.Just explanation is on the 1st time point for this, and the frame difference is 0, does not have the audio frequency and video step-out; On the 2nd time point, the frame difference is 1.
(5) pass through formula: Δ Tn=Δ n/F records the audio frequency and video out of alignment time; In the formula: Δ Tn is the audio frequency and video out of alignment time, and Δ n is the difference between two image frame number, and F is the video frame rate of video sequence file.
Because frame per second was 25 frame/seconds, calculated: the out of alignment time of audio frequency and video=1 frame/(25 frame/second)=0.04 second.
Pass through formula: Δ T=∑ Δ Tn/N records the average out of alignment time of audio frequency and video; In the formula: Δ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
Calculate: the out of alignment time of the average out of alignment time=audio frequency and video of audio frequency and video/2=0.02 second.
In order to avoid error as far as possible, the number that produces audio signal in the audio stream should be greatly, and distributing also should be even.
The inventive method can also utilize picture pick-up device that image and the sound of playing is carried out record.

Claims (5)

1, a kind of method of test tone video coding and decoding system audio frequency and video step-out, it is characterized in that: it may further comprise the steps:
(1) each two field picture in the video sequence file of one section known video frame per second is carried out the frame number sign;
(2) in the audio stream of the video sequence file that step (1) obtains, produce audio signal, and the image frame number on the time point of record generation audio signal;
(3) audio-video document that step (2) is obtained is input to audio/video encoding/decoding system to be measured, and plays;
(4) image frame number on the recorde audio signal play time;
(5) pass through formula: Δ Tn=Δ n/F records the audio frequency and video out of alignment time; In the formula: Δ Tn is the audio frequency and video out of alignment time, and Δ n is the difference between two image frame number, and F is the video frame rate of video sequence file.
2, the method for claim 1 is characterized in that: a plurality of audio signals that in audio stream, produce, obtain a plurality of audio frequency and video out of alignment times, and pass through formula: Δ T=∑ Δ Tn/N records the average out of alignment time of audio frequency and video; In the formula: Δ T is the average out of alignment time of audio frequency and video, and N is the number that produces audio signal in audio stream.
3, the method for claim 1, it is characterized in that: the method for the image frame number on the recorde audio signal play time is: utilize the audio frequency monitoring equipment to detect audio signal, and control image frame number on the capture apparatus recorde audio signal play time by the audio frequency monitoring equipment.
4, the method for claim 1 is characterized in that: the method for the image frame number on the recorde audio signal play time is: utilize picture pick-up device that image and the sound of playing is carried out record.
5, the method for claim 1 is characterized in that: the audio signal that produces on audio stream is a simple signal.
CNB2004100613204A 2004-12-13 2004-12-13 Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system Expired - Fee Related CN100496133C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2004100613204A CN100496133C (en) 2004-12-13 2004-12-13 Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2004100613204A CN100496133C (en) 2004-12-13 2004-12-13 Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system

Publications (2)

Publication Number Publication Date
CN1645944A true CN1645944A (en) 2005-07-27
CN100496133C CN100496133C (en) 2009-06-03

Family

ID=34868808

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2004100613204A Expired - Fee Related CN100496133C (en) 2004-12-13 2004-12-13 Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system

Country Status (1)

Country Link
CN (1) CN100496133C (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101765027A (en) * 2009-10-20 2010-06-30 大连捷成实业发展有限公司 Method for detecting time-domain weighted digital video static frame
CN101902660A (en) * 2010-08-13 2010-12-01 深圳创维-Rgb电子有限公司 Method for testing video output signals of television
CN101616331B (en) * 2009-07-27 2011-07-20 北京汉邦高科数字技术有限公司 Method for testing video frequency frame rate and audio-video synchronous performance
CN103561260A (en) * 2013-10-31 2014-02-05 北京邮电大学 Method and device for measuring lost step between network side audio and video streams based on RTSP
CN104422828A (en) * 2013-08-29 2015-03-18 是德科技股份有限公司 Method and Apparatus for Estimating the Noise Introduced by a Device
CN105657512A (en) * 2016-01-29 2016-06-08 四川长虹电器股份有限公司 Method for monitoring sound-picture asynchronization of digital video
CN105657412A (en) * 2015-12-31 2016-06-08 公安部第三研究所 Video frame rate test method
CN106358039A (en) * 2016-09-07 2017-01-25 深圳Tcl数字技术有限公司 Sound and image synchronization test method and device
WO2018040669A1 (en) * 2016-08-30 2018-03-08 京东方科技集团股份有限公司 Method and apparatus for synchronizing audio and video signal
CN108307191A (en) * 2018-01-03 2018-07-20 西安万像电子科技有限公司 The alignment schemes and device of image data
CN113207009A (en) * 2021-05-11 2021-08-03 中国工商银行股份有限公司 Method, device and system for testing audio-video interaction system with out-of-step sound and picture
CN114071134A (en) * 2022-01-13 2022-02-18 荣耀终端有限公司 Sound-picture synchronization detection method, sound-picture synchronization generation method, electronic equipment and storage medium

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101616331B (en) * 2009-07-27 2011-07-20 北京汉邦高科数字技术有限公司 Method for testing video frequency frame rate and audio-video synchronous performance
CN101765027A (en) * 2009-10-20 2010-06-30 大连捷成实业发展有限公司 Method for detecting time-domain weighted digital video static frame
CN101765027B (en) * 2009-10-20 2013-01-09 大连捷成实业发展有限公司 Method for detecting time-domain weighted digital video static frame
CN101902660A (en) * 2010-08-13 2010-12-01 深圳创维-Rgb电子有限公司 Method for testing video output signals of television
CN101902660B (en) * 2010-08-13 2013-01-23 深圳创维-Rgb电子有限公司 Method for testing video output signals of television
US10120008B2 (en) 2013-08-29 2018-11-06 Keysight Technologies, Inc. Method and apparatus for estimating the noise introduced by a device
CN104422828A (en) * 2013-08-29 2015-03-18 是德科技股份有限公司 Method and Apparatus for Estimating the Noise Introduced by a Device
CN103561260A (en) * 2013-10-31 2014-02-05 北京邮电大学 Method and device for measuring lost step between network side audio and video streams based on RTSP
CN105657412A (en) * 2015-12-31 2016-06-08 公安部第三研究所 Video frame rate test method
CN105657512A (en) * 2016-01-29 2016-06-08 四川长虹电器股份有限公司 Method for monitoring sound-picture asynchronization of digital video
WO2018040669A1 (en) * 2016-08-30 2018-03-08 京东方科技集团股份有限公司 Method and apparatus for synchronizing audio and video signal
CN106358039A (en) * 2016-09-07 2017-01-25 深圳Tcl数字技术有限公司 Sound and image synchronization test method and device
CN106358039B (en) * 2016-09-07 2019-02-01 深圳Tcl数字技术有限公司 Sound draws synchronous detecting method and device
CN108307191A (en) * 2018-01-03 2018-07-20 西安万像电子科技有限公司 The alignment schemes and device of image data
CN113207009A (en) * 2021-05-11 2021-08-03 中国工商银行股份有限公司 Method, device and system for testing audio-video interaction system with out-of-step sound and picture
CN113207009B (en) * 2021-05-11 2022-11-29 中国工商银行股份有限公司 Method, device and system for testing audio-video interaction system with out-of-step sound and picture
CN114071134A (en) * 2022-01-13 2022-02-18 荣耀终端有限公司 Sound-picture synchronization detection method, sound-picture synchronization generation method, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN100496133C (en) 2009-06-03

Similar Documents

Publication Publication Date Title
US8379735B2 (en) Automatic video glitch detection and audio-video synchronization assessment
US9906782B2 (en) Source agnostic audio/visual analysis framework
CN102598688B (en) Streaming encoded video data
KR100884134B1 (en) Methods for timecoding media samples
US20070009049A1 (en) Compressed Timing Indicators for Media Samples
CN100496133C (en) Method for testing audio and video frequency out of step of audio and video frequency coding-decoding system
CN101616331B (en) Method for testing video frequency frame rate and audio-video synchronous performance
CN103535027B (en) The method for handling the sequence of encoded video frame
CN1859584A (en) Video frequency broadcast quality detecting method for medium broadcast terminal device
CN104092920A (en) Audio and video synchronizing method
US8045836B2 (en) System and method for recording high frame rate video, replaying slow-motion and replaying normal speed with audio-video synchronization
CN102026017A (en) Efficient test method for video decoding
CN1320926A (en) Method and device for recording or reproducing on storage medium
EP2429136A1 (en) Method and apparatus for carrying transport stream
CN101827202A (en) Image processing equipment, image processing method and program
CN107483916A (en) The control method of audio frequency and video archival quality detecting system
CN1444398A (en) Video stream index playback system based on key frame
CN109040818A (en) Audio and video synchronization method, storage medium, electronic equipment and system when live streaming
CA2723777A1 (en) Playlist processing
WO2021171259A1 (en) Real-time latency measurement of video streams
CN113055711B (en) Audio and video synchronous detection method and detection system thereof
CN1207922C (en) Method and system for measuring accuracy of video/audio output synchronization, and analysis means
CN100459686C (en) Video reproducing apparatus
CN107968942B (en) Method and system for measuring audio and video time difference of live broadcast platform
US6912011B2 (en) Method and system for measuring audio and video synchronization error of audio/video encoder system and analyzing tool thereof

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090603

Termination date: 20131213