CN100469108C - Real time video image transmission technology - Google Patents

Real time video image transmission technology Download PDF

Info

Publication number
CN100469108C
CN100469108C CN 200610114181 CN200610114181A CN100469108C CN 100469108 C CN100469108 C CN 100469108C CN 200610114181 CN200610114181 CN 200610114181 CN 200610114181 A CN200610114181 A CN 200610114181A CN 100469108 C CN100469108 C CN 100469108C
Authority
CN
China
Prior art keywords
information
video
image
background
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 200610114181
Other languages
Chinese (zh)
Other versions
CN1946144A (en
Inventor
李博航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN 200610114181 priority Critical patent/CN100469108C/en
Publication of CN1946144A publication Critical patent/CN1946144A/en
Application granted granted Critical
Publication of CN100469108C publication Critical patent/CN100469108C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

This invention puts forward a real time transmission technology of video images, which divides a video information region in a video scene into an unchanged background information region and a local region generating changed information, and only one time of collection, storage and transmission is needed for the collection of unchanged information in a video conference system, yet continuous video flows are generated to the local region generating changed information and positions and sizes of the local region are generated as the parameters to collect, store and transmit it and only one time is needed. Complete scene information is re-set up based on the received background information, continuous video flows of the local region generating changed information and the position and size parameters, which can reduce information transmission in the video and image transmission system.

Description

Real-time video, image transfer method
Technical field
The present invention relates to utilize image digitazation technology and Image Information Processing technology is indeclinable background information zone and the regional area that produces information change with the video information area dividing in the video scene, different information areas adopts different processing modes with the transinformation in the effective minimizing video conferencing system, is the effective technology of realizing the real-time video conference system.Especially in the video scene of video conferencing system the fewer environment in moving region can reduce greatly video information data volume, improve the transmission speed of video information in network, belong to technical field of image information processing and information transmission technology field.
Background technology
Current society, the collection of multimedia messages, transmission, reception are becoming the main information communication way, for through the propagation of the image information of digitized processing and Video stream information because its intuitive, easy to understand become people's amusement, information interchange and communication important form.But because the data of image information amount is very big in the digital collection method of present image and Video stream information, and the network bandwidth that is used to transmit is always limited, can not satisfy the amount of image information of increase day by day and the propagation demand of Video stream information amount, become the major obstacle that realizes multimedia information dissemination.In present video, image delivering system and since the big problems affect of amount of video information the real-time of system; Some videos, image delivering system improve real-time by video image spatial resolution and color resolution in the reduction system.
Be to solve the propagation problem of image data information and Video stream information, people have carried out big quantity research at aspects such as the encoding and decoding of information, Information Compression, information modulation, and have released various algorithms and realization standard.But how people reduce the research of amount of information and few for how in information gathering with digitized the time.The present image information and the collection of Video stream information and digitlization mainly are that image taking sensor is the basis, full detail in the scene is generated a width of cloth width of cloth image panorama information by imageing sensor, then panorama information is carried out storing after the digitized processing.Special in the generation of Video stream information, by the per second 25 two field picture requirements of the minimum requirements of satisfying vision, its amount of information is very big.After people required to bring up to the high definition standard for spatial resolution and color colourity, its amount of information was bigger.This has proposed more challenge for the transmission of video information in network.
Problem to be solved by this invention is by reducing video information collection and digitized raw information amount when guaranteeing the video information quality, thereby fundamentally solve the big problem of video information data volume, for the propagation of multimedia messages in network proposed a kind of otherwise effective technique more, and can become the effective technology of realizing real-time video, image delivering system.
Summary of the invention
For addressing the above problem, the present invention proposes the video information area dividing in the video scene is indeclinable background information zone and the regional area that produces information change, only need gather once during for the collection of indeclinable background information in video conferencing system, also only storage is once, only transmit once in transmission of Information when storage; And adopt continuous acquisition, storage, transmission means for the regional area that produces information change; Simultaneously for definite position that produces the regional area of information change, need pre-determine position and the size of the regional area of generation information change at whole panorama scene image, and the position and the size that will produce the regional area of information change gather, store and transmit as parameter, and only need transmit once.The message pick-up of video conferencing system and display end according to the indeclinable background information of receiving, continuously produce information change regional area video flowing and produce the position of regional area of information change and size parameter rebuilds and is complete scene information.Because producing the local region information amount of information change is the part of whole scene amount of information, therefore can effectively reduce the transinformation in video, the image delivering system, be the effective technology of realizing real-time video, image delivering system.
Technical scheme of the present invention is described below:
1, a kind of real-time video, image transfer method comprise the steps:
At transmitting terminal:
1) whole video stream or a plurality of video image are divided into the video image set of the indeclinable substantially segmentation video flowing of a plurality of backgrounds or a plurality of same backgrounds according to the variation of background;
2) gather for the video image of each segmentation video flowing or a plurality of same backgrounds, because its background does not change substantially, background is set at stationary objects, video image set at each segmentation video flowing or a plurality of same backgrounds only needs complete background image of record, in scene,, at first obtain this width of cloth background image information without any under the moving object situation;
3) gather for the video image of each segmentation video flowing or a plurality of same backgrounds, according to the distribution that the information change zone takes place, can become one or more generation information change zone with the information change area dividing takes place, and, and store these parameters with the area size and the relative position in image scene information of rectangular length and width and each generation information change of the relative position parameter tags in image scene information;
Only need gather once during 4) for the collection of indeclinable background information, also only storage is once when storage;
5) adopt continuous acquisition, storage mode for the zone that produces information change;
6) simultaneously for definite position that produces the zone of information change, need pre-determine the zone that produces information change position and size at whole panorama scene image, and will produce information change the zone the position and big or smallly once gather, store as parameter;
7) adopt still image compression, encryption algorithm to handle for background information;
8) Video stream information for the information change zone adopts moving image compression, encryption algorithm to handle;
9) adopt lossless compress, encryption algorithm to handle for parameter information;
10) increase background information sign, region of variation numerical indicant;
11) background information sign, region of variation numerical indicant, background information, information change zone, the parameter information after will compressing discharges in certain sequence;
12) adopt continuous transmission means for the zone of information change;
At receiving terminal:
1) obtains the compressed encoding information of parameter information of Video stream information, the region of variation of background information, the region of variation of video flowing according to Information sign;
2) for background information adopt the still image compression, encryption algorithm is decoded and decompression;
3) Video stream information for the information change zone adopts the moving image compression, encryption algorithm is decoded and decompression;
4) for parameter information adopt lossless compress, encryption algorithm is decoded and decompression;
5) go out the complete video stream information of original scene by software rejuvenation.
2, the background information of described photographed scene is a width of cloth still image of taking when not comprising the object that can move variation, and only needs the background information of a described photographed scene of storage in the constant substantially one section Video stream information of the background information of described photographed scene.
3, the zone of described generation information change is with a rectangle field mark, and identifies the position of rectangular region with a summit of this rectangular region at the relative position information of whole scene, identifies its size with the length and the width of rectangular region.
Distinguish described background information, parameter information, regional continuous information with different flag informations when 4, the video image set of every section video flowing in the set of the video image of described segmentation video flowing or a plurality of same backgrounds or each same background is stored.
5, store described flag information and described background information, parameter information, regional continuous information by certain storage organization during the video image of every section video flowing in the video image of described segmentation video flowing or a plurality of same backgrounds set or each same background set storage.
6, the video image of each segmentation video flowing in the set of the video image of described segmentation video flowing or a plurality of same backgrounds or a plurality of same backgrounds is gathered when storing by time sequencing storage, transmission and the reception of taking; And the video scene information of recovering out at receiving terminal is by former shooting played in order or demonstration.
Because in the transmission course of video stream, the regional area that has only scene is by video flowing mode continuous acquisition and processing, compare present known system with whole scene by video flowing mode continuous acquisition with handle, the amount of information of original processing reduces.Change video, the image delivering system that does not have variation substantially for background informations such as similar video conferencing systems especially, owing to reduced information processing and data quantity transmitted significantly, can realize real-time Transmission better, the image transmission quality can not descend simultaneously.
In the present invention, in order further to reduce the amount of information of background information, when background information may occur because of the variation of the introducing of moving object and irradiation light changing,, then these parameters are sent to receiving terminal by writing down the angle and the intensity parameters of irradiation light.Receiving terminal utilizes optical principle to adopt certain software algorithm can simulate this variation and adjust background information automatically at receiving terminal, and we can not need to transmit the original image that real background produces because of the variation of the introducing of moving object and irradiation light.
In the present invention,, in shooting process, there is a big background, changes the background information of photographed scene by the mobile or focal length variations of capture apparatus in order further to reduce the amount of information of background information.The image information that at first will shoot whole overall background this moment by the moving parameter and the focal length parameter of records photographing equipment, is sent to receiving terminal with these parameters simultaneously then as transmission information.Receiving terminal utilizes optical principle to adopt certain software algorithm can simulate this variation and adjust background information automatically at receiving terminal, and we can not need to transmit real background moving or original image that focal length variations produces because of capture apparatus.
In the reality, most of Video stream information or image collection information all have the image background situation of inconvenience relatively.As video conferencing system, its video background information can not change usually; Take as film shooting or TV play, normally be made of the essentially identical plot section of several background scene, these all are the main directions that the present invention uses.
Description of drawings
Fig. 1 is that real-time video of the present invention, image delivering system are for the video scene area dividing of having only an information change zone and parameter tags schematic diagram.
Fig. 2 is that real-time video of the present invention, image delivering system are for video scene area dividing that two information change zones are arranged and parameter tags schematic diagram.
Fig. 3 is a segmentation video flowing storage organization schematic diagram in real-time video of the present invention, the image delivering system.
Embodiment
Below in conjunction with accompanying drawing and exemplifying embodiment technical scheme of the present invention is further described:
Fig. 1 is that real-time video of the present invention, image delivering system are for the video scene area dividing of having only an information change zone and parameter tags schematic diagram.As shown in Figure 1, for one section video flowing or image collection, has only an information change zone in the scene.Scene is divided into background information part 1, information change zone 3.In order to recover original scene information, also need the relative position information 5 of recorded information region of variation 3 in scene and the rectangular area length of side parameter 2,4 in information change zone 3 at information receiving end.During one section video flowing under taking this scene, only need record one width of cloth background information, the parameter information in the Video stream information of region of variation and information change zone 3 in the scene.Information for background information part 1 adopts still image compression, encryption algorithm to handle; Video stream information for information change zone 3 adopts moving image compression, encryption algorithm to handle, and adopts lossless compress, encryption algorithm to handle for parameter information.These treated information are also transmitted behind the additional information sign by information specific stream order.Information receiving end adopts with the corresponding decoding of transmitting terminal, decompression algorithm and recovers background information, information change zone Video stream information, information change region parameter information, can effectively recover the complete video stream information of original scene then by software.Owing to significantly reduce, can solve the problem that real-time video, image transmission contain much information in the amount of information of information source end.
Fig. 2 is that real-time video of the present invention, image delivering system are for video scene area dividing that two information change zones are arranged and parameter tags schematic diagram.As shown in Figure 2, for one section video flowing or image collection, two information change zones are arranged in the scene.Scene is divided into background information part 6, information change zone 1, information change zone 2 12.In order to recover original scene information, also need relative position information 10, the rectangular area length of side parameter 7 and 9 in information change zone one 8, information change zone two 12 relative position information 14 in scene, the rectangular area length of side parameter 11 and 13 in information change zone two 12 of recorded information region of variation 1 in scene at information receiving end.During one section video flowing under taking this scene, only need record one width of cloth background information, the parameter information in the Video stream information of region of variation and information change zone 8,12 in the scene.Information for background information part 6 adopts still image compression, encryption algorithm to handle; Video stream information for information change zone 8,12 adopts moving image compression, encryption algorithm to handle respectively, adopts lossless compress, encryption algorithm to handle for parameter information.These treated information are also transmitted behind the additional information sign by information specific stream order.Information receiving end adopts with the corresponding decoding of transmitting terminal, decompression algorithm and recovers background information, information change zone Video stream information, information change region parameter information, can effectively recover the complete video stream information of original scene then by software.Owing to significantly reduce, can solve the problem that real-time video, image transmission contain much information in the amount of information of information source end.
Fig. 3 is a segmentation video flowing storage organization schematic diagram in real-time video of the present invention, the image delivering system.For one section video flowing or image collection, after information gathering described in employing the present invention and the processing method, for the ease of the recovery of information, when stored information, need increase Information sign, and the Information sign and the information content are discharged in certain sequence.Be a kind of order example of depositing as shown in Figure 3, the stored information of first section video flowing and order comprise background information sign I, background image information II, region of variation numerical indicant III, region of variation (1,2 ..., n) parameter information (IV 1, IV 2 ..., IV n), region of variation (1,2 ..., n) Video stream information (V 1, V 2 ..., n), deposit next section Video stream information subsequently, repeat first section the order of depositing, as background information sign VI, the background image information VII of next section video flowing ...Just can obtain the background information of video flowing, the Video stream information of region of variation, the parameter information of region of variation at receiving terminal according to Information sign.Just can recover the complete video stream information of original scene by software.If when information gathering, also write down the angle of irradiation light and the moving parameter and the focal length parameter of intensity parameters and/or capture apparatus, these parameters insertions can be deposited in the sequence at segmentation video flowing storage organization, and increase corresponding Information sign.
Owing to when having guaranteed picture quality, amount of information is reduced in a large number in the real-time video among the present invention, the image delivering system, for the real-time Transmission that realizes video, image information provides possibility from the information source end.Compare known video flowing or image processing techniques and system at present, the present invention has information processing capacity still less, can realize real-time Transmission better under same transmission bandwidth condition, therefore the while realizes and popularization easily owing to do not adopt complicated algorithm or technology.
It should be noted last that: above exemplifying embodiment is the unrestricted technical scheme of the present invention in order to explanation only, although the present invention is had been described in detail with reference to above-mentioned example, those skilled in the art is to be understood that: still can make amendment or replacement on an equal basis to the present invention, and replace any modification or the part that do not break away from the spirit and scope of the present invention, and it all should be encompassed in the middle of the claim scope of the present invention.

Claims (6)

1, a kind of real-time video, image transfer method is characterized in that comprising the steps:
At transmitting terminal:
1) whole video stream or a plurality of video image are divided into the video image set of the indeclinable substantially segmentation video flowing of a plurality of backgrounds or a plurality of same backgrounds according to the variation of background;
2) gather for the video image of each segmentation video flowing or a plurality of same backgrounds, because its background does not change substantially, background is set at stationary objects, video image set at each segmentation video flowing or a plurality of same backgrounds only needs complete background image of record, in scene,, at first obtain this width of cloth background image information without any under the moving object situation;
3) gather for the video image of each segmentation video flowing or a plurality of same backgrounds, according to the distribution that the information change zone takes place, can become one or more generation information change zone with the information change area dividing takes place, and, and store these parameters with the area size and the relative position in image scene information of rectangular length and width and each generation information change of the relative position parameter tags in image scene information;
Only need gather once during 4) for the collection of indeclinable background information, also only storage is once when storage;
5) adopt continuous acquisition, storage mode for the zone that produces information change;
6) simultaneously for definite position that produces the zone of information change, need pre-determine the zone that produces information change position and size at whole panorama scene image, and will produce information change the zone the position and big or smallly once gather, store as parameter;
7) adopt still image compression, encryption algorithm to handle for background information;
8) Video stream information for the information change zone adopts moving image compression, encryption algorithm to handle;
9) adopt lossless compress, encryption algorithm to handle for parameter information;
10) increase background information sign, region of variation numerical indicant;
11) background information sign, region of variation numerical indicant, background information, information change zone, the parameter information after will compressing discharges in certain sequence;
12) adopt continuous transmission means for the zone of information change;
At receiving terminal:
1) obtains the compressed encoding information of parameter information of Video stream information, the region of variation of background information, the region of variation of video flowing according to Information sign;
2) for background information adopt the still image compression, encryption algorithm is decoded and decompression;
3) Video stream information for the information change zone adopts the moving image compression, encryption algorithm is decoded and decompression;
4) for parameter information adopt lossless compress, encryption algorithm is decoded and decompression;
5) go out the complete video stream information of original scene by software rejuvenation.
2, real-time video as claimed in claim 1, image transfer method, it is characterized in that, the background information of described photographed scene is a width of cloth still image of taking when not comprising the object that can move variation, and only needs the background information of a described photographed scene of storage in the constant substantially one section Video stream information of the background information of described photographed scene.
3, real-time video as claimed in claim 1, image transfer method, it is characterized in that, the zone of described generation information change is with a rectangle field mark, and identify the position of rectangular region at the relative position information of whole scene with a summit of this rectangular region, identify its size with the length and the width of rectangular region.
4, real-time video as claimed in claim 1, image transfer method, it is characterized in that, distinguish described background information, parameter information, regional continuous information with different flag informations when the every section video flowing in the video image set of described segmentation video flowing or a plurality of same backgrounds or the video image set of each same background are stored.
5, real-time video as claimed in claim 1, image transfer method, it is characterized in that, store described flag information and described background information, parameter information, regional continuous information by certain storage organization during the video image set storage of every section video flowing in the set of the video image of described segmentation video flowing or a plurality of same backgrounds or each same background.
6, real-time video as claimed in claim 1, image transfer method, it is characterized in that each segmentation video flowing in the video image set of described segmentation video flowing or a plurality of same backgrounds or the video image of a plurality of same backgrounds are gathered when storing by time sequencing storage, transmission and the reception of taking; And the video scene information of recovering out at receiving terminal is by former shooting played in order or demonstration.
CN 200610114181 2006-11-01 2006-11-01 Real time video image transmission technology Expired - Fee Related CN100469108C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200610114181 CN100469108C (en) 2006-11-01 2006-11-01 Real time video image transmission technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200610114181 CN100469108C (en) 2006-11-01 2006-11-01 Real time video image transmission technology

Publications (2)

Publication Number Publication Date
CN1946144A CN1946144A (en) 2007-04-11
CN100469108C true CN100469108C (en) 2009-03-11

Family

ID=38045340

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200610114181 Expired - Fee Related CN100469108C (en) 2006-11-01 2006-11-01 Real time video image transmission technology

Country Status (1)

Country Link
CN (1) CN100469108C (en)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668129B (en) * 2008-09-04 2013-01-02 新奥特(北京)视频技术有限公司 Improved camera initial positioning system in virtual studio
JP5392533B2 (en) * 2008-10-10 2014-01-22 ソニー株式会社 Solid-state imaging device, optical device, signal processing device, and signal processing system
CN102737617B (en) 2011-04-01 2014-04-30 华为终端有限公司 Method and device for video image display
CN102769772B (en) * 2011-05-05 2014-12-10 浙江大学 Method and device for evaluating video sequence distortion
CN102724503B (en) * 2012-06-13 2015-04-29 广东威创视讯科技股份有限公司 Video compression method and system
CN102819846B (en) * 2012-06-13 2015-01-21 广东威创视讯科技股份有限公司 Method and system for playing high-definition video
CN103634556B (en) * 2012-08-27 2017-04-19 联想(北京)有限公司 Information transmission method, information receiving method and electronic apparatus
CN103037205B (en) * 2012-12-14 2016-03-02 广东威创视讯科技股份有限公司 Video transmission method and system
CN104254001B (en) * 2013-06-28 2017-08-25 广州华多网络科技有限公司 Long-range sharing method, device and terminal
CN104851119B (en) * 2014-02-13 2019-04-12 腾讯科技(深圳)有限公司 A kind of generation method and device of animation portable network graphic picture
CN105786422B (en) * 2014-12-26 2018-12-14 联想(北京)有限公司 Data processing method and electronic equipment
CN105991999B (en) * 2015-02-15 2019-12-24 联想(北京)有限公司 Information processing method and electronic equipment
CN106878268A (en) * 2016-12-28 2017-06-20 杰创智能科技股份有限公司 The method and system of low-bandwidth high-quality transmitting, monitoring image
CN107318021B (en) * 2017-06-22 2020-10-27 长沙市极云网络科技有限公司 Data processing method and system for remote display
CN107318020B (en) * 2017-06-22 2020-10-27 长沙市极云网络科技有限公司 Data processing method and system for remote display
CN108282674B (en) * 2018-02-05 2020-08-21 天地融科技股份有限公司 Video transmission method, terminal and system
CN108492567A (en) * 2018-04-24 2018-09-04 汪宇明 Monitor terminal, road traffic method for early warning and system
CN110572665B (en) * 2019-09-24 2021-07-23 中国人民解放军国防科技大学 Static background video self-adaptive compression method based on background subtraction
CN111405142A (en) * 2020-03-30 2020-07-10 咪咕视讯科技有限公司 Image processing method, device and computer readable storage medium
CN111757063A (en) * 2020-06-29 2020-10-09 郑州信大捷安信息技术股份有限公司 Video streaming transmission method and system based on 5G environment
CN112312043A (en) * 2020-10-20 2021-02-02 深圳市前海手绘科技文化有限公司 Optimization method and device for deriving animation video
CN113115037B (en) * 2021-06-15 2021-09-14 深圳市中鹏教育科技股份有限公司 Online education method, system, equipment and storage medium
CN115379114A (en) * 2022-07-19 2022-11-22 阿里巴巴(中国)有限公司 Panoramic video processing method and device and electronic equipment
CN115529484A (en) * 2022-09-08 2022-12-27 阿里巴巴(中国)有限公司 Video processing method and device and electronic equipment

Also Published As

Publication number Publication date
CN1946144A (en) 2007-04-11

Similar Documents

Publication Publication Date Title
CN100469108C (en) Real time video image transmission technology
US6810403B2 (en) Visual database system for storing data with time codes
US8495697B1 (en) Techniques to provide an enhanced video replay
CN110012279B (en) 3D point cloud data-based view-division compression and transmission method and system
CN1132406C (en) A picture communication apparatus
CN106878764A (en) A kind of live broadcasting method of virtual reality, system and application thereof
CN104335243B (en) A kind of method and device for handling panorama
CN111971954A (en) Method and apparatus for transmitting 360 degree video using metadata associated with hotspots and ROIs
CN107529064A (en) A kind of self-adaptive encoding method based on VR terminals feedback
EP3456058A1 (en) Bit depth remapping based on viewing parameters
CN106789991A (en) A kind of multi-person interactive method and system based on virtual scene
US20100111489A1 (en) Digital Camera System for Recording, Editing and Visualizing Images
CN109587401A (en) The more scene capture realization method and systems of electronic platform
CN104079833A (en) Method and device for shooting star orbit videos
KR102308604B1 (en) Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices
CN107426491B (en) Implementation method of 360-degree panoramic video
CN113115110B (en) Video synthesis method and device, storage medium and electronic equipment
CN112911304B (en) Encoding-based two-way video compression device and compressed video reconstruction method
CN102724492A (en) Method and system for transmitting and playing video images
US11979544B2 (en) 360 video transmission method, 360 video reception method, 360 video transmission apparatus, and 360 video reception apparatus
CN106713942A (en) Video processing method and video processing device
Pintaric et al. Immersive panoramic video
CN109618093A (en) A kind of panoramic video live broadcasting method and system
CN109644290B (en) Data switching device, data acquisition device, system and method
CN108174119A (en) The control method of photographic device, multispectral picture pick-up device, unmanned plane and medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
DD01 Delivery of document by public notice

Addressee: Li Bohang

Document name: Notification of Termination of Patent Right

C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090311

Termination date: 20121101