CN105741322B - A kind of field of view dividing method based on the fusion of video features layer - Google Patents
A kind of field of view dividing method based on the fusion of video features layer Download PDFInfo
- Publication number
- CN105741322B CN105741322B CN201610072608.4A CN201610072608A CN105741322B CN 105741322 B CN105741322 B CN 105741322B CN 201610072608 A CN201610072608 A CN 201610072608A CN 105741322 B CN105741322 B CN 105741322B
- Authority
- CN
- China
- Prior art keywords
- pixel
- video
- color
- feature
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The invention discloses a kind of field of view dividing methods based on the fusion of video features layer.It includes the following steps：Calculate the color characteristic of each pixel in video；Calculate the dynamic feature of each pixel in video；Calculate the textural characteristics of each pixel in video；The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion, region segmentation is carried out to the visual field in video according to fusion gained feature.The invention comprehensively utilizes the dynamic feature on video image vegetarian refreshments time dimension and the color characteristics and textural characteristics on Spatial Dimension, improve the validity and correctness of field of view segmentation.
Description
Technical field
The present invention relates to video analysis processing technology field more particularly to a kind of visual field areas based on the fusion of video features floor
Domain splitting method.
Background technology
With video technique it is continuous it is ripe decline with cost, video analysis and treatment technology have been widely used for scientific research,
The numerous areas of production and social life.Carrying out region segmentation to the visual field presented in video helps to extract in video
Valuable information, be a kind of important video analysis and treatment technology.
Currently, what is mainly used for reference for the region segmentation method of visual field in video is image region segmentation technology.Commonly
Image region segmentation technology has the method based on color characteristic, the method based on textural characteristics, and the side based on shape feature
Method etc..Obviously, image region segmentation technology is grafted directly in the video object, has ignored the abundant dynamic for including in video
The time variation of feature and video content necessarily leads to the deficiency of field of view segmentation validity and correctness.
Invention content
The abundant dynamic for including in video the purpose of the present invention is overcoming existing field of view dividing method to have ignored is special
The time variation of sign and video content leads to the technical problem of field of view segmentation validity and correctness deficiency, provides one
The field of view dividing method that kind is merged based on video features layer, fully utilizes the dynamic on video image vegetarian refreshments time dimension
Property the feature and color characteristic on Spatial Dimension and textural characteristics, improve the validity and correctness of field of view segmentation.
To solve the above-mentioned problems, the present invention is achieved by the following scheme：
A kind of field of view dividing method based on the fusion of video features layer of the present invention, includes the following steps：
S1：Calculate the color characteristic of each pixel in video；
S2：Calculate the dynamic feature of each pixel in video；
S3：Calculate the textural characteristics of each pixel in video；
S4：The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion, root
Region segmentation is carried out to the visual field in video according to fusion gained feature.
In the technical scheme, it using the field of view dividing method merged based on video features layer, fully utilizes and regards
Dynamic feature on frequency pixel time dimension and the color characteristic on Spatial Dimension and textural characteristics, are utilized in video
The Space Time Joint Distribution feature of visual information can not utilize vision to believe when overcoming to video capture image region segmentation method
The deficiency of the temporal dynamic property feature of breath, the color video that this method is suitable for various resolution ratio fixed to visual field carry out visual field
Region segmentation.
Preferably, the step S1 includes the following steps：
S11：One color feature vector based on RGB color, the RGB face are generated to each pixel of video
Color characteristic vector is as follows：
f1(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t)
Wherein, R (i, j) |tPixel of the video in t frames at coordinate (i, j) is represented on red color channel
Pixel value, G (i, j) |tRepresent pixel pixel on green color channel of the video in t frames at coordinate (i, j)
Value, B (i, j) |tRepresent pixel value of pixel of the video in t frames at coordinate (i, j) on blue color channels；
S12：The RGB color of video is converted into hsv color space；
S13：One color feature vector based on hsv color space, the HSV face are generated to each pixel of video
Color characteristic vector is as follows：
f2(i, j) |t=(H (i, j) |t, S (i, j) |t, V (i, j) |t)
Wherein, H (i, j) |tRepresent pixel pixel on tone channel of the video in t frames at coordinate (i, j)
Value, S (i, j) |tRepresent pixel pixel value on saturation degree channel of the video in t frames at coordinate (i, j), V (i,
j)|tRepresent pixel pixel value on luminance channel of the video in t frames at coordinate (i, j)；
S14：By color feature vector and the color feature vector based on hsv color space based on RGB color into
Row series connection, generates a color feature vector based on double color spaces, which indicates as follows：
f3(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t, H (i, j) |t, S (i, j) |t, V (i, j) |t)。
Preferably, the step S2 includes the following steps：
S21：Convert video to gray processing video；
S22：Background model is built to each pixel in gray processing video；
S23：The number of the conspicuousness gray-value variation occurred on each pixel in statistics gray processing video, conspicuousness
Gray-value variation is defined as：Gray-value variation amplitude on one pixel is beyond the ash set by background model on the pixel
Angle value normal variation range；
S24：The dynamic of each pixel in gray processing video is calculated, the calculation formula of the dynamic of pixel is：
Wherein, Ψ (i, j) |tRepresent pixel of the gray processing video from start frame at t frame time internal coordinates (i, j)
The number of the conspicuousness gray-value variation of upper generation；D (i, j) |tRepresent pixel at coordinate (i, j) gray processing video from
The frequency of conspicuousness gray-value variation, i.e., the dynamic of pixel at coordinate (i, j) occur in start frame to t frame times.
Preferably, the step S3 includes the following steps：
S31：Convert video to gray processing video, calculate gray processing video using original LBP operators is sitting in t frames
The LBP texture values of the pixel at (i, j) are marked, and as the 1st texture eigenvalue W of the pixel1(i, j) |t；
S32：The LBP of pixel of the gray processing video in t frames at coordinate (i, j) is calculated using round LBP operators
Texture value, and as the 2nd texture eigenvalue W of the pixel2(i, j) |t；
S33：By the 1st texture eigenvalue and the 2nd line of pixel of the gray processing video in t frames at coordinate (i, j)
Reason eigenvalue cluster is combined into the texture feature vector of the pixel, i.e.,：f4(i, j) |t=(W1(i, j) |t, W2(i, j) |t)。
Preferably, the step S4 includes the following steps：
S41：The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion,
Obtain fusion feature vector；
S42：Automatically gathered using the fusion feature vector on all pixels point in video when clustering method pair t frames
Alanysis；
S43：All pixels being classified as corresponding to the fusion feature vector of one kind by clustering are divided into together
The region segmentation to visual field in video is completed in one region.
Substantial effect of the invention is：A variety of visual signatures of the video on time and Spatial Dimension are fully utilized,
It is quiet due to increasing including the pixel dynamic feature on time dimension and the color on Spatial Dimension and textural characteristics
Pixel dynamic feature this key message not having in state image uses image region segmentation method to overcome
The problem of validity and correctness deficiency caused by region segmentation is carried out to video.
Description of the drawings
Fig. 1 is the work flow diagram of the present invention；
Fig. 2 is the schematic diagram of original LBP operators in step S31；
Fig. 3 is the schematic diagram of circle LBP operators in step S32.
Specific implementation mode
Below with reference to the embodiments and with reference to the accompanying drawing the technical solutions of the present invention will be further described.
Embodiment：A kind of field of view dividing method based on the fusion of video features layer of the present embodiment, as shown in Figure 1,
Include the following steps：
S1：Calculate the color characteristic of each pixel in video；
S2：Calculate the dynamic feature of each pixel in video；
S3：Calculate the textural characteristics of each pixel in video；
S4：The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion, root
Region segmentation is carried out to the visual field in video according to fusion gained feature.
Step S1 includes the following steps：
S11：One color feature vector based on RGB color, the RGB face are generated to each pixel of video
Color characteristic vector is as follows：
f1(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t)
Wherein, R (i, j) |tPixel of the video in t frames at coordinate (i, j) is represented on red color channel
Pixel value, G (i, j) |tRepresent pixel pixel on green color channel of the video in t frames at coordinate (i, j)
Value, B (i, j) |tRepresent pixel value of pixel of the video in t frames at coordinate (i, j) on blue color channels；
S12：The RGB color of video is converted into hsv color space；
S13：One color feature vector based on hsv color space, the HSV face are generated to each pixel of video
Color characteristic vector is as follows：
f2(i, j) |t=(H (i, j) |t, S (i, j) |t, V (i, j) |t)
Wherein, H (i, j) |tRepresent pixel pixel on tone channel of the video in t frames at coordinate (i, j)
Value, S (i, j) |tRepresent pixel pixel value on saturation degree channel of the video in t frames at coordinate (i, j), V (i,
j)|tRepresent pixel pixel value on luminance channel of the video in t frames at coordinate (i, j)；
S14：By color feature vector and the color feature vector based on hsv color space based on RGB color into
Row series connection, generates a color feature vector based on double color spaces, which indicates as follows：
f3(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t, H (i, j) |t, S (i, j) |t, V (i, j) |t)。
Step S2 includes the following steps：
S21：Gray processing processing is carried out to video, converts video to gray processing video；
S22：Background model is built to each pixel in gray processing video；
S23：The number of the conspicuousness gray-value variation occurred on each pixel in statistics gray processing video, conspicuousness
Gray-value variation is defined as：Gray-value variation amplitude on one pixel is beyond the ash set by background model on the pixel
Angle value normal variation range, i.e. gray-value variation amplitude on a pixel is beyond set by background model on the pixel
Gray value normal variation range is primary, and the conspicuousness gray-value variation number of the pixel adds 1；
S24：The dynamic of each pixel in gray processing video is calculated, the calculation formula of the dynamic of pixel is：
Wherein, Ψ (i, j) |tRepresent pixel of the gray processing video from start frame at t frame time internal coordinates (i, j)
The number of the conspicuousness gray-value variation of upper generation；D (i, j) |tRepresent pixel at coordinate (i, j) gray processing video from
The frequency of generation conspicuousness gray-value variation in start frame to t frame times, i.e., the dynamic of pixel at coordinate (i, j), as
The dynamic of vegetarian refreshments refers to that the frequency of conspicuousness gray-value variation occurs on pixel, the pixel in the low expression video of dynamic
The scene changes at place are small, and dynamic height indicates that the scene changes in video at the pixel are big.
Step S3 includes the following steps：
S31：Convert video to gray processing video, calculate gray processing video using original LBP operators is sitting in t frames
The LBP texture values of the pixel at (i, j) are marked, and as the 1st texture eigenvalue W of the pixel1(i, j) |t, original
LBP operators are as shown in Figure 2；
S32：The LBP of pixel of the gray processing video in t frames at coordinate (i, j) is calculated using round LBP operators
Texture value, and as the 2nd texture eigenvalue W of the pixel2(i, j) |t, round LBP operators are as shown in Figure 3；
S33：By the 1st texture eigenvalue and the 2nd line of pixel of the gray processing video in t frames at coordinate (i, j)
Reason eigenvalue cluster is combined into the texture feature vector of the pixel, i.e.,：f4(i, j) |t=(W1(i, j) |t, W2(i, j) |t)。
Step S4 includes the following steps：
S41：The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion,
Obtain fusion feature vector：
F (i, j) |t=(D (i, j) |t, R (i, j) |t, G (i, j) |t, B (i, j) |t, H (i, j) |t, S (i, j) |t, V (i, j)
|t, W1(i, j) |t, W2(i, j) |t)；
S42：Using the fusion feature vector f (i, j) on all pixels point in video when clustering method pair t frames |tIt carries out
Automatic clustering；
S43：All pixels being classified as corresponding to the fusion feature vector of one kind by clustering are divided into together
The region segmentation to visual field in video is completed in one region.
The dynamic of pixel refers to that the frequency of conspicuousness gray-value variation, the low expression video of dynamic occur on pixel
In scene changes at the pixel it is small, dynamic height indicates that the scene changes in video at the pixel are big, using based on regarding
The field of view dividing method of frequency Feature-level fusion, fully utilize dynamic feature on video image vegetarian refreshments time dimension and
Color characteristic on Spatial Dimension and textural characteristics are utilized the Space Time Joint Distribution feature of visual information in video, overcome
The deficiency of the temporal dynamic property feature of visual information, this method can not be utilized to be applicable in when to video capture image region segmentation method
Field of view segmentation is carried out in the color video to the fixed various resolution ratio of visual field.
Claims (4)
1. a kind of field of view dividing method based on the fusion of video features layer, which is characterized in that include the following steps：
S1：Calculate the color characteristic of each pixel in video；
S2：Calculate the dynamic feature of each pixel in video；
S3：Calculate the textural characteristics of each pixel in video；
S4：The dynamic feature of each pixel in video, color characteristic and textural characteristics are subjected to Feature-level fusion, according to melting
It closes gained feature and region segmentation is carried out to the visual field in video；
The step S2 includes the following steps：
S21：Convert video to gray processing video；
S22：Background model is built to each pixel in gray processing video；
S23：The number of the conspicuousness gray-value variation occurred on each pixel in statistics gray processing video, conspicuousness gray scale
Value variation is defined as：Gray-value variation amplitude on one pixel is beyond the gray value set by background model on the pixel
Normal variation range；
S24：The dynamic of each pixel in gray processing video is calculated, the calculation formula of the dynamic of pixel is：
Wherein, ψ (i, j) |tGray processing video is represented from start frame to occur on pixel at t frame time internal coordinates (i, j)
Conspicuousness gray-value variation number；D (i, j) |tPixel at coordinate (i, j) is represented in gray processing video from start frame
The frequency of conspicuousness gray-value variation, i.e., the dynamic of pixel at coordinate (i, j) occur in t frame times.
2. a kind of field of view dividing method based on the fusion of video features layer according to claim 1, which is characterized in that
The step S1 includes the following steps：
S11：One color feature vector based on RGB color is generated to each pixel of video, the RGB color is special
Sign vector is as follows：
f1(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t)
Wherein, R (i, j) |tRepresent pixel pixel on red color channel of the video in t frames at coordinate (i, j)
Value, G (i, j) |tRepresent pixel pixel value on green color channel of the video in t frames at coordinate (i, j), B
(i, j) |tRepresent pixel value of pixel of the video in t frames at coordinate (i, j) on blue color channels；
S12：The RGB color of video is converted into hsv color space；
S13：One color feature vector based on hsv color space is generated to each pixel of video, the hsv color is special
Sign vector is as follows：
f2(i, j) | t=(H (i, j) |t, S (i, j) |t, V (i, j) |t)
Wherein, H (i, j) |tRepresent pixel pixel value on tone channel of the video in t frames at coordinate (i, j), S
(i, j) |tPixel pixel value on saturation degree channel of the video in t frames at coordinate (i, j) is represented, V (i, j) |tGeneration
Pixel pixel value on luminance channel of the table video in t frames at coordinate (i, j)；
S14：Color feature vector based on RGB color is gone here and there with the color feature vector based on hsv color space
Connection, generates a color feature vector based on double color spaces, which indicates as follows：
f3(i, j) |t=(R (i, j) |t, G (i, j) |t, B (i, j) |t, H (i, j) |t, S (i, j) |t, V (i, j) |t)。
3. a kind of field of view dividing method based on the fusion of video features layer according to claim 1 or 2, feature exist
In the step S3 includes the following steps：
S31：It converts video to gray processing video, gray processing video is calculated in t frames in coordinate using original LBP operators
The LBP texture values of pixel at (i, j), and as the 1st texture eigenvalue W of the pixel1(i, j) |t；
S32：The LBP textures of pixel of the gray processing video in t frames at coordinate (i, j) are calculated using round LBP operators
Value, and as the 2nd texture eigenvalue W of the pixel2(i, j) |t；
S33：1st texture eigenvalue of pixel of the gray processing video in t frames at coordinate (i, j) and the 2nd texture is special
Value indicative is combined as the texture feature vector of the pixel, i.e.,：f4(i, j) |t=(W1(i, j) |t, W2(i, j) |t)。
4. a kind of field of view dividing method based on the fusion of video features layer according to claim 1 or 2, feature exist
In the step S4 includes the following steps：
S41：The dynamic feature of each pixel, color characteristic and textural characteristics in video are subjected to Feature-level fusion, are obtained
Fusion feature vector；
S42：Automatically cluster minute is carried out using the fusion feature vector on all pixels point in video when clustering method pair t frames
Analysis；
S43：All pixels being classified as by clustering corresponding to a kind of fusion feature vector are divided into same
The region segmentation to visual field in video is completed in region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610072608.4A CN105741322B (en) | 2016-02-01 | 2016-02-01 | A kind of field of view dividing method based on the fusion of video features layer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610072608.4A CN105741322B (en) | 2016-02-01 | 2016-02-01 | A kind of field of view dividing method based on the fusion of video features layer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105741322A CN105741322A (en) | 2016-07-06 |
| CN105741322B true CN105741322B (en) | 2018-08-03 |
Family
ID=56242193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610072608.4A Expired - Fee Related CN105741322B (en) | 2016-02-01 | 2016-02-01 | A kind of field of view dividing method based on the fusion of video features layer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105741322B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509830B (en) * | 2017-02-28 | 2020-12-01 | 华为技术有限公司 | Video data processing method and device |
| CN110807398A (en) * | 2019-10-28 | 2020-02-18 | 衢州学院 | Method and device for dividing field area |
| CN110910398B (en) * | 2019-10-28 | 2021-07-20 | 衢州学院 | Video complex scene region segmentation method and device based on decision layer fusion |
| CN110826445B (en) * | 2019-10-28 | 2021-04-23 | 衢州学院 | Method and device for detecting specific target area in colorless scene video |
| CN110866460B (en) * | 2019-10-28 | 2020-11-27 | 衢州学院 | Method and device for detecting specific target area in complex scene video |
| CN110827293B (en) * | 2019-10-28 | 2020-09-08 | 衢州学院 | Method and device for segmenting achromatic scene area based on decision-making layer fusion |
| CN110826446B (en) * | 2019-10-28 | 2020-08-21 | 衢州学院 | Method and device for segmenting field of view region of texture-free scene video |
| CN110910399B (en) * | 2019-10-28 | 2020-09-15 | 衢州学院 | Non-texture scene region segmentation method and device based on decision layer fusion |
| CN110796073B (en) * | 2019-10-28 | 2021-05-25 | 衢州学院 | Method and device for detecting specific target area in non-texture scene video |
| CN110807783B (en) * | 2019-10-28 | 2023-07-18 | 衢州学院 | Efficient visual field region segmentation method and device for achromatic long video |
| CN111028262A (en) * | 2019-12-06 | 2020-04-17 | 衢州学院 | Multi-channel composite high-definition high-speed video background modeling method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101316328A (en) * | 2007-05-29 | 2008-12-03 | 中国科学院计算技术研究所 | News anchor lens detection method based on space-time strip pattern analysis |
| CN102426583A (en) * | 2011-10-10 | 2012-04-25 | 北京工业大学 | Chinese medicine tongue manifestation retrieval method based on image content analysis |
| CN102915544A (en) * | 2012-09-20 | 2013-02-06 | 武汉大学 | Video image motion target extracting method based on pattern detection and color segmentation |
| CN105118049A (en) * | 2015-07-22 | 2015-12-02 | 东南大学 | Image segmentation method based on super pixel clustering |
-
2016
- 2016-02-01 CN CN201610072608.4A patent/CN105741322B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101316328A (en) * | 2007-05-29 | 2008-12-03 | 中国科学院计算技术研究所 | News anchor lens detection method based on space-time strip pattern analysis |
| CN102426583A (en) * | 2011-10-10 | 2012-04-25 | 北京工业大学 | Chinese medicine tongue manifestation retrieval method based on image content analysis |
| CN102915544A (en) * | 2012-09-20 | 2013-02-06 | 武汉大学 | Video image motion target extracting method based on pattern detection and color segmentation |
| CN105118049A (en) * | 2015-07-22 | 2015-12-02 | 东南大学 | Image segmentation method based on super pixel clustering |
Non-Patent Citations (3)
| Title |
|---|
| 基于SOFM的视频对象分割算法的研究;康达辉;《中国优秀硕士学位论文全文数据库 信息科技辑》;20061015(第2006年第10期);第22-30页 * |
| 基于颜色特征和纹理特征的磨粒彩色图像分割;郭恒光 等;《润滑与密封》;20130630;第38卷(第6期);第94-97页 * |
| 多目标矿业复杂图像特征提取与分类;王兰莎;《中国优秀硕士学位论文全文数据库 信息科技辑》;20120515(第2012年第05期);第24-25页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105741322A (en) | 2016-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105741322B (en) | A kind of field of view dividing method based on the fusion of video features layer | |
| CN102063706B (en) | Rapid defogging method | |
| CN106897972A (en) | A kind of self-adapting histogram underwater picture Enhancement Method of white balance and dark primary | |
| CN110599415A (en) | Image contrast enhancement implementation method based on local adaptive gamma correction | |
| CN103248906B (en) | Method and system for acquiring depth map of binocular stereo video sequence | |
| CN106897981A (en) | A kind of enhancement method of low-illumination image based on guiding filtering | |
| EP3249618B1 (en) | Banknote classification and identification method and device based on lab color space | |
| CN105825503A (en) | Visual-saliency-based image quality evaluation method | |
| CN108830800A (en) | The luminance raising Enhancement Method of image under a kind of half-light scene | |
| CN107610093B (en) | Full-reference image quality evaluation method based on similarity feature fusion | |
| CN106875358A (en) | Image enchancing method and image intensifier device based on Bayer format | |
| CN110349097B (en) | Color enhancement method for image significance and image processing device | |
| US20170193644A1 (en) | Background removal | |
| CN105184757A (en) | Food image color enhancement method based on color space characteristics | |
| CN103164847A (en) | Method for eliminating shadow of moving target in video image | |
| CN108550124B (en) | Illumination compensation and image enhancement method based on bionic spiral | |
| CN103106671A (en) | Method for detecting interested region of image based on visual attention mechanism | |
| CN102768758B (en) | Improved color image unsharp masking (USM) method | |
| CN102542526A (en) | Image decolorizing method | |
| CN101873506B (en) | Image processing method for providing depth information and image processing system thereof | |
| US8351724B2 (en) | Blue sky color detection technique | |
| TWI707307B (en) | Image adjusting method | |
| CN110399886B (en) | Screen image JND model construction method | |
| CN110415816B (en) | Skin disease clinical image multi-classification method based on transfer learning | |
| CN111105383A (en) | Image fusion color enhancement method for three-color vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180803 Termination date: 20210201 |