CN105141967B - Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model - Google Patents
Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model Download PDFInfo
- Publication number
- CN105141967B CN105141967B CN201510396817.XA CN201510396817A CN105141967B CN 105141967 B CN105141967 B CN 105141967B CN 201510396817 A CN201510396817 A CN 201510396817A CN 105141967 B CN105141967 B CN 105141967B
- Authority
- CN
- China
- Prior art keywords
- region
- indicates
- jnd
- lcu
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
It is a kind of based on the quick self-adapted loop filtering algorithm that can just perceive distortion model, including establishing JND model, picture frame being divided into region of interest ROI and regions of non-interest RONI using the JND model, RONI is divided into smooth region RONISR interested and texture region RONITR interested using Canny operator, ROI is divided into smooth region ROISR interested and texture region ROITR interested: adaptive-filtering step using Canny operator.The present invention optimizes ALF method in conjunction with pixel domain JND model and Canny operator for HEVC coding structure, removes the perception redundancy in video, improves code efficiency;It can rapidly find out should not ALF processing region substantially reduce the complexity of auto-adaptive loop filter algorithm in the case where not influencing quality.
Description
Technical field
The present invention relates to a kind of quick self-adapted loop circuit filtering methods, more particularly to one kind to be based on just perceiving distortion model
Quick self-adapted loop circuit filtering method, belong to technical field of video coding.
Background technique
A large amount of information also is transmitted along with to the mankind while society is evolving, and the mankind are big to the extraction of information
There are about 70% from vision.Therefore, requirement of the people to video quality, which is taken as own duty, to be in first, because video is most
Whole purpose is for human eye service.However, existing network bandwidth is not able to satisfy the real-time Transmission of high-definition video signal, therefore, depending on
Frequency Coding Compression Technology has great influence to the development of video communication.Higher compression efficiency and support are high in order to obtain
Clearly/ultra high-definition video compress, Liang great International Organization for standardization ISO/IEC and ITU-T have set up Joint Video development group
(Joint Collaborative Team on Video Coding, JCT-VC), and formulated efficient video compression standard
(High Efficiency Video Coding, HEVC).With advanced video compression standard H.264/AVC compared with, HEVC exists
On the basis of reaching H.264/AVC same video quality, the bit rate of half is saved.Compared with 2D video, 3D video can be mentioned
For more life-like and natural visual experience, still, huge the video data volume hinders the extensive use of 3D video, makes it not
It can be applied in daily life well.In order to preferably support the application of stereo 3 D video, ISO/IEC and ITU-
The tissue of T two has formulated the new 3D video compression standard 3D-HEVC based on HEVC again, while providing the reference software of 3D-HEVC
(HEVC-Based Test Model, HTM).
H.264/AVC HEVC is with, being the hybrid coding scheme using block-based prediction, transform and quantization.Cause
This, blocky effect, ringing effect, color spilling and blurring effect for occurring in H.264/AVC video etc., in HEVC video mark
It is still remained in standard.However, the quality for the picture quality that video is finally shown and the performance of loop filtering algorithm have closely
Connection.Therefore, in order to reduce influence of the above-mentioned effect to video quality, HTM uses three kinds of loop filtering technologies: deblocking filtering
(Deblocking Filter, DF), adaptively sampled value complement repay (Sample Adaptive Offset, SAO) and adaptive ring
Road filters (Adaptive Loop Filter, ALF).Deblocking filtering is mainly used to solve blocking artifact to the shadow of video quality
It rings, its core concept is exactly to be judged block boundary to decide whether that deblocking filtering is handled, that is, judges current border
It is puppet boundary caused by real border or blocking artifact.If necessary to deblocking filtering processing, it is also necessary to which judgement is using strong filtering
Mode or weak filter patterns.SAO is mainly used to solve the ring efficiency in HEVC video standard, its core concept is exactly pair
Sampled point is classified, and is then that each type selects optimal offset according to rate distortion costs, finally to each sampled point
Increase corresponding offset to improve the Subjective and objective qualities of video.ALF filter is mainly used to further increase video
Code efficiency, reduce influence of the above-mentioned effect to video quality, its core concept is exactly to make according to wiener-Hough equation
Mean square error between primitive frame and reconstructed frame is minimum.
Just discernable distortion (JND --- Just Noticeable Distortion) model refers to due to human vision system
Various visual shield effects existing for system (HVS --- Human Visual System), so that human eye can only be perceived more than certain
The noise of one threshold value, the threshold value are that minimum discovers distortion.From the definition of JND model it is found that JND model indicates video quality hair
The raw maximum video distortion that when changing, human eye cannot be discovered.Traditional ALF algorithm based on maximum coding unit (LCU) is to every
A LCU uses identical processing step, first calculating Wiener filtering parameter, then judges whether current LCU needs at ALF
Reason, therefore, it does not account for the characteristic of each LCU and the subjective feeling of the mankind.And the mankind are watching video and when image, and
It is not all interested in all the elements of video and image, and the interest level of different zones is also different.Therefore, do not feel
The video distortion in interest region influences video quality little.It currently, is at home and abroad view to the research of adaptive loop filter
One hot spot of frequency coding, but few scholars consider human-eye visual characteristic when reducing filter complexity, because
And the invention is undoubtedly of great significance.
Summary of the invention
The present invention provides a kind of based on the quick self-adapted loop circuit filtering method that can just perceive distortion model.
In order to achieve the above objectives, the technical scheme adopted by the invention is that:
It is a kind of based on the quick self-adapted loop circuit filtering method that can just perceive distortion model, comprising the following steps:
Step 1: establish JND model:
JND (x, y)=Tl(x,y)+Tt(x,y)-Cl,t·min{Tl(x,y),Tt(x,y)} (1)
(1) in formula, TlThe minimum that (x, y) indicates that the brightness shielding effect of the region pixel I (x, y) generates can detection threshold
Value, calculation method are as follows:
(2) in formula,Indicate the average background brightness value of pixel I (x, y) surrounding 5x5 window, calculation method are as follows:
(3) in formula, B (i, j) is the low-pass filter of 5x5:
(1) in formula, TtThe minimum that (x, y) indicates that the contrast shielding effect of the region pixel I (x, y) generates can perceive
Threshold value, calculation method are as follows:
Tt(x, y)=η G (x, y) We (x, y) (5)
(5) in formula, We (x, y) indicates the weighted value of fringe region and texture region, and η indicates the weighting of contrast shielding effect
Coefficient;G (x, y) indicates to carry out pixel I (x, y) maximum value after four different directions edge detections, calculation method are as follows:
Step 2: picture frame is divided into region of interest ROI and regions of non-interest RONI using the JND model:
Wherein, JNDLCUIndicate that the sum of the JND value of all pixels in LCU, the JND value are calculated using the JND model
It arrives;JNDframeIndicate the average value of the sum of JND value of all pixels in picture frame, calculation method are as follows:
Wherein, N indicates the number of LCU in described image frame;
Step 3: RONI being divided into smooth region RONISR interested and texture region interested using Canny operator
RONITR:
Wherein, k indicates the division threshold value of RONISR and RONITR, CannyRONIIndicate the LCU for belonging to regions of non-interest
Ratio shared by middle boundary pixel, calculation method are as follows:
CannyRONI=∑edge/M (12)
Wherein, ∑edgeIndicate the number of boundary pixel in LCU, M indicates the number of pixel in LCU;
Step 4: ROI being divided into smooth region ROISR interested and texture region interested using Canny operator
ROITR:
Wherein, m indicates the division threshold value of ROISR and ROITR, CannyROIIt indicates to belong to side in the LCU of regions of non-interest
Ratio shared by boundary's pixel, calculation method and CannyRONICalculation method it is identical;
Step 5: adaptive loop filter processing: to the LCU in the RONITR and ROITR of described image frame luminance component into
Row loop filtering;LCU in GOP each to described image frame in the RONITR and ROITR of the 0th layer of chromatic component carries out loop
Filtering.
In the step 1, weighted value We (x, the y) value of fringe region and texture region is 1;Contrast shielding effect adds
Weight coefficient η value is 0.018.
Beneficial effects of the present invention:
1, the present invention is directed to 3D-HEVC coding structure, in conjunction with pixel domain JND model and Canny operator, to based on LCU's
ALF algorithm optimizes, and removes the perception redundancy in video, improves the code efficiency of ALF algorithm;
2, the present invention can rapidly find out should not ALF processing region, in the case where not influencing quality, substantially reduce from
Adapt to the complexity of loop filter algorithm.
Detailed description of the invention
Fig. 1 is flow chart of the invention;
Fig. 2 is control experiment treated Poznan_Hall2 video sequence image;
Fig. 3 is the present invention treated Poznan_Hall2 video sequence image;
Fig. 4 is control experiment treated Poznan_Street video sequence image;
Fig. 5 is the present invention treated Poznan_Street video sequence image.
Specific embodiment:
The embodiment of the present invention is described in further detail below in conjunction with attached drawing.
Embodiment:
As shown in Figure 1, a kind of based on the quick self-adapted loop circuit filtering method that can just perceive distortion model, including following step
It is rapid:
Step 1: establish JND model:
JND (x, y)=Tl(x,y)+Tt(x,y)-Cl,t·min{Tl(x,y),Tt(x,y)} (4)
(1) in formula, TlThe minimum that (x, y) indicates that the brightness shielding effect of the region pixel I (x, y) generates can detection threshold
Value, calculation method are as follows:
(2) in formula,Indicate the average background brightness value of pixel I (x, y) surrounding 5x5 window, calculation method are as follows:
(3) in formula, B (i, j) is the low-pass filter of 5x5:
(1) in formula, TtThe minimum that (x, y) indicates that the contrast shielding effect of the region pixel I (x, y) generates can perceive
Threshold value, calculation method are as follows:
Tt(x, y)=η G (x, y) We (x, y) (5)
(5) in formula, We (x, y) indicates the weighted value of fringe region and texture region, value 1, and η indicates that contrast is covered
Effect weighting coefficient, value 0.018;G (x, y) indicate to pixel I (x, y) carry out four different directions edge detections after most
Big value, calculation method are as follows:
Step 2: picture frame is divided into region of interest ROI and regions of non-interest RONI using the JND model:
Wherein, JNDLCUIndicate that the sum of the JND value of all pixels in LCU, the JND value are calculated using the JND model
It arrives;JNDframeIndicate the average value of the sum of JND value of all pixels in picture frame, calculation method are as follows:
Wherein, N indicates the number of LCU in described image frame;
Step 3: RONI being divided into smooth region RONISR interested and texture region interested using Canny operator
RONITR:
Wherein, k indicates the division threshold value of RONISR and RONITR, CannyRONIIndicate the LCU for belonging to regions of non-interest
Ratio shared by middle boundary pixel, calculation method are as follows:
CannyRONI=∑edge/M (12)
Wherein, ∑edgeIndicate the number of boundary pixel in LCU, M indicates the number of pixel in LCU;
Step 4: ROI being divided into smooth region ROISR interested and texture region interested using Canny operator
ROITR:
Wherein, m indicates the division threshold value of ROISR and ROITR, CannyROIIt indicates to belong to side in the LCU of regions of non-interest
Ratio shared by boundary's pixel, calculation method and CannyRONICalculation method it is identical;
Step 5: adaptive loop filter processing: to the LCU in the RONITR and ROITR of described image frame luminance component into
Row loop filtering;LCU in GOP each to described image frame in the RONITR and ROITR of the 0th layer of chromatic component carries out loop
Filtering.
The simulation experiment result is given below, to illustrate performance of the invention.
It is test platform with HTM 6.1, in -3 double-core of CPU Duo, dominant frequency 3.3GHZ, memory 4GB, operating system
It is realized under Windows7 environment.Cycle tests is Balloons, Kendo, Newspaper, GhostTownFly, Poznan_
Hall2, Poznan_Street and Undo_Dancer encode preceding 25 frame.Wherein first three sequence resolution is 1024x768, after
Four sequence resolutions are 1920x1088, and all sequences encode three viewpoints simultaneously.All experiments are under same test environment
It realizes.
Regions of non-interest skips ALF treated that performance evaluation is as shown in table 1.If can be seen that all RONI
LCU without ALF processing, view 0, view 1 and view 2 performance decline clearly, the luminance component of three viewpoints
BD-rate increased separately 0.44%, 0.16% and 0.69%.The ALF treatment process for skipping all regions RONI can be substantially
Degree reduces the complexity of ALF algorithm, but video quality decline is quickly.Therefore, it is necessary to do further division to the region RONI, look for
The LCU for needing ALF to handle in the region RONI out.
It is as shown in table 2 that ratio shared by ALF processing region is not needed in non-smooth region interested.As can be seen that with
The increase of QP value does not need the shared ratio of ALF processing and gradually increases, and averagely reachable 93% or so, highest even up to arrives
98%.Therefore, under the premise of not sacrificing video quality, the ALF treatment process of non-smooth region interested can be skipped, thus
Good compromise processing is accomplished in code efficiency and algorithm complexity.
It is as shown in table 3 that ratio shared by the processing region ALF is not needed in smooth region interested.As can be seen that skipping
The ALF treatment process in the region ROISR can effectively excavate the visual redundancy information in area-of-interest, further decrease ALF
Algorithm complexity.
Compare as shown in table 4 and table 5 about brightness BD-rate and PSNR.It can be seen that it is proposed that algorithm with original
ALF algorithm coding efficiency is suitable, and average 0.05%, the PSNR that only increases of three viewpoint BD-rate averagely only reduces 0.005db.
Slowly, the JND model that we use can accurately find out the coding for not needing ALF processing to the movement of Poznan_Hall2 video sequence
Unit, therefore, it is proposed that algorithm can excavate the video-aware redundancy in the video sequence well, improve the video sequence
The code efficiency of column.Three viewpoint BD-rate have respectively reduced 0.25%, 0.03% and 0.42%, PSNR and have only reduced respectively
0.007db, 0.003db and 0.011db.Poznan_Street video sequence texture is complicated, it is proposed that algorithm be difficult to distinguish
Human eye area-of-interest and regions of non-interest out, it is therefore, worst to the improvement of the video sequence coding performance.On the whole,
It is proposed that algorithm can still obtain preferable coding efficiency.Compared with other algorithms, it is proposed that algorithm BD-rate
Improve 0.59%, PSNR than it and averagely improve 0.036db, therefore, it is proposed that the code efficiency of algorithm be obviously better than it
Its algorithm.
The comparison result of complexity is as shown in table 6.As can be seen that the original ALF algorithm of complexity ratio HTM 6.1 of the invention
Complexity reduces by 51.81%.Therefore, it is proposed that algorithm can excavate the feeling redundancy of video well, can rapidly find out
Should not ALF processing region.By analyze above it is found that it is proposed that the code efficiency of algorithm be better than original ALF algorithm.With
Other algorithms are compared, although it is proposed that algorithm complexity increase 0.18% than it, in code efficiency and complexity
Under comprehensively considering, it is proposed that the coding efficiency of algorithm be better than other algorithms.
Subjective quality comparing result is as Figure 2-Figure 5.As can be seen that Poznan_Hall2 video sequence uses other calculations
After method processing, nearby burr protrusion is more severe for thigh, apparent distortion occurs, however it is proposed that algorithm have substantially out
Existing burr phenomena, this is because it is proposed that algorithm use Canny operator to distinguish the texture information of video, protect well
The quality of Video Edge contour area is protected.After Poznan_Street video sequence uses other algorithm process, personage's buttocks mould
Paste it is more apparent, however it is proposed that algorithm protect the details in figure well, nearby details area is clearly very for personage's buttocks
It is more.Therefore, it is proposed that the subjective quality of algorithm be better than other algorithms.
In conjunction with above each figure as can be seen that experimental result meets expection, in the case where reconstructed image quality does not decline,
Considerably reduce the complexity of auto-adaptive loop filter.
1 regions of non-interest of table skips ALF treated performance evaluation (%)
Ratio shared by ALF processing region (%) is not needed in the non-smooth region interested of table 2
Ratio shared by the region ALF (%) is not needed in the smooth region interested of table 3
4 difference ALF algorithm objective quality brightness Y BD-rate of table compares (%)
5 difference ALF algorithm objective quality PSNR of table compares (db)
6 difference ALF algorithm complexity of table compares (%)
Claims (2)
1. a kind of based on the quick self-adapted loop circuit filtering method that can just perceive distortion model, it is characterised in that: including following step
It is rapid:
Step 1: establish JND model:
JND (x, y)=Tl(x,y)+Tt(x,y)-Cl,t·min{Tl(x,y),Tt(x,y)} (1)
(1) in formula, TlThe minimum that (x, y) indicates that the brightness shielding effect of the region pixel I (x, y) generates can perceive threshold value,
Calculation method are as follows:
(2) in formula,Indicate the average background brightness value of pixel I (x, y) surrounding 5x5 window, calculation method are as follows:
(3) in formula, B (ij) is the low-pass filter of 5x5:
(1) in formula, TtThe minimum that (x, y) indicates that the contrast shielding effect of the region pixel I (x, y) generates can perceive threshold value,
Its calculation method are as follows:
Tt(x, y)=η G (x, y) We (x, y) (5)
(5) in formula, We (x, y) indicates the weighted value of fringe region and texture region;η indicates contrast shielding effect weighting system
Number;G (x, y) indicates to carry out pixel I (x, y) maximum value after four different directions edge detections, calculation method are as follows:
Step 2: picture frame is divided into region of interest ROI and regions of non-interest RONI using the JND model:
Wherein, JNDLCUIndicate that the sum of the JND value of all pixels in LCU, the JND value are calculated using the JND model;
JNDframeIndicate the average value of the sum of JND value of all pixels in picture frame, calculation method are as follows:
Wherein, N indicates the number of LCU in described image frame;
Step 3: RONI being divided into non-smooth region RONISR interested and non-texture region interested using Canny operator
RONITR:
Wherein, k indicates the division threshold value of RONISR and RONITR, CannyRONIIt indicates to belong to boundary in the LCU of regions of non-interest
Ratio shared by pixel, calculation method are as follows:
CannyRONI=∑edge/M (12)
Wherein, ∑edgeIndicate the number of boundary pixel in LCU, M indicates the number of pixel in LCU;
Step 4: ROI is divided into smooth region ROISR interested and texture region ROITR interested using Canny operator:
Wherein, m indicates the division threshold value of ROISR and ROITR, CannyROIIt indicates to belong to boundary pixel in the LCU of area-of-interest
Shared ratio, calculation method and CannyRONICalculation method it is identical;
Step 5: ring adaptive loop filter processing: being carried out to the LCU in the RONITR and ROITR of described image frame luminance component
Road filtering;LCU in GOP each to described image frame in the RONITR and ROITR of the 0th layer of chromatic component carries out loop filter
Wave.
2. the quick self-adapted loop circuit filtering method according to claim 1 that can just perceive distortion model, which is characterized in that
In the step 1, weighted value We (x, the y) value of fringe region and texture region is 1;Contrast shielding effect weighting coefficient η
Value is 0.018.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510396817.XA CN105141967B (en) | 2015-07-08 | 2015-07-08 | Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510396817.XA CN105141967B (en) | 2015-07-08 | 2015-07-08 | Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105141967A CN105141967A (en) | 2015-12-09 |
CN105141967B true CN105141967B (en) | 2019-02-01 |
Family
ID=54727153
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510396817.XA Active CN105141967B (en) | 2015-07-08 | 2015-07-08 | Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105141967B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107547895B (en) * | 2016-06-29 | 2020-02-18 | 腾讯科技(深圳)有限公司 | Image processing method and device |
CN108419081B (en) * | 2018-03-28 | 2020-06-05 | 佛山市顺德区中山大学研究院 | Feature-based interframe coding method |
CN114885165B (en) * | 2022-06-30 | 2023-04-04 | 深圳比特微电子科技有限公司 | Image frame loop filtering method and device, electronic equipment and storage medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104219525A (en) * | 2014-09-01 | 2014-12-17 | 国家广播电影电视总局广播科学研究院 | Perceptual video coding method based on saliency and just noticeable distortion |
CN104469386A (en) * | 2014-12-15 | 2015-03-25 | 西安电子科技大学 | Stereoscopic video perception and coding method for just-noticeable error model based on DOF |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100750138B1 (en) * | 2005-11-16 | 2007-08-21 | 삼성전자주식회사 | Method and apparatus for image encoding and decoding considering the characteristic of human visual system |
US20100086063A1 (en) * | 2008-10-02 | 2010-04-08 | Apple Inc. | Quality metrics for coded video using just noticeable difference models |
-
2015
- 2015-07-08 CN CN201510396817.XA patent/CN105141967B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104219525A (en) * | 2014-09-01 | 2014-12-17 | 国家广播电影电视总局广播科学研究院 | Perceptual video coding method based on saliency and just noticeable distortion |
CN104469386A (en) * | 2014-12-15 | 2015-03-25 | 西安电子科技大学 | Stereoscopic video perception and coding method for just-noticeable error model based on DOF |
Non-Patent Citations (2)
Title |
---|
"AVS encoding optimization with perceptual just noticeable distortion model";Qi Cai.et al;《9th International Conference on Information,Communication & Signal Processing》》;20131113;全文 |
"一种基于多视点视频的低复杂度自适应环路滤波算法";罗丽冬等;《光电子激光》;20140215;全文 |
Also Published As
Publication number | Publication date |
---|---|
CN105141967A (en) | 2015-12-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6158384B2 (en) | Filtering and edge coding | |
KR101433501B1 (en) | Apparatus and method of adaptive offset restoration for video coding | |
JP5270573B2 (en) | Method and apparatus for detecting block artifacts | |
CN106131552B (en) | Method and apparatus for encoding video using adaptive bias processing | |
CN117640942A (en) | Coding method and device for video image | |
CN105141967B (en) | Based on the quick self-adapted loop circuit filtering method that can just perceive distortion model | |
Li et al. | A scalable coding approach for high quality depth image compression | |
KR20140124919A (en) | A method for adaptive illuminance compensation based on object and an apparatus using it | |
Paul | High performance adaptive deblocking filter for H. 264/AVC | |
US8369423B2 (en) | Method and device for coding | |
Sonoda et al. | Depth image compression by colorization for Intel RealSense depth cameras | |
CN113225556B (en) | Video coding method | |
KR101647344B1 (en) | Method and Device for Encoding Depth Image | |
Lan et al. | Multisensor collaboration network for video compression based on wavelet decomposition | |
CN103997653A (en) | Depth video encoding method based on edges and oriented toward virtual visual rendering | |
Xu et al. | Adaptive depth truncation filter for MVC based compressed depth image | |
CN113411575B (en) | Image processing apparatus, method and storage medium for performing quality optimized deblocking | |
CN107547895B (en) | Image processing method and device | |
CN113542864A (en) | Video flower screen area detection method, device, equipment and readable storage medium | |
Dorea et al. | A direction-adaptive in-loop deartifacting filter for video coding | |
KR20030078633A (en) | Method of processing digital images for low-bit rate applications | |
CN103957422A (en) | Depth video encoding method for virtual view rendering based on edge | |
Zhang et al. | Adaptive depth edge sharpening for 3D video depth coding | |
Ko et al. | A new in-loop filter for depth map coding in HEVC | |
CN111327909B (en) | Rapid depth coding method for 3D-HEVC |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |