CN107347153A - The quality evaluating method of longitude and latitude figure panoramic video - Google Patents
The quality evaluating method of longitude and latitude figure panoramic video Download PDFInfo
- Publication number
- CN107347153A CN107347153A CN201610916893.3A CN201610916893A CN107347153A CN 107347153 A CN107347153 A CN 107347153A CN 201610916893 A CN201610916893 A CN 201610916893A CN 107347153 A CN107347153 A CN 107347153A
- Authority
- CN
- China
- Prior art keywords
- latitude
- frame
- longitude
- quality
- panoramic video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/36—Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
Abstract
The invention discloses a kind of quality evaluating method of longitude and latitude figure panoramic video, including:For the panoramic video of two longitude and latitude bitmap-formats to be evaluated with as reference, respectively carry out each frame and divide latitude down-sampling, obtain corresponding two irregular shape videos;Mean square error, the quality and oeverall quality frame by frame between two irregular shape videos are calculated frame by frame, so as to realize the quality evaluation of longitude and latitude figure panoramic video.The advantages of above method is:1. consider the geometry deformation changed between sphere and longitude and latitude figure;2. all information on longitude and latitude figure are considered as far as possible.Therefore, the present invention is better than traditional PSNR and S PSNR.
Description
Technical field
The present invention relates to video quality evaluation technical field, more particularly to a kind of quality evaluation side of longitude and latitude figure panoramic video
Method.
Background technology
With the progress of video acquisition and treatment technology, panoramic video starts to step into the life of people.Panoramic video
It is centered on video camera, video record simultaneously obtained from is carried out to the scenery at each visual angle.People are in viewing panoramic video
When, it can arbitrarily switch visual angle, it is seemingly on the spot in person.Compared with traditional two dimensional surface video, panoramic video can give user
Preferably experience and feeling of immersion are brought, is the indispensable part of following virtual reality technology.
But the original data volume of panoramic video is more much bigger than conventional two-dimensional planar video, there is an urgent need to efficient video
Compression method.In order to weigh the performance of video compress, it is necessary to compare the difference between the video after compression and the video before compression
(distortion), i.e., the video after being compressed using the video before compression as reference pair carry out quality evaluation.
Substantially, what panoramic video recorded is the video information on sphere.Because sphere is not easy to represent, panoramic video is general
Expressed using longitude and latitude bitmap-format.Longitude and latitude figure by the point of sphere according to its longitude, dimension map to two dimensional surface, for example, through
Degree is mapped as row coordinate, latitude is mapped as row coordinate, so as to be rectangle plane by development of a sphere.The video of longitude and latitude bitmap-format can be with
Coding is directly compressed using the encoder of traditional two dimensional surface video.
For conventional two-dimensional planar video, there is reference mass evaluation that Y-PSNR (PSNR) is commonly used.But PSNR pairs
The video of longitude and latitude bitmap-format is less reasonable, because longitude and latitude figure compares sphere there occurs very big geometry deformation, closer to the two poles of the earth
Region is stretched bigger, and PSNR gives same weight for diverse location in plane, it is impossible to accurately embodies sphere
The distortion of itself.
A kind of improved sphere Y-PSNR (S-PSNR) is researched and proposed, this method equably takes one on sphere
A little positions, these positions are mapped on longitude and latitude figure, and only calculate the original video on these positions and the video after compression it
Between distortion.Although so doing and solving the problems, such as geometry deformation, limited on sphere has only been taken to click through due to S-PSNR
Row calculates, and has in fact only used a part of region on longitude and latitude figure, and ignores the distortion in other regions.
The content of the invention
It is an object of the invention to provide a kind of quality evaluating method of longitude and latitude figure panoramic video, so as to pair warp and weft bitmap-format
Panoramic video carries out objective metric with the difference (distortion) before compression upon compression, and then helps to optimize panoramic video compression.
The purpose of the present invention is achieved through the following technical solutions:
A kind of quality evaluating method of longitude and latitude figure panoramic video, including:
For the panoramic video of two longitude and latitude bitmap-formats to be evaluated with as reference, respectively carry out each frame and divide latitude
Down-sampling, obtain corresponding two irregular shape videos;
Mean square error, the quality and oeverall quality frame by frame between two irregular shape videos are calculated frame by frame, so as to real
The quality evaluation of existing longitude and latitude figure panoramic video.
Carrying out each frame divides latitude down-sampling to include:
Mapping relations according to sphere to longitude and latitude figure, the over-sampling rate of every latitude is calculated, further according to over-sampling rate
It is reciprocal that down-sampling is carried out to every latitude.
It is described calculate every latitude over-sampling rate the step of include:
With row coordinate representation latitude, branch's down-sampling is carried out;Assuming that the height of longitude and latitude figure is N, latitude be from -90 degree to+
90 degree, then latitude θ corresponding to the i-th rowiFor:
θi=(180 × i-90 × N-90)/(N-1), i=1,2 ..., N
Assuming that the sample rate of position is 1 under the line, then in latitude θiPosition, over-sampling rate is 1/cos (θi)。
It is described that every latitude progress down-sampling is included according to the reciprocal of over-sampling rate:
Assuming that the width of longitude and latitude figure is M, then need M pixel being down sampled to round (M*cos (θi)) individual pixel, its
Middle round represents to round;
Down-sampling is realized by linear interpolation or cubic interpolation;Wherein, linear interpolation process is as follows:
Order, K=round (M*cos (θi)), the value of original M pixel is respectively x (1) ..., x (M), the value point after interpolation
Not Wei y (1) ..., y (K), then have:
Y (j)=x (j × M/K), j=1,2 ..., K;
If j × M/K is integer, known to corresponding value;
Otherwise, a=floor (j × M/K) is made, b=ceil (j × M/K), floor and ceil are to remove whole and take respectively
It is whole, then have:
Y (j)=(j × M/K-a) × x (b)+(b-j × M/K) × x (a).
Mean square error, quality frame by frame and the oeverall quality calculated frame by frame between two irregular shape videos includes:
Assuming that in the current frame, the difference of pixel value is d corresponding to the i-th row jth row of two irregular shape videosij;
Then mean square error RMSE is:
RMSE=sqrt ((ΣI=1..NΣJ=1..round (M*cos (θi))dij 2)/(ΣI=1..Nround(M*cos(θi))))
Wherein, sqrt represents extraction of square root, θiFor latitude value corresponding to the i-th row;
Then quality Q corresponding to present frame is:
Q=10*log10(c2/RMSE);
Wherein, c is constant;
Then the oeverall quality of video is averaging to obtain by quality frame by frame;Or mean square error is averaging by mean square error frame by frame
Difference, then bring above-mentioned quality into and ask for formula to calculate oeverall quality.
As seen from the above technical solution provided by the invention, it is an advantage of the invention that:1. consider sphere and longitude and latitude
The geometry deformation changed between figure;2. all information on longitude and latitude figure are considered as far as possible.Therefore, the present invention is better than traditional
PSNR and S-PSNR.
Brief description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Accompanying drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for this
For the those of ordinary skill in field, on the premise of not paying creative work, other can also be obtained according to these accompanying drawings
Accompanying drawing.
Fig. 1 is a kind of flow chart of the quality evaluating method of longitude and latitude figure panoramic video provided in an embodiment of the present invention;
Fig. 2 is panoramic video schematic diagram provided in an embodiment of the present invention;
Fig. 3 is the schematic diagram of a certain frame of panoramic video of longitude and latitude bitmap-format provided in an embodiment of the present invention;
Fig. 4 is that the panoramic video of pair warp and weft bitmap-format provided in an embodiment of the present invention obtained after branch's down-sampling
The schematic diagram of a certain frame of irregular shape video.
Embodiment
With reference to the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Ground describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.Based on this
The embodiment of invention, the every other implementation that those of ordinary skill in the art are obtained under the premise of creative work is not made
Example, belongs to protection scope of the present invention.
The embodiment of the present invention provides a kind of quality evaluating method of longitude and latitude figure panoramic video, as shown in figure 1, it is mainly wrapped
Include:
Step 11, the panoramic video for two longitude and latitude bitmap-formats to be evaluated with as reference, are respectively carried out each
Frame divides latitude down-sampling, obtains corresponding two irregular shape videos.
In the embodiment of the present invention, the mapping relations according to sphere to longitude and latitude figure, the over-sampling rate of every latitude is calculated, then
Reciprocal according to over-sampling rate carries out down-sampling to every latitude.Detailed process is as follows:
1) the step of over-sampling rate for calculating every latitude, includes:
With row coordinate representation latitude, branch's down-sampling is carried out;Assuming that the height of longitude and latitude figure is N, latitude be from -90 degree to+
90 degree, then latitude θ corresponding to the i-th rowiFor:
θi=(180 × i-90 × N-90)/(N-1), i=1,2 ..., N
According to sphere calculation formula, θiThe sphere circumference of position is proportional to Rcos (θi), wherein R is the radius of ball;
Assuming that the sample rate of position is 1 under the line, then in latitude θiPosition, over-sampling rate is 1/cos (θi)。
2) every latitude progress down-sampling is included according to the reciprocal of over-sampling rate:
Assuming that the width of longitude and latitude figure is M, then need M pixel being down sampled to round (M*cos (θi)) individual pixel, its
Middle round represents to round;
Down-sampling can be realized by linear interpolation or cubic interpolation;Wherein, linear interpolation process is as follows:
Order, K=round (M*cos (θi)), the value of original M pixel is respectively x (1) ..., x (M), the value point after interpolation
Not Wei y (1) ..., y (K), then have:
Y (j)=x (j × M/K), j=1,2 ..., K;
If j × M/K is integer, known to corresponding value;
Otherwise, a=floor (j × M/K) is made, b=ceil (j × M/K), floor and ceil are to remove whole and take respectively
It is whole, then have:
Y (j)=(j × M/K-a) × x (b)+(b-j × M/K) × x (a).
It will be understood to those skilled in the art that linear interpolation or cubic interpolation mode except being used in embodiment,
Other conventional Downsapling methods can also be used.
As shown in Fig. 2 it is panoramic video schematic diagram;Fig. 3 is the schematic diagram of a certain frame of panoramic video of longitude and latitude bitmap-format;Figure
The signal of a certain frame of the 4 irregular shape videos obtained later for the panoramic video progress branch down-sampling of pair warp and weft bitmap-format
Figure.
It will be understood by those skilled in the art that the panoramic video of two longitude and latitude bitmap-formats to be evaluated with as reference
Handled using aforesaid way.
It will be understood by those skilled in the art that the longitude and latitude figure panoramic video as reference, longitude and latitude figure typically to be evaluated
The prototype version of panoramic video.For example, in the compression of longitude and latitude figure panoramic video, the video before compression is used as reference, after compression
The video of reconstruction carries out quality evaluation.
Step 12, the frame by frame mean square error between two irregular shape videos of calculating, quality and oeverall quality frame by frame,
So as to realize the quality evaluation of longitude and latitude figure panoramic video.
Assuming that in the current frame, the difference of pixel value is d corresponding to the i-th row jth row of two irregular shape videosij;
Then mean square error RMSE is:
RMSE=sqrt ((ΣI=1..NΣJ=1..round (M*cos (θi))dij 2)/(ΣI=1..Nround(M*cos(θi))))
Wherein, sqrt represents extraction of square root, θiFor latitude value corresponding to the i-th row;
Then quality Q corresponding to present frame is:
Q=10*log10(c2/RMSE);
Wherein, c is constant;
Then the oeverall quality of video is averaging to obtain by quality frame by frame;Or mean square error is averaging by mean square error frame by frame
Difference, then bring above-mentioned quality into and ask for formula to calculate oeverall quality, i.e., the RMSE of above formula, final meter are substituted with Averaged Square Error of Multivariate
To Q be expressed as the oeverall quality of video.
It will be understood to those skilled in the art that said process can be to two videos after down-sampling, point different face
The pixel value of color (such as red, green, blue) calculates mean square error and quality respectively, or divides brightness, colourity to calculate mean square error respectively
Difference and quality, etc..
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment can
To be realized by software, the mode of necessary general hardware platform can also be added by software to realize.Based on such understanding,
The technical scheme of above-described embodiment can be embodied in the form of software product, the software product can be stored in one it is non-easily
In the property lost storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.), including some instructions are causing a computer to set
Standby (can be personal computer, server, or network equipment etc.) performs the method described in each embodiment of the present invention.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,
Any one skilled in the art is in the technical scope of present disclosure, the change or replacement that can readily occur in,
It should all be included within the scope of the present invention.Therefore, protection scope of the present invention should be with the protection model of claims
Enclose and be defined.
Claims (5)
- A kind of 1. quality evaluating method of longitude and latitude figure panoramic video, it is characterised in that including:For the panoramic video to be evaluated with as the two longitude and latitude bitmap-formats referred to, respectively carry out each frame and divide under latitude to adopt Sample, obtain corresponding two irregular shape videos;Mean square error, the quality and oeverall quality frame by frame between two irregular shape videos are calculated frame by frame, so as to realize warp The quality evaluation of latitude figure panoramic video.
- 2. the quality evaluating method of longitude and latitude figure panoramic video according to claim 1, it is characterised in that carry out each frame point Latitude down-sampling includes:Mapping relations according to sphere to longitude and latitude figure, the over-sampling rate of every latitude is calculated, further according to the inverse of over-sampling rate Down-sampling is carried out to every latitude.
- 3. the quality evaluating method of longitude and latitude figure panoramic video according to claim 2, it is characterised in that described to calculate often The step of over-sampling rate of bar latitude, includes:With row coordinate representation latitude, branch's down-sampling is carried out;Assuming that the height of longitude and latitude figure is N, latitude is spent from -90 degree to+90, Then latitude θ corresponding to the i-th rowiFor:θi=(180 × i-90 × N-90)/(N-1), i=1,2 ..., NAssuming that the sample rate of position is 1 under the line, then in latitude θiPosition, over-sampling rate is 1/cos (θi)。
- 4. the quality evaluating method of longitude and latitude figure panoramic video according to claim 3, it is characterised in that described to be adopted according to mistake The reciprocal of sample rate includes to every latitude progress down-sampling:Assuming that the width of longitude and latitude figure is M, then need M pixel being down sampled to round (M*cos (θi)) individual pixel, wherein Round represents to round;Down-sampling is realized by linear interpolation or cubic interpolation;Wherein, linear interpolation process is as follows:Order, K=round (M*cos (θi)), the value of original M pixel is respectively x (1) ..., x (M), and the value after interpolation is respectively y (1) ..., y (K), then have:Y (j)=x (j × M/K), j=1,2 ..., K;If j × M/K is integer, known to corresponding value;Otherwise, make a=floor (j × M/K), b=ceil (j × M/K), floor and ceil be respectively remove it is whole and take it is whole, Then have:Y (j)=(j × M/K-a) × x (b)+(b-j × M/K) × x (a).
- 5. the quality evaluating method of longitude and latitude figure panoramic video according to claim 1, it is characterised in that calculate two frame by frame Mean square error between irregular shape video, quality frame by frame and oeverall quality include:Assuming that in the current frame, the difference of pixel value is d corresponding to the i-th row jth row of two irregular shape videosij;Then mean square error RMSE is:RMSE=sqrt ((ΣI=1..NΣJ=1..round (M*cos (θi))dij 2)/(ΣI=1..Nround(M*cos(θi))))Wherein, sqrt represents extraction of square root, θiFor latitude value corresponding to the i-th row;Then quality Q corresponding to present frame is:Q=10*log10(c2/RMSE);Wherein, c is constant;Then the oeverall quality of video is averaging to obtain by quality frame by frame;Or mean square error is averaging by mean square error frame by frame, then Bring above-mentioned quality into and ask for formula to calculate oeverall quality.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610916893.3A CN107347153B (en) | 2016-10-20 | 2016-10-20 | The quality evaluating method of longitude and latitude figure panoramic video |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610916893.3A CN107347153B (en) | 2016-10-20 | 2016-10-20 | The quality evaluating method of longitude and latitude figure panoramic video |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107347153A true CN107347153A (en) | 2017-11-14 |
CN107347153B CN107347153B (en) | 2019-02-26 |
Family
ID=60253768
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610916893.3A Active CN107347153B (en) | 2016-10-20 | 2016-10-20 | The quality evaluating method of longitude and latitude figure panoramic video |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107347153B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108271020A (en) * | 2018-04-24 | 2018-07-10 | 福州大学 | A kind of panoramic video quality evaluating method of view-based access control model attention model |
WO2020000522A1 (en) * | 2018-06-27 | 2020-01-02 | 深圳看到科技有限公司 | Method and device for picture quality assessment after dynamic panoramic video stream cropping |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070088516A1 (en) * | 2005-10-14 | 2007-04-19 | Stephen Wolf | Low bandwidth reduced reference video quality measurement method and apparatus |
CN101227604A (en) * | 2007-01-18 | 2008-07-23 | 上海未来宽带技术及应用工程研究中心有限公司 | Method for detecting network video quality by network injury tolerance |
CN101448173A (en) * | 2008-10-24 | 2009-06-03 | 华为技术有限公司 | Method for evaluating Internet video quality, device and system thereof |
TW201424343A (en) * | 2012-12-03 | 2014-06-16 | Arcadyan Technology Corp | Method and apparatus for testing video signals |
CN105915892A (en) * | 2016-05-06 | 2016-08-31 | 乐视控股(北京)有限公司 | Panoramic video quality determination method and system |
-
2016
- 2016-10-20 CN CN201610916893.3A patent/CN107347153B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070088516A1 (en) * | 2005-10-14 | 2007-04-19 | Stephen Wolf | Low bandwidth reduced reference video quality measurement method and apparatus |
CN101227604A (en) * | 2007-01-18 | 2008-07-23 | 上海未来宽带技术及应用工程研究中心有限公司 | Method for detecting network video quality by network injury tolerance |
CN101448173A (en) * | 2008-10-24 | 2009-06-03 | 华为技术有限公司 | Method for evaluating Internet video quality, device and system thereof |
TW201424343A (en) * | 2012-12-03 | 2014-06-16 | Arcadyan Technology Corp | Method and apparatus for testing video signals |
CN105915892A (en) * | 2016-05-06 | 2016-08-31 | 乐视控股(北京)有限公司 | Panoramic video quality determination method and system |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108271020A (en) * | 2018-04-24 | 2018-07-10 | 福州大学 | A kind of panoramic video quality evaluating method of view-based access control model attention model |
CN108271020B (en) * | 2018-04-24 | 2019-08-09 | 福州大学 | A kind of panoramic video quality evaluating method of view-based access control model attention model |
WO2020000522A1 (en) * | 2018-06-27 | 2020-01-02 | 深圳看到科技有限公司 | Method and device for picture quality assessment after dynamic panoramic video stream cropping |
US11438651B2 (en) | 2018-06-27 | 2022-09-06 | Kandao Technology Co., Ltd. | Method and device for picture quality assessment after dynamic panoramic video stream cropping |
Also Published As
Publication number | Publication date |
---|---|
CN107347153B (en) | 2019-02-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10552989B2 (en) | Point cloud attribute compression method based on KD tree and optimized graph transformation | |
WO2020001168A1 (en) | Three-dimensional reconstruction method, apparatus, and device, and storage medium | |
CN110688692B (en) | Two-three-dimensional linkage and superposition display method based on structured BIM model | |
CN108564527B (en) | Panoramic image content completion and restoration method and device based on neural network | |
CN103345771A (en) | Efficient image rendering method based on modeling | |
CN104715496B (en) | Image prediction method, system and device based on three-dimensional point cloud model under cloud environment | |
CN108347611B (en) | Optimization method of coding block-level Lagrange multiplier for theodolite | |
CN108668108B (en) | Video monitoring method and device and electronic equipment | |
US20170200307A1 (en) | Constructing a 3d structure | |
CN109992809A (en) | A kind of construction method of buildings model, device and storage device | |
CN111435551B (en) | Point cloud filtering method and device and storage medium | |
CN113313832B (en) | Semantic generation method and device of three-dimensional model, storage medium and electronic equipment | |
CN107347153B (en) | The quality evaluating method of longitude and latitude figure panoramic video | |
CN114756937A (en) | Visualization system and method based on UE4 engine and Cesium framework | |
CN111273877B (en) | Linkage display platform and linkage method for live-action three-dimensional data and two-dimensional grid picture | |
CN108243333A (en) | The projecting method and back projection method of panoramic video and picture | |
CN115359173A (en) | Virtual multi-view video generation method and device, electronic equipment and storage medium | |
Andújar et al. | Visualization of Large‐Scale Urban Models through Multi‐Level Relief Impostors | |
CN102281440A (en) | Geometric image based error-protection coding method of three-dimensional mesh model | |
CN110097605B (en) | Point cloud compression encoder key parameter optimization method based on support vector machine | |
CN107093197B (en) | Animation compression method based on local cylindrical coordinates | |
AU2020101803A4 (en) | Method for Optimizing the Coding Block-Level Lagrange Multiplier of Longitude-Latitude Image | |
Dunn et al. | Resolution-defined projections for virtual reality video compression | |
CN110321233B (en) | Model slice transmission method based on tree-shaped storage structure | |
CN113613011A (en) | Light field image compression method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |