CN101127911A - A block size computing method and transmission method for cylinder panorama video code - Google Patents

A block size computing method and transmission method for cylinder panorama video code Download PDF

Info

Publication number
CN101127911A
CN101127911A CN 200710175443 CN200710175443A CN101127911A CN 101127911 A CN101127911 A CN 101127911A CN 200710175443 CN200710175443 CN 200710175443 CN 200710175443 A CN200710175443 A CN 200710175443A CN 101127911 A CN101127911 A CN 101127911A
Authority
CN
China
Prior art keywords
video
code
block size
size
coding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200710175443
Other languages
Chinese (zh)
Other versions
CN100542298C (en
Inventor
代锋
张勇东
张冬明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Computing Technology of CAS
Original Assignee
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Computing Technology of CAS filed Critical Institute of Computing Technology of CAS
Priority to CN 200710175443 priority Critical patent/CN100542298C/en
Publication of CN101127911A publication Critical patent/CN101127911A/en
Application granted granted Critical
Publication of CN100542298C publication Critical patent/CN100542298C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The utility model provides a calculation method of block size of cylindrical panoramic video coding and a transmission method, which is characterized in that: a code rate of the panoramic video coding is got without direct blocking pre-coding to the cylindrical panoramic video; blocking pre-coding to the video is operated according to an optional block size, and the code rate in the block size is got; the increment of the two code rate is computed; the impact factor of the video is computed based on the increment; and the optimal coding block size is computed based on the impact factor. The utility model also provides a transmission method of the cylindrical panoramic video block coding based on the block size computed by the calculation method. The utility model has the advantages of computing the optimal coding block size by twice pre-coding, and having lowest transmission data size in the same visual quality of users by using the optimal block size to code the cylindrical panoramic video.

Description

A kind of block size computing method of cylinder panorama video code and transmission method
Technical field
The present invention relates to video and image coding and decoding field, particularly the block size computing method of cylinder panorama video code and based on the transmission technology of the cylinder panorama video code of piecemeal.
Background technology
Panoramic video is a kind of emerging medium, compares with traditional video, and it is big to have a visual field, resolution height, characteristics such as data volume is big.Utilize panoramic video, observer's viewpoint is constant, change direction of observation and can observe whole scenes on every side, and common two-dimensional video has only reacted certain part of panoramic video.
Cylinder panorama video is a kind of common panoramic video, and it is equivalent to a virtual video camera, and the three-dimensional body in the space is projected on the cylinder.The generation of cylinder panorama video can utilize multi-cam or the collection of single camera acquisition system to form.Because projection surface is a cylinder, so above-below direction has some blind spots.
Because the field range of panoramic video is 5~6 times of ordinary video, providing to the user under the situation of identical visual quality, the data volume of panoramic video is 5~6 times of ordinary video.If according to traditional video transmission scheme, the use of panoramic video under network environment becomes difficult.But, because at synchronization, certain part that the required content of seeing of user is a panoramic video is so block encoding and transmission become the common scheme of panoramic video Network Transmission.
In the block encoding of panoramic video, the size of piecemeal is for the code efficiency and the transmission region important influence of panoramic video, and these two factors need are directly determining data quantity transmitted.If the encoding block size is little, then transmission region is less, but code efficiency can be lower; If the encoding block size is big, then code efficiency is higher, but transmission region is also bigger.So under identical visual quality, different encoding block sizes, it is different needing data quantity transmitted.But in systems more both domestic and external, do not consider the size of piecemeal basically now, generally according to the experience decision, this has just caused the increase of transmitted data amount to block size.
Summary of the invention
The objective of the invention is provides a kind of computational methods fast and effectively for the selection of cylinder panorama video code piece size, makes under local visual situation identical in quality required data quantity transmitted minimum.
To achieve these goals, the invention provides a kind of cylinder panorama video code block size computing method, utilize the width of coding panoramic video, the panoramic video peak width that relates to, and the information calculations such as factor of influence of current video obtain optimum code piece size.Comprise following steps:
Cylinder panorama video does not carry out the direct precoding of piecemeal, the code check R of the panorama video code that obtains 0
Cylinder panorama video carries out the piecemeal precoding according to an optional piece size a, obtains the encoder bit rate R under piece size a a
According to formula ω a=(R a-R 0)/R 0, calculate code check R aThan code check R 0Increment ω a
According to formula C=ω aAW/ (W-a) calculates the factor of influence C of this video, and wherein W represents the width of this video;
According to formula a me = [ CM / ( 1 - C W ) ] , Calculate the optimum code piece size a of this video Me, wherein M is the target area width.
Based on this block size computing method, the present invention also provides a kind of cylinder panorama video code transmission method, comprises the steps:
Calculate the piece size a that gets according to said method MeThis video is carried out piecemeal;
Behind the piecemeal, the sequence of each piece is independently encoded;
Select the data behind the required coding to transmit.
By cylinder panorama video code block size computing method provided by the invention, can access a piece size that compression efficiency is higher and transmission region is more appropriate that makes panoramic video.Need the data quantity transmitted minimum thereby make.
Description of drawings
Below, describe embodiments of the invention in conjunction with the accompanying drawings in detail, wherein:
Fig. 1 is a cylinder panorama video piecemeal schematic diagram.
Fig. 2 is ω aWith encoding block number and ω aGraph of a relation with the encoding block size.
Fig. 3 is the operational flow figure of cylinder panorama video optimum code block size computing method.
Fig. 4 is compression of cylinder panorama video piecemeal and transmission schematic diagram.
Embodiment
The partial view visual quality is determined by panoramic video original coding quality.In the process of coding, control the quality of panorama video code with quantization parameter (QP).Need under the certain condition of QP, calculate block size a and make the data quantity transmitted minimum, even transmission code rate r aMinimum block size a.So at first calculate transmission code rate r aFunctional relation with block size a.
If the width of target area is a M macro block, dividing block size is a macro block, as shown in Figure 1.Because the position that the target area is dropped in the panoramic video can change, thereby the number of relevant piecemeal also can change thereupon.A kind of situation be the right margin of target area in the scope of a ', relevant piecemeal number is N ', another kind of situation is that the right margin of target area is at a " scope in, relevant piecemeal number is N ".Then:
Formula 1
The probability that makes the target area drop in a ' scope is P ', and a " probability in the scope is P " is dropped in the target area.Thereby, when minute block size is a, the relevant range width W aDesired value can calculate by following formula:
W a = N ′ · a · P ′ ′ + N ′ ′ · a · P ′ ′
Formula 2
Transmission code rate is the code check sum of all relevant piecemeals that need transmit, and calculates with following formula.
r a = N a N · R a = W a W · R a Formula 3
Wherein, r aTransmission code rate when the expression block size is a, R aThe encoder bit rate of expression block size panoramic video when being a, the code check sum of all block encodings just, N aComprise the number of the macro block of target area when representing the panoramic video block size for a, N represents panoramic video macro block sum, W aThe width sum of the macro block that comprises the target area when expression panoramic video block size is a, W represents the width of panoramic video.
In order to calculate under the certain situation of original coding quality the optimum code piece size a of transmission code rate minimum Me, the most direct method utilizes above-mentioned formula to calculate the optimum code size by enumerating mode exactly.Embodiment obtains panorama video code code check R at first each possible block size is all encoded a, calculate the width W of relevant piecemeal then according to formula 2, calculate transmission code rate r according to formula 3 then again a, by the transmission code rate of more different block sizes, the block size that obtains the transmission code rate minimum is exactly the optimum code size at last.Example forest, village, bridge and four sequences of assembly hall as shown in table 2 enumerate result of calculation.
Adopt each possible block size to encode to panoramic video, through relatively finding the optimum code piece, such method is unpractical in actual applications.The present invention proposes a kind of piece of optimum code fast size Selection method based on the inverse ratio model.
At first define λ aBe R aAnd R 0Ratio.
λ a = R a R 0 Formula 4
R wherein aPanorama video code code check when being of a size of a for encoding block, R 0Be the panoramic video encoder bit rate during piecemeal not.According to the relation between panorama video code block size and the code efficiency, λ a is a real number more than or equal to 1 all the time.
By formula 3, formula 4 can obtain:
r a = W a W · R a = λ a · R 0 · W a W . Formula 5
For same encoding condition and panoramic video, different encoding block size a, R 0With W is identical, so definition:
P aaW aFormula 6
Make P as long as calculate aMinimum encoding block size promptly gets optimum code piece size.
Further, definition
ω a = R a - R 0 R 0 = λ a - 1 . Formula 7
Table 1 has been listed the bridge sequence under different encoding block sizes, different ω a,, used 4 different quantization parameters (QP) and encoded for each encoding block size.
Table Bridge 1 beam sequence ω aWith the encoding block size relationship
The encoding block number Encoding block size a ??ω a
??QP=28 ??QP=32 ??QP=36 ??QP=40
??1 ??3 ??4 ??6 ??8 ??10 ??12 ??15 ??20 ??30 ??40 ??60 ??120 ??120 ??40 ??30 ??20 ??15 ??12 ??10 ??8 ??6 ??4 ??3 ??2 ??1 ??0.00% ??0.25% ??0.39% ??0.62% ??0.80% ??0.78% ??1.21% ??1.50% ??1.86% ??2.92% ??3.94% ??5.96% ??11.96% ??0.00% ??0.41% ??0.50% ??0.95% ??1.11% ??1.17% ??1.83% ??2.18% ??2.81% ??4.59% ??6.16% ??9.33% ??18.47% ??0.00% ??0.52% ??0.87% ??1.42% ??1.87% ??2.06% ??2.89% ??3.61% ??4.70% ??7.38% ??9.96% ??14.94% ??29.38% ??0.00% ??0.83% ??1.47% ??2.15% ??3.03% ??3.20% ??4.67% ??5.80% ??7.51% ??11.73% ??15.65% ??23.37% ??45.56%
The data of table in the arrangement, ω aAnd the relation between encoding block number and the encoding block size is shown in Fig. 2 a.ω aAnd proportional relation between the encoding block number, owing to be inverse relation between encoding block number and the encoding block size, so ω aAnd be inverse relation between the encoding block size, shown in Fig. 2 b.
Because ω aBe inverse relation with the encoding block size, the present invention sets up a model between the two, describes between the two to concern.
ω a = C a - C W . Formula 8
For same video C is constant, when the panoramic video block encoding, and a=W, at this time ω a=0, meet us for ω aDefinition.
Can obtain the computing formula of C by formula 8:
C = ω a · a · W W - a . Formula 11
Thus, video is encoded by any one encoding block size a, calculate ω a, then can computational constant C.
By formula 6,7,8 calculate:
P a = W a · λ a = ( M + a ) · ( 1 + ω a ) = ( M + a ) · ( 1 + C a - C W ) . Formula 9
Calculate P aTo the derivative of a is 0 point, is P aExtreme point, this moment a value be optimum code piece size.
P a ′ = 0 ⇒ a me = [ CM 1 - C W ] . Formula 10
Can calculate optimum code piece size a in view of the above Me
According to above-mentioned analysis, according to the width of panoramic video, information calculations encoding block sizes such as target area width.This computational methods embodiment is as follows as shown in Figure 3:
At first, cylinder panorama video to be encoded is not carried out piecemeal, integral body is carried out precoding, the code check R of the panorama video code that obtains 0Then, select a piece size a arbitrarily, carry out the piecemeal precoding to panoramic video, obtain the encoder bit rate R under piece size a according to this piece size for cylinder panorama video to be encoded aAgain according to formula ω a=(R a-R 0)/R 0Calculate precoding code check R aThan code check R 0Increment ω aAgain then, according to formula C=ω aAW/ (W-a) calculates the factor of influence C of current cylinder panorama video, and wherein W represents the width of panoramic video.At last, according to formula a me = [ CM / ( 1 - C W ) ] , Calculate the optimum code piece size a of cylinder panorama video Me, wherein M is the target area width.
According to the block size computing method of above-mentioned cylinder panorama video code, the transmission method of cylinder panorama video provided by the invention, as shown in Figure 4, concrete steps are as follows:
At first calculate the piece size that gets panoramic video is carried out piecemeal according to the block size computing method of above-mentioned cylinder panorama video code.
According to above piecemeal, the sequence of each piece is independently encoded then.
Select the data behind the needed coding to transmit again then.Can select data according to the current visual angle of user at this.Transmission medium can be the appropriate combination of internet, wireless network, local area network (LAN), optic network, other suitable transmission medium or these transmission mediums.
Last decoding end receives after the data, and these piece sequences are independently decoded and projective transformation, obtains required image.
Table 2 encoding block size and transmission code rate corresponding relation
Target image The piece size Code check Size results of the present invention Target image The piece size Code check Size results of the present invention
Forest ??120 ??1169.51 ??2 The village ??120 ??1532.30 ??2
??40 ??501.99 ??40 ??658.15
??30 ??419.01 ??30 ??549.88
??20 ??335.82 ??20 ??439.99
??15 ??294.30 ??15 ??385.58
??12 ??269.40 ??12 ??355.04
??10 ??253.48 ??10 ??334.28
??9 ??245.06 ??9 ??324.93
??8 ??237.00 ??8 ??313.86
??7 ??228.95 ??7 ??303.66
??6 ??221.24 ??6 ??292.81
??5 ??214.00 ??5 ??285.28
??4 ??206.19 ??4 ??275.20
??3 ??199.67 ??3 ??267.96
??2 ??194.89 ??2 ??264.60
??1 ??197.49 ??1 ??268.43
Bridge ??120 ??2258.58 ??2 Assembly hall ??120 ??1428.35 ??3
??40 ??970.41 ??40 ??620.26
??30 ??809.77 ??30 ??517.14
??20 ??649.31 ??20 ??415.00
??15 ??569.17 ??15 ??364.61
??12 ??520.25 ??12 ??334.28
??10 ??489.85 ??10 ??317.12
??9 ??474.28 ??9 ??306.59
??8 ??458.48 ??8 ??296.18
??7 ??442.84 ??7 ??289.59
??6 ??427.26 ??6 ??282.14
??5 ??412.52 ??5 ??272.52
??4 ??398.51 ??4 ??263.64
Target image The piece size Code check Size results of the present invention Target image The piece size Code check Size results of the present invention
??3 ??385.68 ??3 ??259.50
??2 ??376.08 ??2 ??261.07
??1 ??379.30 ??1 ??261.79
Table 2 has illustrated in the transmission of cylinder panorama video image the encoding block size to the influence of actual transmissions code check, its with resolution be 4 image sequences of 1920x352 at quantization parameter QP=28, be transmitted as example under the condition of target area M=20 and describe.For being target image with forest, village, bridge and assembly hall, carry out the coding transmission test respectively with the different masses size, can observe different images optimum block size separately from each piece size corresponding code rate.For example, when target image was forest, the piece size equaled at 2 o'clock, and code check is 194.89, and this value is the minimum value of all different masses size encoder bit rates.The piece size that adopts the inventive method to calculate also is 2, and is consistent with the optimal value of enumerating test.
Enumerate calculating optimal result and the inventive method comparison sheet as a result during table 3 different Q P
Image sequence ??QP Full blast piece size
??1920x352 ??2880x528
Enumerate result of calculation The inventive method result Enumerate result of calculation The inventive method result
Forest ??28 ??2 ??2 ??2 ??2
??32 ??2 ??2 ??3 ??3
??36 ??2 ??2 ??3 ??3
??40 ??3 ??3 ??4 ??4
Bridge ??28 ??2 ??2 ??2 ??2
??32 ??2 ??2 ??3 ??3
??36 ??2 ??2 ??3 ??3
??40 ??3 ??3 ??4 ??4
The village ??28 ??2 ??2 ??3 ??3
??32 ??2 ??2 ??3 ??3
??36 ??3 ??3 ??4 * ??3 *
??40 ??3 ??3 ??4 ??4
Assembly hall ??28 ??3 ??3 ??3 ??3
??32 ??3 ??3 ??4 ??4
??36 ??4 ??4 ??5 * ??6 *
??40 ??4 ??4 ??6 ??6
It is that 4 sequences of target image are utilized the actual test of piece size coding full blast piece size that obtains and the optimum piece size comparative result that obtains with formula of the present invention enumerated that table 3 has illustrated under different quantization parameter QP respectively with forest, village, bridge and assembly hall.
Above by analysis form learns, through enumerating method to calculate full blast block size and the block size basically identical that utilizes the present invention to calculate, what have only band asterisk in the table 4 is not quite identical, but the two is also very approaching.
Should be noted that and understand, under the situation that does not break away from the desired the spirit and scope of the present invention of accompanying Claim, can make various modifications and improvement the present invention of foregoing detailed description.Therefore, the scope of claimed technical scheme is not subjected to the restriction of given any specific exemplary teachings.

Claims (4)

1. the block size computing method of a cylinder panorama video code comprises the following steps:
Cylinder panorama video does not carry out the direct precoding of piecemeal, the code check R of the panorama video code that obtains 0
This video carries out the piecemeal precoding according to an optional piece size a, obtains the encoder bit rate R under piece size a a
According to formula ω a=(R a-R 0)/R 0, calculate described code check R aThan described code check R 0Increment ω a
According to formula C=ω aAW/ (W-a) calculates the factor of influence C of this video, and wherein W represents the width of panoramic video;
According to formula a me = [ CM / ( 1 - C W ) ] , Calculate the optimum code piece size a of this video Me, wherein M is the target area width.
2. a cylinder panorama video code transmission method comprises the following steps:
Calculate the piece size a that gets according to the method for claim 1 MeDescribed panoramic video is carried out piecemeal;
After the described piecemeal, the sequence of each piece is independently encoded;
Data are transmitted after selecting required coding.
3. method as claimed in claim 2, it is characterized in that described transfer of data after, decoding end receives this data, and described sequence independently decoded and projective transformation.
4. method as claimed in claim 2 is characterized in that the data based user perspective behind the described required coding is selected.
CN 200710175443 2007-09-29 2007-09-29 A kind of block size computing method of cylinder panorama video code and transmission method Active CN100542298C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200710175443 CN100542298C (en) 2007-09-29 2007-09-29 A kind of block size computing method of cylinder panorama video code and transmission method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200710175443 CN100542298C (en) 2007-09-29 2007-09-29 A kind of block size computing method of cylinder panorama video code and transmission method

Publications (2)

Publication Number Publication Date
CN101127911A true CN101127911A (en) 2008-02-20
CN100542298C CN100542298C (en) 2009-09-16

Family

ID=39095807

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200710175443 Active CN100542298C (en) 2007-09-29 2007-09-29 A kind of block size computing method of cylinder panorama video code and transmission method

Country Status (1)

Country Link
CN (1) CN100542298C (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103780909A (en) * 2012-10-22 2014-05-07 古如罗技微系统公司 Encoder, decoder, coding method, and decoding method
CN106060513A (en) * 2016-06-29 2016-10-26 深圳市优象计算技术有限公司 Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing
CN106412582A (en) * 2016-10-21 2017-02-15 北京大学深圳研究生院 Panoramic video region of interest description method and coding method
CN108156484A (en) * 2016-12-05 2018-06-12 奥多比公司 Virtual reality video flowing of the priority processing based on segment is distributed using adaptation rate
CN110506291A (en) * 2017-04-05 2019-11-26 联发科技股份有限公司 Handle the method and apparatus with the frame based on projection at least one perspective plane generated using Non-uniformed mapping
US11004173B2 (en) 2017-03-13 2021-05-11 Mediatek Inc. Method for processing projection-based frame that includes at least one projection face packed in 360-degree virtual reality projection layout
US11057643B2 (en) 2017-03-13 2021-07-06 Mediatek Inc. Method and apparatus for generating and encoding projection-based frame that includes at least one padding region and at least one projection face packed in 360-degree virtual reality projection layout

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6529634B1 (en) * 1999-11-08 2003-03-04 Qualcomm, Inc. Contrast sensitive variance based adaptive block size DCT image compression
CN1235411C (en) * 2003-10-17 2006-01-04 中国科学院计算技术研究所 Flow-line-based frame predictive mode coding acceleration method
JP4047879B2 (en) * 2005-08-23 2008-02-13 松下電器産業株式会社 Motion vector detection apparatus and motion vector detection method

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103780909A (en) * 2012-10-22 2014-05-07 古如罗技微系统公司 Encoder, decoder, coding method, and decoding method
CN106060513A (en) * 2016-06-29 2016-10-26 深圳市优象计算技术有限公司 Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing
CN106060513B (en) * 2016-06-29 2017-11-21 深圳市优象计算技术有限公司 A kind of code stream caching method for cylinder three-dimensional panoramic video netcast
CN106412582A (en) * 2016-10-21 2017-02-15 北京大学深圳研究生院 Panoramic video region of interest description method and coding method
CN106412582B (en) * 2016-10-21 2019-01-29 北京大学深圳研究生院 The description method of panoramic video area-of-interest and coding method
CN108156484A (en) * 2016-12-05 2018-06-12 奥多比公司 Virtual reality video flowing of the priority processing based on segment is distributed using adaptation rate
US11004173B2 (en) 2017-03-13 2021-05-11 Mediatek Inc. Method for processing projection-based frame that includes at least one projection face packed in 360-degree virtual reality projection layout
US11057643B2 (en) 2017-03-13 2021-07-06 Mediatek Inc. Method and apparatus for generating and encoding projection-based frame that includes at least one padding region and at least one projection face packed in 360-degree virtual reality projection layout
CN110506291A (en) * 2017-04-05 2019-11-26 联发科技股份有限公司 Handle the method and apparatus with the frame based on projection at least one perspective plane generated using Non-uniformed mapping
CN110506291B (en) * 2017-04-05 2021-05-14 联发科技股份有限公司 Video processing method and device

Also Published As

Publication number Publication date
CN100542298C (en) 2009-09-16

Similar Documents

Publication Publication Date Title
CN100542298C (en) A kind of block size computing method of cylinder panorama video code and transmission method
CN103237216B (en) The decoding method and coding and decoding device of depth image
CN101366279B (en) Adaptive motion search range
CN101552924B (en) Spatial prediction method for video coding
CN101263713B (en) Method and apparatus for motion compensation by irregular shapes
CN101273635A (en) Apparatus and method for encoding and decoding multi-view picture using camera parameter, and recording medium storing program for executing the method
CN102970526B (en) A kind of method obtaining transform block size and module
CN104902275B (en) A kind of method for controlling video communication quality dessert
CN105681805B (en) Video coding, coding/decoding method and its inter-frame prediction method and device
CN101584215A (en) Integrated spatial-temporal prediction
CN102630011A (en) Compressive perceptual coding and decoding method and system in video sensor network
CN106059640B (en) A kind of VLC secret signalling transmitting terminal design method based on QoS
CN106576170B (en) The method and system that adaptive search-window for Video coding positions
US20100290518A1 (en) Multi-view image coding apparatus and method
JP2005341580A (en) Method and apparatus for image interpolation system based on motion estimation and compensation
CN106657961A (en) Hybrid digital-analog video coding for stereoscopic video
US10554997B2 (en) Video coding/decoding method, encoder, and decoder
US20120113221A1 (en) Image processing apparatus and method
CN107105245B (en) High speed JPEG method for compressing image based on TMS320C6678 chip
CN108174200A (en) Image transfer method and device
CN104767997A (en) Video-oriented visual feature encoding method and device
CN101990763B (en) Generation of an order-2n transform from an order-n transform
CN104301724B (en) Method for processing video frequency, encoding device and decoding device
CN105122808A (en) Method and apparatus of disparity vector derivation for three-dimensional and multi-view video coding
CN104618714B (en) A kind of stereo video frame importance appraisal procedure

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
C14 Grant of patent or utility model