CN116193133A - Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth - Google Patents
Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth Download PDFInfo
- Publication number
- CN116193133A CN116193133A CN202111434723.9A CN202111434723A CN116193133A CN 116193133 A CN116193133 A CN 116193133A CN 202111434723 A CN202111434723 A CN 202111434723A CN 116193133 A CN116193133 A CN 116193133A
- Authority
- CN
- China
- Prior art keywords
- video
- code rate
- module
- air interface
- dash
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
A code rate self-adaptive adjusting device and method based on wireless air interface bandwidth includes: the invention utilizes the obtained air interface bandwidth to calculate the network rate and combines the buffer information to obtain the optimal transmission code rate; and selecting and outputting the transmission code rate of the video clips to be requested from the selectable code rate list according to the optimal transmission code rate, so that the transmission code rate is selected by taking the throughput and the video buffer information into consideration.
Description
Technical Field
The invention relates to a technology in the field of wireless image transmission, in particular to a code rate self-adaptive adjustment device and method based on wireless air interface bandwidth.
Background
The current internet video streaming and downloading is mainly realized by a code rate self-Adaptive (ABR) algorithm, the self-adaptive code rate is generally based on a dynamic self-adaptive streaming (DynamicAdaptive Streaming over HTTP, DASH) protocol, and high-quality video and audio contents can be transmitted to a networking television, a set-top box and a mobile terminal device through a network by using a common video and audio file communication protocol of HTTP. In the video playing process, a client based on a dynamic adaptive streaming over HTTP (DynamicAdaptive Streaming over HTTP, DASH) protocol periodically evaluates the available bandwidth of the current network and the performance of the current network according to a code rate adaptive algorithm, and selects an optimal transmission code rate to download video clips. The existing code rate self-adaptive method is mainly based on client information and can be divided into two main categories: the method is based on network throughput of the client and based on the length of the video playing buffer zone of the client. The code rate adaptive algorithm based on the network throughput of the client is mainly to estimate the future throughput to adjust the code rate. The disadvantage of this algorithm is that the rate jitter occurs in case of abrupt changes in the network bandwidth and that video stuck easily occurs without considering the buffer. The code rate self-adaptive algorithm based on the length of the video playing buffer zone of the client is to divide the buffer length into a plurality of buffer grades, and different code rate adjustment schemes are adopted at different buffer grades. The algorithm has the defects that an optimal solution for code rate level division is difficult to find, frequent jitter of the code rate is easy to be caused by excessive division levels, and network change cannot be perceived in time due to too few division levels.
Disclosure of Invention
Aiming at the problems that in the prior art, the code rate is difficult to adaptively process when the network condition is suddenly changed, and the code rate is high when the air interface load is large, the video playing buffer area is exhausted when the video buffer area is small, and the like, the invention provides a code rate adaptive adjustment device and a code rate adaptive adjustment method based on wireless air interface bandwidth, which calculate the network rate by utilizing the obtained air interface bandwidth and acquire the optimal transmission code rate by combining buffer information; and selecting and outputting the transmission code rate of the video clips to be requested from the selectable code rate list according to the optimal transmission code rate, so that the transmission code rate is selected by taking the throughput and the video buffer information into consideration.
The invention is realized by the following technical scheme:
the invention relates to a code rate self-adaptive adjusting device based on wireless air interface bandwidth, which comprises: a DASH storage module, a data acquisition module, and a bandwidth prediction module located at an edge computing (MEC) server, and a data receiving module, a code rate adaptive module, and a DASH play module located at a DASH client, wherein: the DASH storage module stores video clips with different code rates and resolutions and responds to a DASH client video clip request instruction from the base station; the data acquisition module acquires user data and base station configuration information from the base station; the bandwidth prediction module predicts the future available air interface bandwidth by utilizing a long-short-period memory (LSTM) algorithm according to received Modulation and Coding Strategy (MCS), reference Signal Receiving Quality (RSRQ), reference Signal Receiving Power (RSRP) and physical resource block (DLOccupiePRBNUM) data used in a downlink, tracks and perceives the change information of the bandwidth in advance and sends the change information to a user client to help to carry out code rate self-adaption; the data receiving module and the bandwidth predicting module establish WebSocket communication to receive the predicted available air interface bandwidth and output the available air interface bandwidth information to the code rate self-adaptive module; the code rate self-adaptive module decides and selects the transmission code rate of the video clip to be requested from the selectable code rate list according to the available air interface bandwidth information and the video cache information in the DASH playing module, and then outputs the transmission code rate of the video clip to be requested to the DASH playing module; the DASH playing module acquires the video index file and the video segment data in the DASH storage module, sends a video segment request with a corresponding code rate to the DASH storage module according to the transmission code rate requested by the code rate self-adaptive module, stores the received video segment in a video playing buffer area, displays and plays the video segment, and sends video cache information to the code rate self-adaptive module.
The user data is self configuration information describing the individual users currently connected to the base station and status information of the connection between each user and the base station.
The self configuration information includes parameter configurations of an RRC (Radio Resource Control ) layer, a MAC (Media Access Control, media access control) layer and a PDCP (Packet Data Convergence Protocol ) layer.
The base station configuration information includes: the transmission power of the signal, the code modulation mode, etc.
The tracking and early sensing technology specifically includes capturing data related to the air interface bandwidth, such as Modulation and Coding Strategy (MCS), reference Signal Receiving Quality (RSRQ), reference Signal Receiving Power (RSRP), and physical resource block (DLOccupiePRBNum) used in the downlink, reported by the user in real time through the base station IP interface, and predicting future air interface bandwidth change condition by using long-short-term memory (LSTM) algorithm.
The structure of the selectable code rate list is MPD (Media Presentation Description, media description file), the MPD is an XML (Extensible Markup Language ) file, and the MPD completely represents all information of the video, including video length, code rate and resolution of different video clips and clip duration, and is specifically constructed and maintained by a DASH storage module.
The structure of the video playing buffer zone is a temporary storage space, and the downloaded video clips are firstly stored in the video playing buffer zone and then played one by one according to the sequence. When the time for downloading the video clips is longer than the video playing time, the length of the video playing buffer zone is reduced, and when the length of the video playing buffer zone is reduced to 0, the video is blocked, and a new video clip must be downloaded and stored in the buffer zone to continue playing the video. And the DASH playing module is specifically used for constructing and maintaining.
The invention relates to a wireless air interface bandwidth code rate self-adaptive adjustment method based on the device, which comprises the steps of obtaining an available air interface bandwidth predicted value through a data receiving module, calculating the available air interface bandwidth to obtain the future network rate, obtaining the length of a cache area of a current video player, selecting the corresponding video code rate as a request code rate of a next video block, and sending the request code rate to a DASH playing module.
The selection of the corresponding video code rate means that: when the video buffer length is greater than the preset value upper limit B H When the video code rate R is the largest, the maximum video code rate R is selected max . At this time, the buffer area has enough redundancy to bear the maximum code rate of downloadingThe risk of jamming can be brought; when the video buffer length is smaller than the preset value lower limit B L When the video code rate R is the smallest min The method comprises the steps of carrying out a first treatment on the surface of the When the video buffer length is at the preset value lower limit B L And a preset upper limit B H And when the video code rate is determined by adopting the available air interface bandwidth based on prediction.
The preset upper limit B H According to the length of each video block being l=4s, the maximum length of the buffer is L max =20s, calculate to obtain the preset value B H ＝L max -l; the lower limit B of the preset value L ＝l*1.5＝6s。
The available air interface bandwidth based on prediction refers to: network rate C k Multiplying the video index file by a compromise factor of 0.9 to prevent the selection of video code rate which cannot be tolerated by network bandwidth, and then selecting a network rate R which is not lower than the compromise in the video index file k+1 ≤C k *0.9。
Technical effects
The invention predicts the user air interface bandwidth by acquiring the information of the base station and transmits the information to the user client for code rate decision, and the prediction not only has the historical bandwidth information of the current user but also contains the resource scheduling information of multiple users, thereby improving the accuracy of the code rate decision of the user client. The method further sets a playing buffer zone length threshold, selects and downloads the video with high code rate in the scene with larger buffer zone length, and selects the video with low code rate in the scene with smaller buffer zone length; the rest scenes adopt the user air interface bandwidth data to make decision selection, so that the probability of video blocking is reduced while the high code rate is ensured.
Drawings
FIG. 1 is a schematic diagram of a system according to the present invention;
FIG. 2 is a flow chart of an embodiment;
fig. 3 implements a schematic diagram of a scenario.
Detailed Description
As shown in fig. 1, this embodiment relates to a code rate adaptive adjustment device based on wireless air interface bandwidth, including: the system comprises a DASH storage module, a data acquisition module and a bandwidth prediction module which are positioned on an edge computing (MEC) server, and a data receiving module, a code rate self-adaption module and a DASH playing module which are positioned on a DASH client.
The DASH storage module includes: a video storage unit and a video request response unit, wherein: the video storage unit constructs a video MPD file and stores video clips with different code rates and resolutions; the video request response unit responds to the media index file analysis sent by the client-side DASH playing module, and then selects the corresponding video fragment for transmission.
The data acquisition module comprises: the device comprises a data grabbing unit and a data storage unit, wherein: the data grabbing unit acquires user data and base station configuration information through a corresponding IP interface of the base station, wherein the user data and the base station configuration information comprise received Modulation and Coding Strategy (MCS), reference Signal Receiving Quality (RSRQ), reference Signal Receiving Power (RSRP) and physical resource block (DLOccupiePRBNUM) data used in a downlink; the data storage unit stores the data obtained by the data grabbing unit.
The bandwidth prediction module comprises: a data processing unit and a bandwidth prediction unit, wherein: the data processing unit obtains the data of the data storage unit in the data acquisition module and sequentially carries out unit conversion and normalization calculation processing on the data; the bandwidth prediction unit loads the LSTM prediction model, takes the processed data of the data processing unit as the input of the prediction model, and predicts the air interface bandwidth at the next moment.
The data receiving module comprises: a data communication unit and a data forwarding unit, wherein: the data communication unit establishes WebSocket communication with the air interface bandwidth prediction module to obtain a predicted air interface bandwidth; the data forwarding unit forwards the obtained data to the code rate self-adaptive module.
The code rate self-adaptive module comprises: video information unit and code rate adjustment unit, wherein: the video information unit obtains the air interface bandwidth data in the data receiving module and the video buffer area length information in the DASH playing module, and converts the air interface bandwidth into network rate; the code rate adjusting unit gives consideration to the network rate and the length information of the video buffer area given by the data processing unit, and decides the transmission code rate of the video clip to be requested from the selectable code rate list.
The DASH playing module comprises: video playback unit and video request unit, wherein: the video playing unit takes out the video clips from the video playing buffer zone to play and detects the length information of the current video buffer zone to output to the code rate self-adaptive module. The video request unit obtains the request code rate of the code rate self-adaptive module, requests the corresponding video clips from the MEC DASH storage module, and stores the downloaded video clips into a playing buffer zone of the video playing unit.
Through specific practical experiments, a test environment is set to be that three UEs are simultaneously connected to a base station as shown in fig. 3, wherein low-rate uniformity (Group a) of an experiment scene is that background UE1 and background UE2 simultaneously and circularly play 500k videos, high-rate non-uniformity (Group B) of the experiment scene is that background UE1 circularly plays 5M videos, background UE2 circularly plays 1M videos, and the performance of the videos is evaluated by QoE functions, so that the results shown in table 1 are obtained.
The QoE function:
wherein: k is the number of downloaded video blocks, R k Coding bit rate (code rate) for video block k, q (R k ) Rate R of video code rate k Mapping to user's perception of video quality, T k For downloading code rate R k Buffering time (click time), |q (R) k+1 )-q(R k ) And the I is the change (smoothness) of video quality, and the weighting coefficients of mu, tau video clip and smoothness are respectively 15 and 1.
TABLE 1 code rate adjustment Performance
As shown in table 1, the average code rate/QoE performance of the device in the Group a and Group B scenes is 2.1/1.6 times and 1.8/2.5 times of the Baseline algorithm, respectively, so that the invention can fully utilize the wireless air interface transmission capability to improve the code rate of the user for watching the video, and simultaneously ensure smaller cartoon time and video jitter to improve the quality of the user for watching the video.
Compared with the prior art, the invention utilizes MEC to acquire the base station user data and the base station configuration information in real time at the edge of the wireless access network, utilizes long-short-term memory (LSTM) algorithm to predict the air interface bandwidth of the user, and the acquired base station information contains the channel quality information of the current wireless link and the resource scheduling information among the users, so that the change of the air interface bandwidth of the user can be accurately perceived. And secondly, the available air interface bandwidth information and the video playing buffer area length information of the user are considered in the code rate self-adaptive algorithm design, so that the defect that video is stuck or unclear due to the fact that the video code rate is improved by a single optimized variable is avoided.
The foregoing embodiments may be partially modified in numerous ways by those skilled in the art without departing from the principles and spirit of the invention, the scope of which is defined in the claims and not by the foregoing embodiments, and all such implementations are within the scope of the invention.
Claims (10)
1. The utility model provides a code rate self-adaptation adjusting device based on wireless air interface bandwidth which characterized in that includes: the system comprises a DASH storage module, a data acquisition module, a bandwidth prediction module, a data receiving module, a code rate self-adaptive module and a DASH playing module, wherein the DASH storage module, the data acquisition module and the bandwidth prediction module are positioned at an edge computing server, and the data receiving module, the code rate self-adaptive module and the DASH playing module are positioned at a DASH client side, wherein: the DASH storage module stores video clips with different code rates and resolutions and responds to a DASH client video clip request instruction from the base station; the data acquisition module acquires user data and base station configuration information from the base station; the bandwidth prediction module predicts the future available air interface bandwidth by utilizing a long-period memory algorithm according to the received modulation and coding strategy, reference signal receiving quality, reference signal receiving power and physical resource block data used in a downlink, tracks and perceives the change information of the bandwidth in advance and sends the change information to a user to help to carry out code rate self-adaption; the data receiving module and the bandwidth predicting module establish WebSocket communication to receive the predicted available air interface bandwidth and output the available air interface bandwidth information to the code rate self-adaptive module; the code rate self-adaptive module decides and selects the transmission code rate of the video clip to be requested from the selectable code rate list according to the available air interface bandwidth information and the video cache information in the DASH playing module, and then outputs the transmission code rate of the video clip to be requested to the DASH playing module; the DASH playing module acquires the video index file and the video segmentation data in the DASH storage module, sends a video segment request with a corresponding code rate to the DASH storage module according to the transmission code rate requested by the code rate self-adaptive module, stores the received video segment in a video playing buffer area, displays and plays the video segment, and sends video cache information to the code rate self-adaptive module;
the user data is self configuration information describing each user currently connected to the base station and status information of connection between each user and the base station; the configuration information comprises parameter configuration of an RRC layer, a MAC layer and a PDCP layer;
the base station configuration information includes: the transmitting power of the signal and the code modulation mode.
2. The adaptive code rate adjusting device based on wireless air interface bandwidth according to claim 1, wherein the tracking and sensing in advance specifically comprises the technical means of capturing data related to the air interface bandwidth reported by a user in real time through a base station IP interface and predicting future air interface bandwidth change conditions by using a long-short-term memory algorithm;
the data related to the air interface bandwidth comprises: modulation and coding strategy, reference signal reception quality, reference signal reception power, physical resource blocks used in the downlink.
3. The adaptive adjustment device of code rate based on wireless air interface bandwidth according to claim 1, wherein the DASH storage module comprises: a video storage unit and a video request response unit, wherein: the video storage unit constructs a video MPD file and stores video clips with different code rates and resolutions; the video request response unit responds to the media index file analysis sent by the client-side DASH playing module, and then selects the corresponding video fragment for transmission.
4. The adaptive adjustment device for code rate based on wireless air interface bandwidth according to claim 1, wherein the bandwidth prediction module comprises: a data processing unit and a bandwidth prediction unit, wherein: the data processing unit obtains the data of the data storage unit in the data acquisition module and sequentially carries out unit conversion and normalization calculation processing on the data; the bandwidth prediction unit loads the LSTM prediction model, takes the processed data of the data processing unit as the input of the prediction model, and predicts the air interface bandwidth at the next moment.
5. The device for adaptive adjustment of code rate based on wireless air interface bandwidth according to claim 1, wherein the code rate adaptive module comprises: video information unit and code rate adjustment unit, wherein: the video information unit obtains the air interface bandwidth data in the data receiving module and the video buffer area length information in the DASH playing module, and converts the air interface bandwidth into network rate; the code rate adjusting unit gives consideration to the network rate and the length information of the video buffer area given by the data processing unit, and decides the transmission code rate of the video clip to be requested from the selectable code rate list.
6. The adaptive rate adjustment device based on wireless air interface bandwidth according to claim 1, wherein the video playing buffer area has a structure of a temporary storage space, the downloaded video clips are firstly stored in the video playing buffer area, then the video clips are played one by one according to the sequence, the length of the video playing buffer area is reduced when the time of downloading the video clips is longer than the video playing time, and when the length of the video playing buffer area is reduced to 0, the video is blocked, and then a new video clip must be downloaded and stored in the buffer area to continue playing the video, and the DASH playing module is specifically used for construction and maintenance.
7. The method for adaptively adjusting the code rate of the wireless air interface bandwidth based on the device for adaptively adjusting the code rate according to any one of claims 1 to 6 is characterized in that a data receiving module obtains an available air interface bandwidth predicted value and calculates an available air interface bandwidth to calculate a future network rate, obtains the length of a cache area of a current video player and selects a corresponding video code rate as a request code rate of a next video block to be sent to a DASH playing module.
8. The adaptive adjustment method of code rate according to claim 7, wherein said selecting a corresponding video code rate means: when the video buffer length is greater than the preset value upper limit B H When the video code rate R is the largest, the maximum video code rate R is selected max At this time, the buffer area has enough redundancy to bear the blocking risk possibly brought by the maximum code rate of the download; when the video buffer length is smaller than the preset value lower limit B L When the video code rate R is the smallest min The method comprises the steps of carrying out a first treatment on the surface of the When the video buffer length is at the preset value lower limit B L And a preset upper limit B H And when the video code rate is determined by adopting the available air interface bandwidth based on prediction.
9. The adaptive adjustment method of code rate according to claim 7, wherein said preset value upper limit B H According to the length of each video block being l=4s, the maximum length of the buffer is L max =20s, calculate to obtain the preset value B H ＝L max -l; the lower limit B of the preset value L ＝l*1.5＝6s。
10. The adaptive adjustment method of code rate according to claim 7, wherein the available air interface bandwidth based on prediction is: network rate C k Multiplying the video index file by a compromise factor of 0.9 to prevent the selection of video code rate which cannot be tolerated by network bandwidth, and then selecting a network rate R which is not lower than the compromise in the video index file k+1 ≤C k *0.9。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111434723.9A CN116193133A (en) | 2021-11-29 | 2021-11-29 | Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111434723.9A CN116193133A (en) | 2021-11-29 | 2021-11-29 | Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116193133A true CN116193133A (en) | 2023-05-30 |
Family
ID=86451078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111434723.9A Pending CN116193133A (en) | 2021-11-29 | 2021-11-29 | Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116193133A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103235A (en) * | 2022-06-21 | 2022-09-23 | 咪咕互动娱乐有限公司 | Data transmission method, device, equipment and medium |
-
2021
- 2021-11-29 CN CN202111434723.9A patent/CN116193133A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103235A (en) * | 2022-06-21 | 2022-09-23 | 咪咕互动娱乐有限公司 | Data transmission method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2717536B1 (en) | Processing method, distribution server, client and system for streaming media | |
| El Essaili et al. | QoE-based traffic and resource management for adaptive HTTP video delivery in LTE | |
| AU2013296976B2 (en) | Methods for quality-aware adaptive streaming over hypertext transfer protocol | |
| EP3238402B1 (en) | Method and client apparatus for dynamic rate adaptation of a stream of multimedia contents in a wireless communication network | |
| US10721715B2 (en) | Link-aware streaming adaptation | |
| CN104040992B (en) | There is the Media Stream of improved efficiency in mobile network | |
| US7782779B2 (en) | Apparatus and method for transmitting a multimedia data stream | |
| Han et al. | AMVS-NDN: Adaptive mobile video streaming and sharing in wireless named data networking | |
| KR101904053B1 (en) | Apparatus and method for processing a multimedia data in terminal equipment | |
| EP2904757B1 (en) | Throttling a media stream for transmission via a radio access network | |
| KR101982290B1 (en) | Streaming system and method based on contents characteristic for improving perceived quality of adaptive streaming service | |
| EP3563540B1 (en) | Method and system for providing variable quality streaming video services in mobile communication networks | |
| US20120266198A1 (en) | Fast Binding of a Cloud Based Streaming Server Structure | |
| KR102185876B1 (en) | Apparatus and method for http adaptive streaming in wireless network environment | |
| US9131251B2 (en) | Use of a receive-window size advertised by a client to a content server to change a video stream bitrate streamed by the content server | |
| US20220400295A1 (en) | Method And System For Pre-Positioning And Post Positioning Content In A Content Distribution System | |
| Jabbar et al. | Developing a video buffer framework for video streaming in cellular networks | |
| CN116193133A (en) | Code rate self-adaptive adjusting device and method based on wireless air interface bandwidth | |
| KR101966588B1 (en) | Method and apparatus for receiving video contents | |
| KR101827427B1 (en) | Video signal transmitting apparatus and method thereof | |
| Rodrigues et al. | QoE-aware scheduling algorithm for adaptive HTTP video delivery in wireless networks | |
| JP2022091710A (en) | Multiple-protocol prediction and in-session adaptation in video streaming | |
| KR101428980B1 (en) | Method and server for providing contents | |
| Li et al. | Pipeline-based chunk scheduling to improve abr performance in DASH system | |
| CN115604507A (en) | Video transmission device based on multi-site cooperation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |