CN114422810A - Multipath live broadcast synchronous calibration method based on mobile terminal director station - Google Patents
Multipath live broadcast synchronous calibration method based on mobile terminal director station Download PDFInfo
- Publication number
- CN114422810A CN114422810A CN202111330933.3A CN202111330933A CN114422810A CN 114422810 A CN114422810 A CN 114422810A CN 202111330933 A CN202111330933 A CN 202111330933A CN 114422810 A CN114422810 A CN 114422810A
- Authority
- CN
- China
- Prior art keywords
- time
- live broadcast
- director
- machine position
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26241—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the time of distribution, e.g. the best time of the day for inserting an advertisement or airing a children program
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Abstract
The invention relates to a multi-channel live broadcast synchronous calibration method based on a mobile terminal broadcasting guide table, which comprises the steps that the mobile terminal broadcasting guide table creates a broadcasting guide example, creates a plurality of mobile terminal camera positions, generates machine position serial codes and forms two-dimensional codes; the director establishes a long connection with the long connection service; the live broadcast client adds the director through the two-dimensional code generated by scanning, establishes long connection between the machine position and a long connection server, and establishes a time mapping relation between the machine position and the director according to example information; and each machine position respectively sends the established mapping relation to a streaming media server, the live broadcast client performs stream pushing operation on the streaming media server, and the streaming media server performs decoding, timestamp calibration and synchronous operation after receiving a machine position push message and performs recoding. The method and the device furthest reduce the influence of the network environment on the multi-channel live broadcast synchronization problem, and control the multi-channel live broadcast stream synchronization problem at millisecond level so as to meet the live broadcast synchronization requirement under the environment with poor network conditions.
Description
Technical Field
The invention relates to the technical field of IPTV, in particular to a multi-channel live broadcast synchronous calibration method based on a mobile terminal broadcast director.
Background
With the rapid development of the network informatization process, short videos and live broadcasts are gradually merged into the daily life of people, and the production and life modes of people are changed silently. Meanwhile, the development of subdivision industries in various fields is promoted, such as: live tape, news numbers, etc. Video has become the wind gap for mobile internet. This puts higher demands on the live video processing technology.
On-line teaching, on-line meeting, on-line operation and the like are all the applications of the live broadcast form of multiple machine positions and multiple terminals in each scene, and the requirements on the real-time performance and the synchronism among all paths of video streams are very high. Especially for remote operation, real-time and synchronous operation is required to be controlled in millisecond level. Multiple cameras are bound to the problem of synchronization of multiple paths of live broadcast videos, the playing of the traditional multiple paths of live broadcast videos is not processed by a synchronization related mechanism, so that the synchronization of the multiple paths of live broadcast completely depends on a network environment, and due to the fact that the network may have various problems such as delay, jitter, congestion and packet loss, the abnormal terminal causes synchronization problems of different degrees. In this case, if the synchronization calibration process is not performed, the user experience is very poor, and a serious adverse effect may be caused.
Disclosure of Invention
In order to overcome the defects, the invention aims to provide a method for multi-channel live broadcast synchronous calibration based on a mobile terminal director, provides a multi-channel live broadcast synchronous calibration idea, and aims to solve the problem of multi-channel live broadcast asynchronism in various network environments.
The technical scheme adopted by the invention for solving the technical problems is as follows: a method for multi-channel live broadcast synchronous calibration based on a mobile terminal broadcast director comprises the following core design steps of establishing a time mapping relation between a live broadcast client and the broadcast director, synchronously calibrating a live broadcast stream frame timestamp according to the mapping relation, then synchronizing and converging according to a multi-channel live broadcast stream synchronization strategy, reducing the influence of a network environment on multi-channel live broadcast synchronization problems to the maximum extent, controlling the multi-channel live broadcast asynchronous problems within a certain range, and comprising the following specific steps:
firstly, a mobile terminal director platform creates a director example, completes registration on a streaming media server, creates a multi-station unique serial code which comprises example information, and generates a two-dimensional code;
secondly, the director and the long connection server establish long connection;
thirdly, scanning codes by the live broadcast client to join in the director, establishing long connection between the live broadcast client and the long connection server, and analyzing the two-dimension codes to obtain instance information;
fourthly, the live broadcast client acquires the current time stamp TSclt1Meanwhile, according to the example information, a request is sent to the broadcasting guide station through long connection, and a time stamp TS when the broadcasting guide station is obtaineddtr；
Fifthly, the live broadcast client side obtains the time TS of the directordtrThereafter, local time TS is acquired againclt2；
Sixthly, acquiring time TS twice according to the live broadcast clientclt1、TSclt2Calculating RTT (round trip time), and establishing a mapping relation between live broadcast client time and director time, wherein the mapping formula is as follows:
seventhly, sending a time mapping relation established between the live broadcast client and the director to a streaming media server, wherein the time mapping relation contains a unique identifier generated by the director obtained through analysis, and the streaming media server caches the mapping relation;
eighthly, the live broadcast client performs live broadcast stream pushing operation to the streaming media server, wherein the live broadcast client contains the only identification of the live broadcast client;
ninth, after receiving the push data of the live broadcast client, the streaming media service acquires the mapping relation between the time of the client and the time of the director from a cache according to the unique identifier of the client, decodes the data of the live broadcast frame, extracts the information of a time stamp TS (timestamp), calculates a new time stamp by taking the time of the director as a standard according to the acquired mapping relation, and rewrites the new time stamp;
and step ten, the streaming media server performs frame data synchronization, confluence and coding operations on the multi-channel messages according to the new timestamp, and pushes the multi-channel messages to a Content Delivery Network (CDN), so as to complete the multi-channel live streaming data synchronization operation.
The invention has the following beneficial effects:
the method comprises the steps of establishing a time mapping relation between a live broadcast client and a broadcast director, synchronously calibrating a live broadcast stream frame timestamp according to the mapping relation, and then synchronizing and converging according to a multi-path live broadcast stream synchronization strategy; the influence of the network environment on the multi-channel live broadcast synchronization problem is reduced to the maximum extent, and the multi-channel live broadcast stream synchronization problem is controlled to be millisecond level so as to meet the live broadcast synchronization requirement under the environment with poor network conditions.
Drawings
FIG. 1 is a timing diagram of the process of the present invention.
Fig. 2 is a schematic diagram of the multi-channel live stream synchronous confluence according to the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
A method based on multi-channel live broadcast synchronous calibration of a mobile terminal broadcast director is characterized in that research and development construction are carried out by taking the mobile terminal broadcast director as a basic platform, related functions of broadcast directing, live broadcast stream pushing and live broadcast watching are integrated into the same application, a plurality of mobile terminals install the application, and the specific implementation mode is as follows:
firstly, a mobile terminal director platform creates a director example, completes registration in a streaming media server, creates a plurality of mobile terminal camera positions, generates a two-dimensional code of each position serial code (unique identification containing director example information), and creates two live broadcast clients, namely a position A and a position B for description;
step two, the broadcasting guide station establishes long connection with the long connection service through a WSS (Web socket) protocol;
step three, the two machine positions generate two-dimensional codes through scanning in the step one and add the two-dimensional codes into the director, long connection between the machine positions and a long connection server is established, the two-dimensional codes are analyzed, and example information is obtained;
step four, establishing a time mapping relation between the machine position and the broadcasting station according to the example information, comprising the following steps:
4.1, the current time TS is recorded by the stationclt1Initiating a request to the director through the long connection according to the instance information to obtain the current time TS of the directordtrAfter the time of the director is obtained, the current time TS of the current position is obtained againclt2；
4.2 obtaining time TS according to two machine positionsclt1、TSclt2Calculating RTT (round trip time), and establishing a mapping relation between the machine bit time and the time of the program director, wherein the mapping formula is as follows:
the machine position A calculates RTT and establishes a mapping relation example with the time of the program director:
A. calculating RTT
The local time is respectively acquired before and after the acquisition of the director time: 1629967263000, i.e. TSclt1＝1629967263000,TSclt21629967263600, the director returns this instance for the time: 1629967266000, i.e. TSdtr1629967266000, then:
RTT＝(TSclt2-TSclt1)/2＝(1629967263600-1629967263000)/2＝300；
b. establishing a mapping relation with the time of the director
And b, according to the effective data obtained by the step a and the RTT calculation result, considering the RTT influence, and estimating the machine position time:
TSclt＝Tclt1+RTT＝1629967263000+300＝1629967263300
the established mapping relationship is:
step five, each machine position respectively sends the established mapping relation to a streaming media server, meanwhile, the string code generated in the step one is included, the streaming media server caches the mapping relation, the caching time is 5 hours, the live broadcast lasts for 5 hours at most, and the examples of the caching relation are as follows:
step six, the two machine positions respectively carry out stream pushing operation on the stream media server, the sender position only identifies, and the live stream data message is pushed;
step seven, after receiving the machine position push message, the streaming media server performs decoding, timestamp calibration and synchronization operations, and the operation method comprises the following steps:
7.1, obtaining the unique serial code of the machine position, obtaining the mapping relation between the time of the local position and the time of the broadcasting station in the data cache according to the unique serial code, and mapping the time stamp TS of the local positioncltMapping director time stamp TSdtr；
7.2, decoding the direct broadcasting stream frame data, extracting the time stamp TS (timestamp) information, and recalculating the brand new time stamp TS according to the mapping relation between the time of obtaining the local machine position and the time of the broadcasting stationnewThe calculation method is as follows:
TSnew＝TS+TSdtr-TSclt
new time stamp TS to be calculatednewRewriting the time stamp of the frame to realize time stamp calibration operation, wherein the frame data time stamp calibration example carries out the following operation based on the calculation result of the example in the above steps:
assume that the timestamp TS of one frame is extracted as: 1629967304000, the new timestamp of this frame is:
TSnew＝TS+TSdtr-TSclt
＝1629967304000+1629967266000+1629967263300
＝1629967306700
7.3, performing synchronous operation processing on multiple paths of live broadcast streams, as shown in fig. 2, wherein the live broadcast stream data is composed of uninterrupted frame data queues and is processed according to a first-in first-out principle, merging operation is performed on frame data with the same time stamp in each live broadcast data queue in the current director example, the same time stamp is not absolute, frame data in a time period of which the time stamp is lower than 1/2 frames are judged, the frame data of the time stamp are judged to be merged, assuming that the current live broadcast frame rate is 20 frames/second, the time period of each frame is 50ms, 1/2 frames is +/-25 ms, namely merging operation is performed on the frame data in the +/-25 ms of the multiple paths of live broadcast queues, and frame data synchronization logic is completed, namely, the time difference is controlled within a level of 50ms by the multiple paths of video synchronization operation;
step eight, after the synchronous calibration and the confluence of the multi-channel live streaming data are completed in the streaming media server, recoding is carried out, and the streaming is pushed to the CDN for acceleration and other operations;
step nine, completing the synchronous processing flow of the multi-path live broadcast streams, and enabling the user side to conduct synchronous viewing experience on the multi-path live broadcast video guided by the mobile terminal;
step ten, after the director/live broadcast is finished, sending an instruction to delete the time mapping relation between the local machine position and the director station.
The present invention is not limited to the above embodiments, and any structural changes made under the teaching of the present invention shall fall within the protection scope of the present invention, which is similar or similar to the technical solutions of the present invention.
The techniques, shapes, and configurations not described in detail in the present invention are all known techniques.
Claims (8)
1. A multi-channel live broadcast synchronous calibration method based on a mobile terminal broadcast director is characterized by comprising the following steps: comprises the steps of (a) preparing a mixture of a plurality of raw materials,
the mobile terminal director platform creates a director example, completes registration in a streaming media server, creates a plurality of mobile terminal camera positions, generates each camera position serial code and forms a two-dimensional code;
the director establishes a long connection with the long connection service;
the live broadcast client adds the two-dimensional code generated by scanning into a broadcast guide, establishes long connection between a machine position and a long connection server, analyzes the two-dimensional code and acquires instance information;
establishing a time mapping relation between the machine position and the broadcasting station according to the example information;
each machine position respectively sends the established mapping relation to a streaming media server, and the streaming media server caches the mapping relation, wherein the mapping relation comprises a machine position serial code;
the live broadcast client carries out stream pushing operation to the stream media server, sends the unique identifier of the machine position, and pushes the live broadcast stream data message;
after receiving the machine position push message, the streaming media server performs decoding, timestamp calibration and synchronization operations;
after the synchronous calibration and confluence of the multi-channel live streaming data are completed in the streaming media server, recoding is carried out, and the multi-channel live streaming data are pushed to a content distribution network to complete the synchronous operation of the multi-channel live streaming data.
2. The method according to claim 1, wherein the method comprises the following steps: the method for establishing the time mapping relationship between the machine position and the broadcasting station comprises the following steps:
live broadcast client side obtains current time stamp TSclt1Meanwhile, according to the example information, a request is sent to the broadcasting guide station through long connection, and a time stamp TS when the broadcasting guide station is obtaineddtr；
Live broadcast client side obtains director time TSdtrThereafter, local time TS is acquired againclt2；
Obtaining time TS twice according to live broadcast clientclt1、TSclt2Calculating the Round Trip Time (RTT), and establishing a mapping relation between the live client time and the director time, wherein the mapping formula is as follows:
wherein TScltFor local bit time, TSdtrIs the director's station time.
3. The method as claimed in claim 2, wherein the method comprises the step of performing multi-channel live broadcast synchronization calibration based on the mobile-end director stationThe method comprises the following steps: the method for calculating the return time RTT comprises the steps of respectively acquiring local time, namely TS before and after the acquisition of the time of the directorclt1、TSclt2And then:
RTT＝(TSclt2-TSclt1)/2。
4. the method according to claim 1, wherein the method comprises the following steps: the timestamp calibration mode is as follows: and extracting the time stamp, calculating a new time stamp according to the mapping relation between the time of the local machine position and the time of the broadcasting station, and rewriting.
5. The method according to claim 4, wherein the method comprises the following steps: new time stamp TSnewThe calculation method of (c) is as follows:
TSnew＝TS+TSdtr-TScltwherein TScltFor local bit time, TSdtrIs the director's station time.
6. The method according to claim 1, wherein the method comprises the following steps: the method for synchronous operation comprises the following steps: live streaming data is composed of uninterrupted frame data queues and is processed according to the first-in first-out principle, frame data with the same time stamp in each live data queue in the director example are subjected to confluence operation, the same time stamp is not absolute, the frame data with the time stamp being smaller than 1/2 frames in the long time are judged to be the frame data of the time stamp, and confluence is carried out.
7. The method according to claim 1, wherein the method comprises the following steps: and after the synchronous operation flow of the multi-channel live broadcast stream data is completed, the user side can perform synchronous viewing experience on the multi-channel live broadcast video guided by the mobile terminal.
8. The method according to claim 1, wherein the method comprises the following steps: and the method also comprises the step that when the direct broadcasting of the live broadcasting client end or the direct broadcasting is finished, the direct broadcasting station sends an instruction to delete the time mapping relation between the local machine position and the direct broadcasting station.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111330933.3A CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111330933.3A CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114422810A true CN114422810A (en) | 2022-04-29 |
| CN114422810B CN114422810B (en) | 2022-10-04 |
Family
ID=81265947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111330933.3A Active CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114422810B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116866624A (en) * | 2023-06-09 | 2023-10-10 | 北京医百科技有限公司 | Method and system for copying and sharing configuration information of guide table |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095130A1 (en) * | 2016-11-24 | 2018-05-31 | 广州华多网络科技有限公司 | Method and system for determining duration of live streaming |
| CN112351294A (en) * | 2020-10-27 | 2021-02-09 | 广州赞赏信息科技有限公司 | Method and system for frame synchronization among multiple machine positions of cloud director |
| CN112866733A (en) * | 2021-01-05 | 2021-05-28 | 广东中兴新支点技术有限公司 | Cloud director synchronization system and method of multiple live devices |
| CN113163222A (en) * | 2021-03-31 | 2021-07-23 | 杭州奥点科技股份有限公司 | Video frame synchronization method, system, equipment and readable storage medium |
| CN113453086A (en) * | 2021-03-29 | 2021-09-28 | 北京国际云转播科技有限公司 | Multi-machine-position synchronous cloud broadcasting guiding method, system, storage medium and video encoder |
| CN113542897A (en) * | 2021-05-19 | 2021-10-22 | 广州速启科技有限责任公司 | Audio and video live broadcast method suitable for multi-view live broadcast |
-
2021
- 2021-11-11 CN CN202111330933.3A patent/CN114422810B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095130A1 (en) * | 2016-11-24 | 2018-05-31 | 广州华多网络科技有限公司 | Method and system for determining duration of live streaming |
| CN112351294A (en) * | 2020-10-27 | 2021-02-09 | 广州赞赏信息科技有限公司 | Method and system for frame synchronization among multiple machine positions of cloud director |
| CN112866733A (en) * | 2021-01-05 | 2021-05-28 | 广东中兴新支点技术有限公司 | Cloud director synchronization system and method of multiple live devices |
| CN113453086A (en) * | 2021-03-29 | 2021-09-28 | 北京国际云转播科技有限公司 | Multi-machine-position synchronous cloud broadcasting guiding method, system, storage medium and video encoder |
| CN113163222A (en) * | 2021-03-31 | 2021-07-23 | 杭州奥点科技股份有限公司 | Video frame synchronization method, system, equipment and readable storage medium |
| CN113542897A (en) * | 2021-05-19 | 2021-10-22 | 广州速启科技有限责任公司 | Audio and video live broadcast method suitable for multi-view live broadcast |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116866624A (en) * | 2023-06-09 | 2023-10-10 | 北京医百科技有限公司 | Method and system for copying and sharing configuration information of guide table |
| CN116866624B (en) * | 2023-06-09 | 2024-03-26 | 北京医百科技有限公司 | Method and system for copying and sharing configuration information of guide table |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114422810B (en) | 2022-10-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6317872B2 (en) | Decoder for synchronizing the rendering of content received over different networks and method therefor | |
| CN100588250C (en) | Method and system for rebuilding free viewpoint of multi-view video streaming | |
| US11758209B2 (en) | Video distribution synchronization | |
| CN101917389B (en) | Network television direct broadcasting system | |
| CN100429901C (en) | Method and structure for realizing live channel switching in Internet protocol audio-video broadcast network | |
| CN100499805C (en) | Free viewpoint video IP network transmission method based on light field exaggeration | |
| CN109361945A (en) | The meeting audiovisual system and its control method of a kind of quick transmission and synchronization | |
| CN110602522B (en) | Multi-path real-time live webRTC stream synthesis method | |
| US20180176278A1 (en) | Detecting and signaling new initialization segments during manifest-file-free media streaming | |
| CN112291498B (en) | Audio and video data transmission method and device and storage medium | |
| CN108494792A (en) | A kind of flash player plays the converting system and its working method of hls video flowings | |
| US20230319371A1 (en) | Distribution of Multiple Signals of Video Content Independently over a Network | |
| CN112866725A (en) | Live broadcast control method and device | |
| CN114422810B (en) | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director | |
| CN113194278A (en) | Conference control method and device and computer readable storage medium | |
| CN110460898B (en) | Video processing method, system, device and machine readable medium | |
| CN104469259A (en) | Cloud terminal video synthesis method and system | |
| US20230328308A1 (en) | Synchronization of multiple content streams | |
| CN114885198A (en) | Mixed network-oriented accompanying sound and video collaborative presentation system | |
| Westerink et al. | A live intranet distance learning system using MPEG-4 over RTP/RTSP | |
| KR101999235B1 (en) | Method and system for providing hybrid broadcast broadband service based on mmtp | |
| Klinovskii et al. | Analysis of Methods for Reducing Latency in Internet TV | |
| CN115022665A (en) | Live broadcast making method and device, multimedia processing equipment and multimedia processing system | |
| Zhao | Research on Computer network multimedia transmission system | |
| CN116708860A (en) | Live broadcast system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |