CN113207016B - Virtual machine image frame rate control method, network equipment and storage medium - Google Patents

Virtual machine image frame rate control method, network equipment and storage medium Download PDF

Info

Publication number
CN113207016B
CN113207016B CN202110336922.XA CN202110336922A CN113207016B CN 113207016 B CN113207016 B CN 113207016B CN 202110336922 A CN202110336922 A CN 202110336922A CN 113207016 B CN113207016 B CN 113207016B
Authority
CN
China
Prior art keywords
image
virtual machine
frame rate
client
fps
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110336922.XA
Other languages
Chinese (zh)
Other versions
CN113207016A (en
Inventor
方杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
New H3C Big Data Technologies Co Ltd
Original Assignee
New H3C Big Data Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by New H3C Big Data Technologies Co Ltd filed Critical New H3C Big Data Technologies Co Ltd
Priority to CN202110336922.XA priority Critical patent/CN113207016B/en
Publication of CN113207016A publication Critical patent/CN113207016A/en
Application granted granted Critical
Publication of CN113207016B publication Critical patent/CN113207016B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • G06F9/452Remote windowing, e.g. X-Window System, desktop virtualisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44008Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet

Abstract

The application provides a virtual machine image frame rate control method, network equipment and a storage medium, wherein the virtual machine image frame rate control method comprises the following steps: saving a virtual machine configuration file; receiving a display card instruction of the virtual machine and recording the instruction time of the display card; analyzing the video card instruction to obtain a virtual machine image and a surface of the image; judging whether the frame rate of the images sent by the virtual machine is higher than a set maximum frame rate max _ fps; if the frame rate is in the set range, sending the image to the client in a compression coding mode; if the frame rate is higher than the set range, a timer is started, and the images are transmitted to the client in the full-screen streaming mode of the images at the time interval of 1/max _ fps. According to the embodiment of the application, the highest frame rate is set, and different image transmission modes are adopted for the images at different frame rates. The high frame rate image is directly transmitted through a full screen stream mode, so that the problems of decoding blockage and delay blockage of the high frame rate image transmitted to the client are solved.

Description

Virtual machine image frame rate control method, network device and storage medium
Technical Field
The present disclosure relates to the field of communications technologies, and in particular, to a method, a network device, and a storage medium for controlling an image frame rate of a virtual machine.
Background
The virtual machine (cloud desktop) is based on a virtual desktop infrastructure, and can host the desktop environment of a user to a high-performance server in the form of a cloud desktop, so that the user experience almost the same as that of a physical computer is provided for the user. The method is widely applied to the solution of the cloud desktop such as daily office, school teaching and the like.
When the image on the virtual machine (cloud desktop) is rapidly updated, if the image is sent to the client in a general compression coding mode, a large amount of bandwidth is occupied, and the problem of terminal decoding congestion is caused.
Disclosure of Invention
In order to overcome the problems in the related art, the present specification provides a virtual machine image frame rate control method, a network device, and a storage medium.
According to a first aspect of an embodiment of the present specification, there is provided a virtual machine image frame rate control method,
saving a configuration file of the virtual machine, wherein the configuration file at least comprises the highest frame rate max _ fps of image transmission;
receiving a display card instruction of the virtual machine, and recording the instruction time of the display card;
analyzing the video card instruction to obtain an image of the virtual machine and a surface of the image;
judging whether the frame rate of the received images sent by the virtual machine is higher than a set maximum frame rate max _ fps;
if the frame rate is in the set range, sending the image to the client in a compression coding mode;
if the frame rate is higher than the set range, a timer is started, and the images are sent to the client in the full-screen streaming mode of the images at the time interval of 1/max _ fps.
Preferably, the configuration file further comprises a timeout time timeout for the timer to time the acquisition of the image,
if the time interval exceeds 1/max _ fps and no video card instruction sent by the virtual machine is received, starting a timer, and judging the difference between the current recording time of the timer and the time when the last video card instruction is received;
if the difference value is smaller than timeout, acquiring a full-screen streaming mode image of the image and sending the full-screen streaming mode image to the client;
if the difference is larger than or equal to timeout, the image is not updated, the timer is closed, and the image is sent to the client in a compression coding mode.
Preferably, under the condition that the difference value is less than timeout, the full-screen streaming mode image is transmitted to the client, and the timer is reset, and the time interval is 1/max _ fps.
Further, the step of judging whether the frame rate of the received image sent by the virtual machine is higher than the set maximum frame rate max _ fps comprises the following steps:
finding the corresponding image data on the shared storage of the virtual machine through the image storage address in the surface;
and judging whether the frame rate of the currently found image is higher than the set highest frame rate max _ fps.
Preferably, the method comprises the steps of receiving a display card instruction of the virtual machine, establishing protocol connection with the virtual machine before recording the display card instruction time, and acquiring identification information of the virtual machine.
Further, the identification information includes at least ip and port information of the virtual machine.
The embodiment of the present application further provides a network device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the program to perform the above-mentioned method for controlling the image frame rate of the virtual machine.
Finally, the present embodiment also provides a storage medium, on which computer program instructions are stored, and the program instructions are executed by a processor to implement the above-mentioned method for controlling the image frame rate of the virtual machine.
The technical scheme provided by the embodiment of the specification can have the following beneficial effects:
in the embodiment of the present description, by setting the highest frame rate, the image adopts different image transmission modes at different frame rates. The high frame rate image is directly transmitted through a full screen stream mode, so that the problems of decoding blockage and delay blockage when the high frame rate image is transmitted to a client are solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present specification and together with the description, serve to explain the principles of the specification.
FIG. 1 is a schematic diagram of networking of an embodiment of the present application;
FIG. 2 is a flowchart of a method for controlling a frame rate of a virtual machine according to the present application;
fig. 3 is a flowchart of sending an image when a virtual machine is not timed according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this description. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.
In order to solve the problems in the background art, the application provides a virtual machine image frame rate control method, and a client acquires image information on a virtual machine through a host. In the networking, a client needs to establish link connection with a host machine at first, the host machine needs to establish protocol connection with a virtual machine, and the host machine acquires identification information of the virtual machine. The host typically has multiple virtual machines (virtual machine 1, virtual machine 2, and virtual machine 3) hanging down, and the host is connected with multiple clients (client 1, client 2, and client 3), as in fig. 1. The embodiment of the present application takes an example where the client 1 requests to acquire an image on the virtual machine 1 through the host as an example. In addition, the protocol established between the host and the virtual machine in the above embodiment may be, for example, an existing RDP or spice protocol, or may be a custom protocol. Since it is not the point of the invention of the present application, it will not be described herein in detail. The host machine and each virtual machine client only have one protocol connection process, and the host machine and each virtual machine client are connected to the server side of the host machine through the process and used for image transmission. After the host machine and the virtual machine are connected through the protocol, the host machine obtains the identification information of the virtual machine, the identification information at least comprises ip and port information of the virtual machine, and one-to-one connection between a server side of the host machine and the progress of the virtual machine and image information interaction are achieved. The host machine stores configuration files of each virtual machine. The configuration file is mainly the highest frame rate max _ fps for sending the image, and the timeout time timeout for timer timing to acquire the image. And analyzing the display card instruction to obtain the image, the image storage address information and the like of the corresponding virtual machine.
As shown in fig. 2, the specific interaction process is as follows:
the client 1 requests the host machine for the image of the virtual machine 1, so that the host machine sends an image request to the virtual machine 1 after receiving the request;
the host machine receives a display card instruction of the virtual machine 1 and records the arrival time of the display card instruction;
the host machine analyzes the display card instruction to obtain the image of the corresponding virtual machine and the surface of the image; judging whether the frame rate of the received image transmitted from the virtual machine 1 is higher than the set highest frame rate max _ fps; if the frame rate is in the set range, the image is sent to the client 1 in a compression encoding mode; if the frame rate is higher than the set range, a timer is started, and the image is transmitted to the client 1 in the full-screen streaming mode of the image at a time interval of 1/max _ fps. It should be noted here that the full screen streaming mode refers to transmitting an image in the form of a full screen video. And the image transmitted by the general compression coding method is transmitted in the form of a common picture or a photo.
The surface in the application refers to a storage address of a host server for storing a current display sending image of a virtual machine. The storage address refers to the storage address of the image stored on the shared storage of all the down-hanging virtual machines of the host.
Because the virtual machine 1 may not send a display card instruction to the host at the set time interval of 1/max _ fps, starting a timer of a host server process, judging a difference value between the current recording time of the timer and the time when the last previous instruction is received, and if the difference value is less than timeout, acquiring a full-screen flow mode image of the image and sending the full-screen flow mode image to the client 1; if the difference is greater than or equal to timeout, the image is not updated, the timer is closed, and the image is sent to the client 1 in a compression coding manner, as shown in fig. 3.
In the above embodiment, the method for determining whether the frame rate of receiving the image transmitted by the virtual machine 1 is higher than the set maximum frame rate max _ fps includes:
finding image data corresponding to the shared storage of all the virtual machines through the image storage address in the surface;
and judging whether the frame rate of the currently found image is higher than the set highest frame rate max _ fps.
According to the embodiment of the application, the highest frame rate is set, and different image transmission modes are adopted for the images at different frame rates. The high frame rate image is directly transmitted through a full screen stream mode, so that the problems of decoding blockage and delay blockage when the high frame rate image is transmitted to a client are solved.
The embodiment of the present application further provides a network device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the processor executes the method for controlling the image frame rate of the virtual machine according to the foregoing embodiment.
The embodiment of the present application further provides a storage medium, on which computer program instructions are stored, and the program instructions are executed by a processor to implement the method for controlling the image frame rate of the virtual machine according to the above embodiment.
It will be understood that the present description is not limited to the precise arrangements described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims (8)

1. A virtual machine image frame rate control method is characterized in that,
saving a configuration file of a virtual machine, wherein the configuration file at least comprises the highest frame rate max _ fps of image transmission;
receiving a display card instruction of a virtual machine, and recording the instruction time of the display card; the video card instruction time is the time when the video card instruction arrives;
analyzing the graphics card instruction to obtain an image of the virtual machine and a surface of the image; the surface is a storage address of a stored virtual machine for currently displaying and sending an image, and the storage address refers to a storage address of the image stored in shared storage of all the under-hung virtual machines;
judging whether the frame rate of the received images sent by the virtual machine is higher than a set maximum frame rate max _ fps;
if the frame rate is in the set range, sending the image to the client in a compression coding mode;
if the frame rate is higher than the set range, a timer is started, and the images are sent to the client in the full-screen streaming mode of the images at the time interval of 1/max _ fps.
2. The method for controlling image frame rate of a virtual machine according to claim 1, wherein the configuration file further comprises a timer for timing a timeout time timeout for acquiring the image,
if the time interval is over 1/max _ fps, the graphics card instruction sent by the virtual machine is not received, starting the timer, and judging the difference between the current recording time of the timer and the time when the last graphics card instruction is received;
if the difference value is smaller than timeout, acquiring a full-screen streaming mode image of the image and sending the full-screen streaming mode image to the client;
if the difference is larger than or equal to timeout, the image is not updated, the timer is closed, and the image is sent to the client in a compression coding mode.
3. The method of claim 2, wherein the full-screen streaming mode image is transmitted to the client and the timer is reset at a time interval of 1/max _ fps if the difference is less than timeout.
4. The method of claim 1, wherein determining whether the frame rate of the received image transmitted by the virtual machine is higher than a set maximum frame rate max _ fps comprises:
finding corresponding image data on the shared storage of the virtual machine through an image storage address in the surface;
and judging whether the frame rate of the currently found image is higher than the set highest frame rate max _ fps.
5. The method as claimed in claim 1, wherein a graphics card command of the virtual machine is received, a protocol connection with the virtual machine is established before the time of the graphics card command is recorded, and identification information of the virtual machine is obtained.
6. The method according to claim 5, wherein the identification information includes at least ip and port information of the virtual machine.
7. A network device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the method of controlling the image frame rate of the virtual machine according to any one of claims 1 to 6.
8. A storage medium having computer program instructions stored thereon, wherein the program instructions, when executed by a processor, are adapted to implement a virtual machine image frame rate control method as claimed in any one of claims 1 to 6.
CN202110336922.XA 2021-03-29 2021-03-29 Virtual machine image frame rate control method, network equipment and storage medium Active CN113207016B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110336922.XA CN113207016B (en) 2021-03-29 2021-03-29 Virtual machine image frame rate control method, network equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110336922.XA CN113207016B (en) 2021-03-29 2021-03-29 Virtual machine image frame rate control method, network equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113207016A CN113207016A (en) 2021-08-03
CN113207016B true CN113207016B (en) 2022-05-27

Family

ID=77025865

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110336922.XA Active CN113207016B (en) 2021-03-29 2021-03-29 Virtual machine image frame rate control method, network equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113207016B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113973224A (en) * 2021-09-18 2022-01-25 阿里巴巴(中国)有限公司 Method for transmitting media information, computing device and storage medium
CN117640627B (en) * 2024-01-25 2024-04-09 微网优联科技(成都)有限公司 Collaborative work method and system realized through cloud computer

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101170694A (en) * 2007-10-19 2008-04-30 上海广电(集团)有限公司中央研究院 A method for real time monitoring signals of encoding device
CN103747077A (en) * 2013-12-31 2014-04-23 华为技术有限公司 Transmission mechanism adjusting method, server side and client
CN105611213A (en) * 2016-01-04 2016-05-25 京东方科技集团股份有限公司 Image processing method, image play method and related device and system
CN109168013A (en) * 2018-09-19 2019-01-08 广州华多网络科技有限公司 A kind of method, apparatus, equipment and computer readable storage medium for taking out frame
CN110493596A (en) * 2019-09-02 2019-11-22 西北工业大学 A kind of video coding framework neural network based
CN110519633A (en) * 2019-08-02 2019-11-29 广州视源电子科技股份有限公司 Wireless screen transmission method, apparatus, system and intelligent interaction plate

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4777274B2 (en) * 2007-02-19 2011-09-21 キヤノン株式会社 Video playback apparatus and control method thereof

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101170694A (en) * 2007-10-19 2008-04-30 上海广电(集团)有限公司中央研究院 A method for real time monitoring signals of encoding device
CN103747077A (en) * 2013-12-31 2014-04-23 华为技术有限公司 Transmission mechanism adjusting method, server side and client
CN105611213A (en) * 2016-01-04 2016-05-25 京东方科技集团股份有限公司 Image processing method, image play method and related device and system
CN109168013A (en) * 2018-09-19 2019-01-08 广州华多网络科技有限公司 A kind of method, apparatus, equipment and computer readable storage medium for taking out frame
CN110519633A (en) * 2019-08-02 2019-11-29 广州视源电子科技股份有限公司 Wireless screen transmission method, apparatus, system and intelligent interaction plate
CN110493596A (en) * 2019-09-02 2019-11-22 西北工业大学 A kind of video coding framework neural network based

Also Published As

Publication number Publication date
CN113207016A (en) 2021-08-03

Similar Documents

Publication Publication Date Title
CN111399964B (en) Cloud desktop platform based on video streaming technology
US7640358B2 (en) Methods and systems for HTTP streaming using an intelligent HTTP client
US7970966B1 (en) Method and apparatus for providing a low-latency connection between a data processor and a remote graphical user interface over a network
CN108810636B (en) Video playing method, virtual reality equipment, server, system and storage medium
RU2368940C2 (en) Synchronised graphic data and region data for systems of remote handling graphic data
CN113207016B (en) Virtual machine image frame rate control method, network equipment and storage medium
CN102158553A (en) Method and device for playing multi-media files for remote desktop
EP3503570A1 (en) Method of transmitting video frames from a video stream to a display and corresponding apparatus
US20010013128A1 (en) Data reception/playback method, data reception/playback apparatus, data transmission method, and data transmission apparatus
US8843983B2 (en) Video decomposition and recomposition
CN105867870B (en) The echo method and apparatus of combination window
KR101942270B1 (en) Media playback apparatus and method including delay prevention system
CN105025294B (en) A kind of transmission of video control device and method
US10404606B2 (en) Method and apparatus for acquiring video bitstream
CN114363667A (en) Client, server, receiving method and sending method
JP2001092752A (en) System for distributing picture data and recording medium used for the same
JP2001242876A (en) Method and device for receiving and reproducing data and method and device for transmitting data
CN106686635B (en) Data transmission method and device based on control and configuration protocol of wireless access point
CN102439935A (en) Media adaptation method and apparatus
JP3967443B2 (en) Image data transmission / reception system, transmitting apparatus thereof, receiving apparatus thereof, and storage medium storing program thereof
CN113254130B (en) Method for adaptively adjusting resolution of cloud desktop
EP3440842A1 (en) Media stream prioritization
CN114827692A (en) System for operating cloud desktop based on smart television
TW201421980A (en) Steaming connection management method and system
CN113301438A (en) Cloud desktop video playing method based on underlying virtualization technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant