CN114125324B - Video stitching method and device, electronic equipment and storage medium - Google Patents

Video stitching method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114125324B
CN114125324B CN202111315996.1A CN202111315996A CN114125324B CN 114125324 B CN114125324 B CN 114125324B CN 202111315996 A CN202111315996 A CN 202111315996A CN 114125324 B CN114125324 B CN 114125324B
Authority
CN
China
Prior art keywords
video
frame
image
frames
image frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111315996.1A
Other languages
Chinese (zh)
Other versions
CN114125324A (en
Inventor
胡天舒
郭汉奇
韩钧宇
洪智滨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111315996.1A priority Critical patent/CN114125324B/en
Publication of CN114125324A publication Critical patent/CN114125324A/en
Priority to JP2022152697A priority patent/JP2023070068A/en
Priority to EP22198168.1A priority patent/EP4177887A1/en
Priority to US17/959,727 priority patent/US20230145443A1/en
Application granted granted Critical
Publication of CN114125324B publication Critical patent/CN114125324B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/038—Cross-faders therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Studio Devices (AREA)
  • Television Systems (AREA)
  • Studio Circuits (AREA)

Abstract

The disclosure provides a video stitching method, a video stitching device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to the technical field of deep learning and computer vision. The specific implementation scheme is as follows: inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video; sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1; the first video and the second video are stitched into a target video based on the intermediate frame and the L image frames in the first video and the L image frames in the second video. The embodiment of the application can realize smooth transition between videos, can greatly reduce video splicing difficulty, can also improve splicing speed and reduce cost.

Description

Video stitching method and device, electronic equipment and storage medium
Technical Field
The disclosure relates to the technical field of artificial intelligence, and further relates to the technical field of deep learning and computer vision, in particular to a video stitching method, a video stitching device, electronic equipment and a storage medium.
Background
With the increasing importance of entertainment and leisure, the video media such as movies and short videos are gradually and tightly connected with the lives of people, so that more and more video creation demands are also emerging from our society. In capturing and authoring such video content, it is often necessary to capture multiple shots and then stitch the shot clips together.
Disclosure of Invention
The disclosure provides a video stitching method, a video stitching device, electronic equipment and a storage medium.
In a first aspect, the present application provides a video stitching method, the method comprising:
inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video;
sequentially selecting L image frames from the first video to the second video according to the sequence from the back to the front and the sequence from the front to the back respectively; wherein L is a natural number greater than 1;
and splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video.
In a second aspect, the present application provides a video stitching apparatus, the apparatus comprising: the device comprises a frame inserting module, a selecting module and a splicing module; wherein,
The frame inserting module is used for inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video;
the selecting module is used for sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1;
the splicing module is used for splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video.
In a third aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video stitching method described in any of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a storage medium having stored thereon a computer program which, when executed by a processor, implements the video stitching method of any embodiment of the present application.
In a fifth aspect, a computer program product is provided which, when executed by a computer device, implements the video stitching method according to any of the embodiments of the present application.
According to the technology, the problems that in the prior art, a large amount of manpower is required to be consumed for realizing video splicing through manual PS, the speed is low, and the time consumption is long are solved; and the technical problem of high price and high cost is solved, and the technical scheme provided by the application can realize smooth transition between videos, can greatly reduce video splicing difficulty, can also improve splicing speed and reduce cost.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a schematic flow chart of a video stitching method according to an embodiment of the present application;
fig. 2 is a second flow chart of a video stitching method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a first principle of a video stitching method according to an embodiment of the present application;
Fig. 4 is a third flow chart of a video stitching method according to an embodiment of the present application;
fig. 5 is a second schematic diagram of a video stitching method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a video stitching device according to an embodiment of the present application;
fig. 7 is a block diagram of an electronic device for implementing a video stitching method of an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example 1
Fig. 1 is a schematic flow chart of a video stitching method provided in an embodiment of the present application, where the method may be performed by a video stitching apparatus or an electronic device, and the apparatus or the electronic device may be implemented by software and/or hardware, and the apparatus or the electronic device may be integrated into any intelligent device with a network communication function. As shown in fig. 1, the video stitching method may include the steps of:
S101, inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video.
In this step, the electronic device may insert an intermediate frame between the last image frame of the first video and the first image frame of the second video. Specifically, the electronic device may input the last image frame of the first video and the first image frame of the second video into a pre-built image model, through which one image frame is output as an intermediate frame between the last image frame of the first video and the first image frame of the second video.
S102, sequentially selecting L image frames in the first video and the second video according to a sequence from back to front and a sequence from front to back respectively; wherein L is a natural number greater than 1.
In this step, the electronic device may sequentially select L image frames in the first video and the second video in the order from back to front and the order from front to back, respectively; wherein L is a natural number greater than 1. For example, assume that a first video is video a and a second video is video B; when the value of L is 5, the electronic device may select 5 image frames in the video a in order from back to front, where the image frames are respectively: A_N-4, A_N-3, A_N-2, A_N-1 and A_N; at the same time, 5 image frames, b_1, b_2, b_3, b_4 and b_5, respectively, can be selected from the video B in the order from front to back.
And S103, splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video.
In this step, the electronic device may stitch the first video and the second video into the target video based on the intermediate frame and the L image frames in the first video and the L image frames in the second video. Specifically, the electronic device may first insert L-2 image frames between each of the second last image frame to the first last L-1 image frame of the first video and the intermediate frame, respectively, as candidate transition frames between the first video and the intermediate frame; meanwhile, L-2 image frames are respectively inserted between each of the second image frame to the L-1 image frame of the second video and the intermediate frame to serve as candidate transition frames between the second video and the intermediate frame; and splicing the first video and the second video into a target video based on the last L image frame, the middle frame and the L image frame of the second video of the first video, the candidate transition frame between the first video and the middle frame and the candidate transition frame between the second video and the middle frame.
In the existing video stitching technology, after stitching multiple segments of shots, the synthesized video often has a jump at the stitching position, especially for the shot with a person, even if the machine position is motionless, the smooth transition between the two shots is impossible due to the shaking of the gesture of the person. Moreover, the existing video stitching technology is often realized through manual PS, so that a great deal of manpower is required to be consumed, the speed is low, and the time consumption is long; and is expensive and costly. The video splicing method provided by the embodiment of the application can realize smooth transition between videos, can greatly reduce video splicing difficulty, can improve splicing speed and reduce cost.
According to the video stitching method, an intermediate frame is inserted between the last image frame of the first video and the first image frame of the second video; sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1; and splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video. That is, the present application can smoothly connect the first video and the second video together based on the intermediate frame and the L image frames in the first video and the L image frames in the second video, so as to avoid occurrence of a very obvious jump trace. In the existing video splicing method, the method is often realized through manual PS, a large amount of manpower is required to be consumed, the speed is low, and the time consumption is long; and is expensive and costly. Because the method adopts the technical means of realizing video splicing based on the intermediate frames, L image frames in the first video and L image frames in the second video, the problems that a great deal of manpower is required to be consumed for realizing video splicing through manual PS in the prior art, the speed is low and the time consumption is long are overcome; the technical scheme provided by the application can realize smooth transition between videos, greatly reduce video splicing difficulty, improve splicing speed and reduce cost; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example two
Fig. 2 is a second flowchart of a video stitching method according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 2, the video stitching method may include the steps of:
s201, an intermediate frame is inserted between the last image frame of the first video and the first image frame of the second video.
S202, sequentially selecting L image frames in the first video and the second video according to a sequence from back to front and a sequence from front to back respectively; wherein L is a natural number greater than 1.
S203, respectively inserting L-2 image frames between each of the last image frame to the last L-1 image frame of the first video and the intermediate frame as candidate transition frames between the first video and the intermediate frame.
In this step, the electronic device may insert L-2 image frames between each of the penultimate image frame through the penultimate L-1 image frame of the first video and the intermediate frame, respectively, as candidate transition frames between the first video and the intermediate frame. For example, assume that a first video is video a and a second video is video B; when the value of L is 5, the electronic device may select 5 image frames in the video a in order from back to front, where the image frames are respectively: A_N-4, A_N-3, A_N-2, A_N-1 and A_N; at the same time, 5 image frames, b_1, b_2, b_3, b_4 and b_5, respectively, can be selected from the video B in the order from front to back. In this step, the electronic device may insert 3 image frames between the intermediate frame and a_n_1, between the intermediate frame and a_n_2, and between the intermediate frame and a_n_3, respectively, as candidate transition frames between the first video and the intermediate frame.
S204, respectively inserting L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame.
In this step, the electronic device may insert L-2 image frames between each of the second image frame through the L-1 th image frame of the second video and the intermediate frame, respectively, as candidate transition frames between the second video and the intermediate frame. For example, assume that a first video is video a and a second video is video B; when the value of L is 5, the electronic device may select 5 image frames in the video a in order from back to front, where the image frames are respectively: A_N-4, A_N-3, A_N-2, A_N-1 and A-N; at the same time, 5 image frames, b_1, b_2, b_3, b_4 and b_5, respectively, can be selected from the video B in the order from front to back. In this step, the electronic device may insert 3 image frames between the intermediate frame and b_2, the intermediate frame and b_3, and the intermediate frame and b_4, respectively, as candidate transition frames between the second video and the intermediate frame.
S205, splicing the first video and the second video into a target video based on the last L image frame of the first video, the middle frame, the L image frame of the second video, and the candidate transition frame between the first video and the middle frame and the candidate transition frame between the second video and the middle frame.
In this step, the electronic device may stitch the first video and the second video into the target video based on the last L image frame of the first video, the intermediate frame, the L-th image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame. Specifically, the electronic device may first select an image frame from L-2 image frames between each of the last L-1 image frames to the last image frame of the first video and the intermediate frame, as a target transition frame corresponding to each of the last L-1 image frames to the last image frame of the first video; then, respectively selecting one image frame from L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the middle frame as a target transition frame corresponding to each of the second image frame to the L-1 image frame of the second video; and splicing the first video and the second video into target video based on the last L image frame, the middle frame and the L image frame of the second video of the first video, the target transition frame corresponding to each image frame from the last L-1 image frame to the last second image frame of the first video and the target transition frame corresponding to each image frame from the second image frame to the L-1 image frame of the second video. Preferably, the electronic device may select the first image frame to the L-2 image frame from the L-2 image frames between each of the last L-1 image frame to the last second image frame of the first video and the intermediate frame, as the target transition frame corresponding to each of the last L-1 image frame to the last second image frame of the first video. Meanwhile, the electronic device may further select, from the second to the L-2 image frames between each of the L-1 th to the L-2 th image frames of the second video and the intermediate frame, the L-2 th to the first image frame as the target transition frame corresponding to each of the second to the L-1 th image frames of the second video.
Fig. 3 is a schematic diagram of a first principle of a video stitching method according to an embodiment of the present application. As shown in fig. 3, assume that the first video is video a and the second video is video B; when the value of L is 5, the electronic device may select 5 image frames in the video a in order from back to front, where the image frames are respectively: A_N-4 (N-4 in the figure), A_N-3 (N-3 in the figure), A_N-2 (N-2 in the figure), A_N-1 (N-1 in the figure) and A_N (N in the figure); at the same time, 5 image frames, respectively b_1 (1 in the figure), b_2 (2 in the figure), b_3 (3 in the figure), b_4 (4 in the figure) and b_5 (5 in the figure), can be selected from the video B in the order from front to back. Then the electronic equipment can respectively insert 3 image frames between the intermediate frame and the image frame N-1, between the intermediate frame and the image frame N-2 and between the intermediate frame and the image frame N-3 to serve as candidate transition frames between the first video and the intermediate frame; specifically, 3 image frames inserted between the intermediate frame and the image frame N-1 may be expressed as: a_N-1_1, a_N-1_2, a_N-1_3; the 3 image frames inserted between the intermediate frame and the image frame N-2 can be expressed as: a_N-2_1, a_N-2_2, a_N-2_3; the 3 image frames inserted between the intermediate frame and the image frame N-3 can be expressed as: a_N-3_1, a_N-3_2, a_N-3_3. Meanwhile, the electronic equipment can also respectively insert 3 image frames between the intermediate frame and the image frame 2, between the intermediate frame and the image frame 3 and between the intermediate frame and the image frame 4 to serve as candidate transition frames between the second video and the intermediate frame; specifically, 3 image frames inserted between the intermediate frame and the image frame 2 can be expressed as: b_2_3, b_2_2, b_2_1; the 3 image frames inserted between the intermediate frame and the image frame 3 can be expressed as: b_3_3, b_3_2, b_3_1; the 3 image frames inserted between the intermediate frame and the image frame 4 can be expressed as: b_4_3, b_4_2, b_4_1. Then, taking A_N-3_1, A_N-2_2 and A_N-1_3 as target transition frames corresponding to N-3, N-2 and N-1 respectively; b_2_3, b_3_2 and b_4_1 are respectively used as target transition frames corresponding to the image frame 2, the image frame 3 and the image frame 4; and finally, connecting N-4, A_N-3_1, A_N-2_2, A_N-1_3, an intermediate frame, B_2_3, B_3_2, B_4_1 and an image frame 4 to obtain a target video after splicing the video A and the video B.
According to the video stitching method, an intermediate frame is inserted between the last image frame of the first video and the first image frame of the second video; sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1; and splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video. That is, the present application can smoothly connect the first video and the second video together based on the intermediate frame and the L image frames in the first video and the L image frames in the second video, so as to avoid occurrence of a very obvious jump trace. In the existing video splicing method, the method is often realized through manual PS, a large amount of manpower is required to be consumed, the speed is low, and the time consumption is long; and is expensive and costly. Because the method adopts the technical means of realizing video splicing based on the intermediate frames, L image frames in the first video and L image frames in the second video, the problems that a great deal of manpower is required to be consumed for realizing video splicing through manual PS in the prior art, the speed is low and the time consumption is long are overcome; the technical scheme provided by the application can realize smooth transition between videos, greatly reduce video splicing difficulty, improve splicing speed and reduce cost; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example III
Fig. 4 is a third flow chart of a video stitching method according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 4, the video stitching method may include the steps of:
s401, inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video.
S402, sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1.
S403, respectively inserting M image frames between each of the last image frame to the last L-1 image frames of the first video and the intermediate frame as candidate transition frames between the first video and the intermediate frame.
In this step, the electronic device may insert M image frames between each of the second last image frame to the first last L-1 image frame of the first video and the intermediate frame, respectively, as candidate transition frames between the first video and the intermediate frame. For example, the electronic device may insert 9 image frames between the intermediate frame and image frame N-1, the intermediate frame and image frame N-2, and the intermediate frame and image frame N-3, respectively, as candidate transition frames between the first video and the intermediate frame; meanwhile, 9 image frames are inserted between the intermediate frame and the image frame 2, between the intermediate frame and the image frame 3, and between the intermediate frame and the image frame 4, respectively, as candidate transition frames between the second video and the intermediate frame.
S404, respectively inserting M image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; wherein M is a natural number greater than 1.
S405, splicing the first video and the second video into a target video based on the last L image frame of the first video, the middle frame, the L image frame of the second video, and the candidate transition frame between the first video and the middle frame and the candidate transition frame between the second video and the middle frame.
In this step, the electronic device may stitch the first video and the second video into the target video based on the last L image frame of the first video, the intermediate frame, the L-th image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame. Specifically, the present application may sample non-linearly based on some non-linear function, such as sigmoid, cosine, so that the transition curve is smoother.
Fig. 5 is a second schematic diagram of a video stitching method according to an embodiment of the present application. As shown in fig. 5, assume that the first video is video a and the second video is video B; when the value of L is 5, the electronic device may select 5 image frames in the video a in order from back to front, where the image frames are respectively: A_N-4 (N-4 in the figure), A_N-3 (N-3 in the figure), A_N-2 (N-2 in the figure), A_N-1 (N-1 in the figure) and A_N (N in the figure); at the same time, 5 image frames, respectively b_1 (1 in the figure), b_2 (2 in the figure), b_3 (3 in the figure), b_4 (4 in the figure) and b_5 (5 in the figure), can be selected from the video B in the order from front to back. The application can sample in a nonlinear way based on nonlinear functions such as sigmoid and cosine, and the play sequence after splicing is A_N-4, A_N-3_2, A_N-2_4, A_N-1_7, intermediate frame, B_2_7, B_3_4, B_4_2 and B_5.
According to the video stitching method, an intermediate frame is inserted between the last image frame of the first video and the first image frame of the second video; sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1; and splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video. That is, the present application can smoothly connect the first video and the second video together based on the intermediate frame and the L image frames in the first video and the L image frames in the second video, so as to avoid occurrence of a very obvious jump trace. In the existing video splicing method, the method is often realized through manual PS, a large amount of manpower is required to be consumed, the speed is low, and the time consumption is long; and is expensive and costly. Because the method adopts the technical means of realizing video splicing based on the intermediate frames, L image frames in the first video and L image frames in the second video, the problems that a great deal of manpower is required to be consumed for realizing video splicing through manual PS in the prior art, the speed is low and the time consumption is long are overcome; the technical scheme provided by the application can realize smooth transition between videos, greatly reduce video splicing difficulty, improve splicing speed and reduce cost; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example IV
Fig. 6 is a schematic structural diagram of a video stitching device according to an embodiment of the present application. As shown in fig. 6, the apparatus 600 includes: a frame inserting module 601, a selecting module 602 and a splicing module 603; wherein,
the frame inserting module 601 is configured to insert an intermediate frame between a last image frame of the first video and a first image frame of the second video;
the selecting module 602 is configured to sequentially select L image frames in the first video and the second video according to a sequence from back to front and a sequence from front to back, respectively; wherein L is a natural number greater than 1;
the stitching module 603 is configured to stitch the first video and the second video into a target video based on the intermediate frame, and L image frames in the first video and L image frames in the second video.
Further, the stitching module 603 is specifically configured to insert L-2 image frames between each of the last second image frame to the last L-1 image frame of the first video and the intermediate frame, as candidate transition frames between the first video and the intermediate frame; inserting L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
Further, the stitching module 603 is specifically configured to select one image frame from L-2 image frames between each of the last L-1 image frames to the last second image frame of the first video and the intermediate frame, as a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video; respectively selecting one image frame from L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as a target transition frame corresponding to each of the second image frame to the L-1 image frame of the second video; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L-th image frame of the second video, a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video, and a target transition frame corresponding to each of the second image frames to the L-1-th image frames of the second video.
Further, the stitching module 603 is specifically configured to select, from L-2 image frames between each of the last L-1 image frames to the last second image frame of the first video and the intermediate frame, a first image frame to a L-2 image frame as a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video.
Further, the stitching module 603 is specifically configured to select, from L-2 image frames between each of the second image frame to the L-1 th image frame of the second video and the intermediate frame, the L-2 th image frame to the first image frame as a target transition frame corresponding to each of the second image frame to the L-1 th image frame of the second video.
Further, the stitching module 603 is specifically configured to insert M image frames between each of the image frames from the last second image frame to the last L-1 image frame of the first video and the intermediate frame, as candidate transition frames between the first video and the intermediate frame; inserting M image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; wherein M is a natural number greater than 1; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
The video stitching device can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment may be referred to the video stitching method provided in any embodiment of the present application.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.
Example five
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the apparatus 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Various components in device 700 are connected to I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as a video stitching method. For example, in some embodiments, the video stitching method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM 702 and/or communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the video stitching method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the video stitching method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (14)

1. A method of video stitching, the method comprising:
inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video;
sequentially selecting L image frames from the first video to the second video according to the sequence from the back to the front and the sequence from the front to the back respectively; wherein L is a natural number greater than 1;
splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video;
Wherein stitching the first video and the second video to a target video based on the intermediate frame and L image frames in the first video and L image frames in the second video comprises:
and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
2. The method of claim 1, wherein the candidate transition frames between the first video and the intermediate frame and the candidate transition frames between the second video and the intermediate frame comprise:
inserting L-2 image frames between each of the last second image frame to the last L-1 image frame of the first video and the intermediate frame respectively as candidate transition frames between the first video and the intermediate frame;
l-2 image frames are inserted between each of the second image frame to the L-1 image frame of the second video and the intermediate frame respectively as candidate transition frames between the second video and the intermediate frame.
3. The method of claim 2, wherein stitching the first video and the second video into a target video based on a next to last L image frame of the first video, the intermediate frame, an L-th image frame of the second video, and a candidate transition frame between the first video and the intermediate frame and a candidate transition frame between the second video and the intermediate frame comprises:
respectively selecting one image frame from L-2 image frames between each image frame from the L-1 image frame to the second-to-last image frame of the first video and the intermediate frame, and taking the image frame as a target transition frame corresponding to each image frame from the L-1 image frame to the second-to-last image frame of the first video;
respectively selecting one image frame from L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as a target transition frame corresponding to each of the second image frame to the L-1 image frame of the second video;
and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L-th image frame of the second video, a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video, and a target transition frame corresponding to each of the second image frames to the L-1-th image frames of the second video.
4. A method according to claim 3, wherein selecting one image frame among the L-2 image frames between each of the L-1 image frames to the second-to-last image frame of the first video and the intermediate frame, respectively, as the target transition frame corresponding to each of the L-2 image frames to the second-to-last image frame of the first video, comprises:
and respectively selecting a first image frame to a L-2 image frame from each of the L-1 image frames to the second last image frame of the first video and the L-2 image frames between the intermediate frames as target transition frames corresponding to each of the L-1 image frames to the second last image frame of the first video.
5. A method according to claim 3, wherein selecting one image frame among L-2 image frames between each of the second image frame to the L-1 th image frame of the second video and the intermediate frame as the target transition frame corresponding to each of the second image frame to the L-1 th image frame of the second video includes:
and respectively selecting the L-2 image frames from the first image frame from each of the second image frame to the L-1 image frame of the second video and the L-2 image frames between the intermediate frame as target transition frames corresponding to each of the second image frame to the L-1 image frame of the second video.
6. The method of claim 1, wherein stitching the first video and the second video into a target video based on the intermediate frame and L image frames in the first video and L image frames in the second video comprises:
respectively inserting M image frames between each of the last image frame to the last L-1 image frame of the first video and the intermediate frame as candidate transition frames between the first video and the intermediate frame;
inserting M image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; wherein M is a natural number greater than 1;
and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
7. A video stitching device, the device comprising: the device comprises a frame inserting module, a selecting module and a splicing module; wherein,
The frame inserting module is used for inserting an intermediate frame between the last image frame of the first video and the first image frame of the second video;
the selecting module is used for sequentially selecting L image frames in the first video and the second video according to the sequence from back to front and the sequence from front to back respectively; wherein L is a natural number greater than 1;
the splicing module is used for splicing the first video and the second video into a target video based on the intermediate frame, L image frames in the first video and L image frames in the second video;
wherein, splice module includes:
and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
8. The apparatus according to claim 7, wherein the stitching module is configured to insert L-2 image frames between each of the second last image frame to the first last L-1 image frame of the first video and the intermediate frame, respectively, as candidate transition frames between the first video and the intermediate frame; inserting L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
9. The apparatus of claim 8, wherein the stitching module is specifically configured to select, from L-2 image frames between each of the last L-1 image frames to the last second image frame of the first video and the intermediate frame, one image frame as a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video; respectively selecting one image frame from L-2 image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as a target transition frame corresponding to each of the second image frame to the L-1 image frame of the second video; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L-th image frame of the second video, a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video, and a target transition frame corresponding to each of the second image frames to the L-1-th image frames of the second video.
10. The apparatus of claim 9, wherein the stitching module is specifically configured to select, from L-2 image frames between each of the last L-1 image frames to the last second image frame of the first video and the intermediate frame, a first image frame to a L-2 image frame, respectively, as a target transition frame corresponding to each of the last L-1 image frames to the last second image frame of the first video.
11. The apparatus of claim 9, wherein the stitching module is specifically configured to select, from L-2 image frames between each of the second image frame to the L-1 th image frame of the second video and the intermediate frame, the L-2 th image frame to the first image frame as the target transition frame corresponding to each of the second image frame to the L-1 th image frame of the second video.
12. The apparatus according to claim 7, wherein the stitching module is specifically configured to insert M image frames between each of the second last image frame to the first last L-1 image frames of the first video and the intermediate frame, as candidate transition frames between the first video and the intermediate frame; inserting M image frames between each of the second image frame to the L-1 image frame of the second video and the intermediate frame as candidate transition frames between the second video and the intermediate frame; wherein M is a natural number greater than 1; and splicing the first video and the second video into a target video based on the last L image frame of the first video, the intermediate frame, the L image frame of the second video, and the candidate transition frame between the first video and the intermediate frame and the candidate transition frame between the second video and the intermediate frame.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.
CN202111315996.1A 2021-11-08 2021-11-08 Video stitching method and device, electronic equipment and storage medium Active CN114125324B (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202111315996.1A CN114125324B (en) 2021-11-08 2021-11-08 Video stitching method and device, electronic equipment and storage medium
JP2022152697A JP2023070068A (en) 2021-11-08 2022-09-26 Video stitching method, apparatus, electronic device, and storage medium
EP22198168.1A EP4177887A1 (en) 2021-11-08 2022-09-27 Video stitching method and apparatus, electronic device, and storage medium
US17/959,727 US20230145443A1 (en) 2021-11-08 2022-10-04 Video stitching method and apparatus, electronic device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111315996.1A CN114125324B (en) 2021-11-08 2021-11-08 Video stitching method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114125324A CN114125324A (en) 2022-03-01
CN114125324B true CN114125324B (en) 2024-02-06

Family

ID=80377591

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111315996.1A Active CN114125324B (en) 2021-11-08 2021-11-08 Video stitching method and device, electronic equipment and storage medium

Country Status (4)

Country Link
US (1) US20230145443A1 (en)
EP (1) EP4177887A1 (en)
JP (1) JP2023070068A (en)
CN (1) CN114125324B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114679605B (en) * 2022-03-25 2023-07-18 腾讯科技（深圳）有限公司 Video transition method, device, computer equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1261756A (en) * 1998-11-20 2000-08-02 通用仪器公司 Video data joint in asymptotic refreshing video data stream
EP3494706A4 (en) * 2016-09-01 2019-06-12 Samsung Electronics Co., Ltd. Image streaming method and electronic device for supporting the same
CN111294644A (en) * 2018-12-07 2020-06-16 腾讯科技（深圳）有限公司 Video splicing method and device, electronic equipment and computer storage medium
CN111640187A (en) * 2020-04-20 2020-09-08 中国科学院计算技术研究所 Video splicing method and system based on interpolation transition
CN111970562A (en) * 2020-08-17 2020-11-20 Oppo广东移动通信有限公司 Video processing method, video processing device, storage medium and electronic equipment
CN112200739A (en) * 2020-09-30 2021-01-08 北京大米科技有限公司 Video processing method and device, readable storage medium and electronic equipment
CN113542651A (en) * 2021-05-28 2021-10-22 北京迈格威科技有限公司 Model training method, video frame interpolation method and corresponding device

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5359712A (en) * 1991-05-06 1994-10-25 Apple Computer, Inc. Method and apparatus for transitioning between sequences of digital information
US20150155009A1 (en) * 2013-12-03 2015-06-04 Nokia Corporation Method and apparatus for media capture device position estimate- assisted splicing of media
EP3013055A1 (en) * 2014-10-23 2016-04-27 Thomson Licensing Video frame set processing cost management method, apparatus and related computer program product
CN111385490B (en) * 2018-12-28 2021-07-13 清华大学 Video splicing method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1261756A (en) * 1998-11-20 2000-08-02 通用仪器公司 Video data joint in asymptotic refreshing video data stream
EP3494706A4 (en) * 2016-09-01 2019-06-12 Samsung Electronics Co., Ltd. Image streaming method and electronic device for supporting the same
CN111294644A (en) * 2018-12-07 2020-06-16 腾讯科技（深圳）有限公司 Video splicing method and device, electronic equipment and computer storage medium
CN111640187A (en) * 2020-04-20 2020-09-08 中国科学院计算技术研究所 Video splicing method and system based on interpolation transition
CN111970562A (en) * 2020-08-17 2020-11-20 Oppo广东移动通信有限公司 Video processing method, video processing device, storage medium and electronic equipment
CN112200739A (en) * 2020-09-30 2021-01-08 北京大米科技有限公司 Video processing method and device, readable storage medium and electronic equipment
CN113542651A (en) * 2021-05-28 2021-10-22 北京迈格威科技有限公司 Model training method, video frame interpolation method and corresponding device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
A video forensic technique for detecting frame deletion and insertion;A. Gironi;《2014 IEEE International Conference on Acoustics, Speech and Signal Processing》;全文 *
改进同余变换不变特征的视频拼接算法;杨英杰;《计算机应用与软件》;全文 *

Also Published As

Publication number Publication date
JP2023070068A (en) 2023-05-18
EP4177887A1 (en) 2023-05-10
US20230145443A1 (en) 2023-05-11
CN114125324A (en) 2022-03-01

Similar Documents

Publication Publication Date Title
CN113436100B (en) Method, apparatus, device, medium, and article for repairing video
CN114693934B (en) Training method of semantic segmentation model, video semantic segmentation method and device
CN114125324B (en) Video stitching method and device, electronic equipment and storage medium
CN113360683B (en) Method for training cross-modal retrieval model and cross-modal retrieval method and device
CN113453073A (en) Image rendering method and device, electronic equipment and storage medium
CN116594563A (en) Distributed storage capacity expansion method and device, electronic equipment and storage medium
CN113873323B (en) Video playing method, device, electronic equipment and medium
CN113139463B (en) Method, apparatus, device, medium and program product for training a model
CN115499707A (en) Method and device for determining video similarity
CN116862762A (en) Video superdivision method, device, equipment and storage medium
CN114979471B (en) Interface display method, device, electronic equipment and computer readable storage medium
CN113542802A (en) Video transition method and device
CN113836455A (en) Special effect rendering method, device, equipment, storage medium and computer program product
CN112560987A (en) Image sample processing method, device, equipment, storage medium and program product
CN113490044B (en) Video playing method and device, electronic equipment and storage medium
CN112541346A (en) Abstract generation method and device, electronic equipment and readable storage medium
CN113627354B (en) A model training and video processing method, which comprises the following steps, apparatus, device, and storage medium
CN112784997B (en) Annotation rechecking method, device, equipment, storage medium and program product
CN115934181B (en) Data loading method, device, electronic equipment and storage medium
CN113438428B (en) Method, apparatus, device and computer-readable storage medium for automated video generation
CN116486195B (en) Training method of video recommendation model, video recommendation method and device
CN114786051B (en) Video rendering method and device, electronic equipment and storage medium
CN113691866B (en) Video processing method, device, electronic equipment and medium
CN114125135B (en) Video content presentation method and device, electronic equipment and storage medium
CN114173177B (en) Video processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant