CN113259715A - Method and device for processing multi-channel video data, electronic equipment and medium - Google Patents

Method and device for processing multi-channel video data, electronic equipment and medium Download PDF

Info

Publication number
CN113259715A
CN113259715A CN202110496334.2A CN202110496334A CN113259715A CN 113259715 A CN113259715 A CN 113259715A CN 202110496334 A CN202110496334 A CN 202110496334A CN 113259715 A CN113259715 A CN 113259715A
Authority
CN
China
Prior art keywords
fragment
video
frame
fragments
same
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110496334.2A
Other languages
Chinese (zh)
Inventor
王辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Xiaopeng Motors Technology Co Ltd
Original Assignee
Guangzhou Xiaopeng Motors Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Xiaopeng Motors Technology Co Ltd filed Critical Guangzhou Xiaopeng Motors Technology Co Ltd
Priority to CN202110496334.2A priority Critical patent/CN113259715A/en
Publication of CN113259715A publication Critical patent/CN113259715A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Abstract

The embodiment of the invention provides a processing method, a device, electronic equipment and a computer storage medium of multi-channel video data, in the processing process of video data, in particular to the remote transmission process of video data, video fragments simultaneously sent by multi-channel transmission channels can be obtained, the fragment serial number of each video fragment and the target frame identifier of the video frame are obtained, then the video fragments which belong to the same frame identifier and are sent by different transmission channels are added to the same preset fragment set according to the fragment serial number, then the video fragments of a plurality of fragment sets are simultaneously packaged to generate the video frame, on one hand, the stability of a network is ensured by a plurality of transmission channels, on the other hand, the video data is processed by the modes of fragment, multi-channel transmission, packaging and the like, under the condition of reducing transmission data units, the real-time performance and the reliability of data stream transmission are ensured.

Description

Method and device for processing multi-channel video data, electronic equipment and medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method for processing multiple channels of video data, an apparatus for processing multiple channels of video data, an electronic device, and a computer-readable storage medium.
Background
With the development of network communication technology, especially 5G communication technology, the application of data remote transmission is becoming more and more popular, for example, remote driving of vehicles, remote control of devices, remote monitoring, and the like. In the remote transmission process related to the video data, the data volume of the transmitted video data is large due to the fact that corresponding application scenes are often changed greatly in real time, and in the data transmission process, on one hand, the stability of a network cannot be guaranteed, on the other hand, when a receiving end obtains the video data and packages the video data due to the fact that the conditions of data packet loss, time delay and the like exist, the application of the data remote transmission is affected due to the problems of incomplete data, poor transmission real-time performance and reliability and the like.
Disclosure of Invention
The embodiment of the invention provides a method and a device for processing multi-channel video data, electronic equipment and a computer readable storage medium, which are used for solving or partially solving the problems of unstable network and low real-time and reliability of data stream transmission in the processing process of video data in the prior art.
In order to solve the above problem, an embodiment of the present invention discloses a method for processing multiple paths of video data, including:
acquiring video fragments simultaneously transmitted by multiple transmission channels;
acquiring the fragment serial number of each video fragment and the target frame identification of the video frame to which the video fragment belongs;
according to the fragment serial numbers, video fragments which are sent by different transmission channels and belong to the same frame identifier are added to the same preset fragment set;
and simultaneously packaging the video fragments of the fragment set to generate a video frame.
Optionally, before adding video segments belonging to the same frame identifier and sent by different transmission channels to the same preset segment set according to the segment sequence numbers, the method further includes:
if the preset fragment set is an empty set, acquiring a first timestamp of a first video fragment added to the fragment set, taking a time point corresponding to the first timestamp as a package start time of the fragment set, and taking the target frame identifier as a set identifier of the fragment set.
Optionally, before adding video segments belonging to the same frame identifier and sent by different transmission channels to the same preset segment set according to the segment sequence numbers, the method further includes:
if the preset fragment set is a non-empty set, acquiring a set identifier corresponding to the fragment set, and determining the time difference between the current video fragment and the fragment set;
if the target frame identification and the set identification belong to different frame identifications and the time difference is larger than a first preset frame interval, removing historical video fragments of the fragment set;
and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking the target frame identifier as the set identifier of the fragment set.
Optionally, the method further comprises:
if the target frame identification and the set identification belong to the same frame identification and the time difference is greater than a second preset frame interval, removing the historical video fragment of the fragment set;
and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking the target frame identifier as the set identifier of the fragment set.
Optionally, the determining a time difference between the current video slice and the slice set includes:
acquiring a second timestamp of the current video fragment and the package starting time of the fragment set;
and calculating the time difference between the video fragment and the fragment set by adopting the second time stamp of the current video fragment.
Optionally, the adding, according to the fragment sequence number, video fragments belonging to the same frame identifier and sent by different transmission channels to the same preset fragment set includes:
and adding the video fragments which belong to the same frame identifier and are sent by different transmission channels to the fragment set corresponding to the frame identifier according to the sequence of the sequence numbers of the fragments.
Optionally, after the video segments belonging to the same frame identifier and sent by different transmission channels are added to the same preset segment set according to the segment sequence numbers, the method further includes:
and if the video fragments with the same fragment sequence number exist in the same fragment set, selecting one of the video fragments with the same fragment sequence number for discarding.
Optionally, the simultaneously packaging the video slices of the slice set to generate a video frame includes:
acquiring fragment marks and fragment pointer information of video fragments in each fragment set;
and if the fragment mark of the video fragment contains the tail mark and the fragment pointer information points to the last video fragment, packaging the video fragments in the fragment set to generate a corresponding video frame.
Optionally, the simultaneously packaging the video slices of the slice set to generate a video frame includes:
acquiring fragment marks of the video fragments in the fragment sets;
if the fragment mark comprises a head mark and an end mark, calculating a difference value between a maximum fragment serial number and a minimum fragment serial number in the video fragments;
and if the difference value is the same as the number of the video fragments in the fragment set, grouping each video fragment in the fragment set to generate a video frame corresponding to the fragment set.
Optionally, after the video segments of the segment sets are grouped at the same time and a video frame is generated, the method further includes:
acquiring a target fragment serial number of the last video fragment in the fragment set, and taking the target fragment serial number as a comparison fragment serial number of the fragment set;
and clearing the video fragments in the fragment set, and returning to the step of receiving the video fragments sent by each transmission channel.
The embodiment of the invention also discloses a processing device of the multi-channel video data, which comprises:
the video fragment acquisition module is used for acquiring video fragments simultaneously sent by multiple transmission channels;
the target frame identification acquisition module is used for acquiring the fragment serial number of each video fragment and the target frame identification of the video frame to which the video fragment belongs;
the video fragment processing module is used for adding video fragments which belong to the same frame identifier and are sent by different transmission channels to the same preset fragment set according to the fragment serial numbers;
and the video frame generating module is used for packaging the video fragments of the fragment set at the same time to generate a video frame.
Optionally, the apparatus further comprises:
and the set identifier determining module is used for acquiring a first time stamp of a video slice added to the slice set if a preset slice set is an empty set, taking a time point corresponding to the first time stamp as the package starting time of the slice set, and taking the target frame identifier as the set identifier of the slice set.
Optionally, the apparatus further comprises:
the time difference determining module is used for acquiring a set identifier corresponding to a fragment set and determining the time difference between a current video fragment and the fragment set if the preset fragment set is a non-empty set;
the video fragment clearing module is used for clearing the historical video fragments of the fragment set if the target frame identification and the set identification belong to different frame identifications and the time difference is greater than a first preset frame interval;
and the set identifier determining module is used for acquiring a first timestamp of a first video segment added to the segment set, taking a time point corresponding to the first timestamp as a group package starting time of the segment set, and taking the target frame identifier as a set identifier of the segment set.
Optionally, the apparatus further comprises:
the video fragment clearing module is used for clearing the historical video fragments of the fragment set if the target frame identifier and the set identifier belong to the same frame identifier and the time difference is greater than a second preset frame interval;
and the set identifier determining module is used for acquiring a first timestamp of a first video segment added to the segment set, taking a time point corresponding to the first timestamp as a group package starting time of the segment set, and taking the target frame identifier as a set identifier of the segment set.
Optionally, the time difference determining module is specifically configured to:
acquiring a second timestamp of the current video fragment and the package starting time of the fragment set;
and calculating the time difference between the video fragment and the fragment set by adopting the second time stamp of the current video fragment.
Optionally, the video slicing processing module is specifically configured to:
and adding the video fragments which belong to the same frame identifier and are sent by different transmission channels to the fragment set corresponding to the frame identifier according to the sequence of the sequence numbers of the fragments.
Optionally, the apparatus further comprises:
and the video fragment discarding module is used for selecting one of the video fragments with the same fragment sequence number to discard if the video fragments with the same fragment sequence number exist in the same fragment set.
Optionally, the video frame generation module includes:
the information acquisition submodule is used for acquiring the fragment marks of the video fragments in each fragment set and the fragment pointer information;
and the video frame generation sub-module is used for packing the video fragments in the fragment set to generate a corresponding video frame if the fragment marks of the video fragments comprise the tail mark and the fragment pointer information points to the last video fragment.
Optionally, the video frame generation module includes:
a fragment mark acquisition sub-module, configured to acquire a fragment mark of each video fragment in each fragment set;
a difference value calculating submodule, configured to calculate a difference value between a maximum fragment sequence number and a minimum fragment sequence number in the video fragments if the fragment flag includes a head flag and a tail flag;
and the video frame generation sub-module is used for packaging each video fragment in the fragment set to generate a video frame corresponding to the fragment set if the difference value is the same as the number of the video fragments in the fragment set.
Optionally, the apparatus further comprises:
the fragment serial number determining module is used for acquiring a target fragment serial number of the last video fragment in the fragment set and taking the target fragment serial number as a comparison fragment serial number of the fragment set;
and the video fragment clearing module is used for clearing the video fragments in the fragment set and returning to the step of receiving the video fragments sent by each transmission channel.
The embodiment of the invention also discloses an electronic device, which comprises:
one or more processors; and
one or more machine-readable media having instructions stored thereon, which when executed by the one or more processors, cause the electronic device to perform the method as described above.
Embodiments of the present invention also disclose a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method as described above.
The embodiment of the invention has the following advantages:
in the embodiment of the present invention, in the process of processing video data, especially in the process of remote transmission of video data, video fragments simultaneously sent by multiple transmission channels may be obtained, and fragment serial numbers of the video fragments and target frame identifiers of the video frames that belong to the video fragments are obtained, then video fragments that belong to the same frame identifier and are sent by different transmission channels are added to the same preset fragment set according to the fragment serial numbers, and then video fragments of a plurality of fragment sets are packaged at the same time to generate a video frame.
Drawings
Fig. 1 is a flowchart illustrating steps of a method for processing multiple paths of video data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a communication architecture provided in an embodiment of the present invention;
fig. 3 is a block diagram of a processing apparatus for multiple paths of video data according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
As an example, with the rapid development of the 5G mobile communication network, the characteristics of high speed, low delay and large connection are well suited for the application ground based on the 5G network remote data transmission. Such as remote driving of the vehicle, remote control of the equipment, remote monitoring, etc. In the process of remote transmission of video data, a sending end can transmit collected video data to a receiving end through a communication network, the video data are processed by the receiving end, in the transmission process, the data volume of the video data to be transmitted is large due to real-time change of the video data, in addition, in the data transmission process, on one hand, the stability of the network cannot be ensured, and on the other hand, when the receiving end obtains the video data and packages the video data due to the conditions of data packet loss, time delay and the like, the application of the data remote transmission is influenced due to the problems of incomplete data, poor transmission real-time performance and reliability and the like.
In view of the above, one of the core invention points of the embodiments of the present invention is that in a process of processing video data, especially in a process of remote transmission of video data, video fragments simultaneously transmitted through multiple transmission channels are obtained, and a fragment sequence number of each video fragment and a target frame identifier of a video frame to which the video fragment belongs are obtained, then video fragments belonging to the same frame identifier and transmitted through different transmission channels are added to the same preset fragment set according to the fragment sequence numbers, and then video fragments of a plurality of fragment sets are simultaneously packaged to generate a video frame.
Specifically, referring to fig. 1, a flowchart illustrating steps of a method for processing multiple paths of video data according to an embodiment of the present invention is shown, which may specifically include the following steps:
step 101, acquiring video fragments simultaneously transmitted by multiple transmission channels;
in the embodiment of the present invention, referring to fig. 2, a schematic communication diagram of an apparatus provided in the embodiment of the present invention is shown, where a collection device may be a terminal apparatus for collecting video data; the transmission device can be a 5G communication module which is arranged in the acquisition device and used for connecting the acquisition device with an external network, on one hand, the transmission device is in communication connection with the acquisition device of the acquisition device to realize internal communication of the acquisition device, and on the other hand, the transmission device is connected with an access network, the Internet and the like of an operator through a 5G air interface so that the acquisition device transmits data to a receiving end; the group packaging device can be cloud processing equipment, can process the data sent by the acquisition device, and can restore the scene corresponding to the video data so as to make a corresponding decision according to the scene.
Optionally, different transmission devices may be configured with a Destination Network Address Translation (DNAT) rule, so as to implement DNAT Translation on a packet with an IP Address as a Destination Address, so as to translate the packet to an IP Address corresponding to the group packaging device, so that when a multicast packet sent by the acquisition device reaches the transmission device, the multicast packet may be transmitted to the group packaging device by a plurality of independent communication channels, so that the group packaging device performs corresponding processing. It should be noted that the transmission device may also be an independent gateway located outside the acquisition device, and the acquisition device may establish a communication connection with an external transmission device through the acquisition device to access a network outside the acquisition device, which is not limited in the present invention.
In specific implementation, the acquisition device can acquire corresponding image data through the camera to obtain corresponding video frames, then slices each video frame to generate at least one video slice corresponding to each video frame, and slices the video frames, so that the size of data transmission can be reduced, and the stability and reliability of data transmission can be improved. Then the acquisition device sends the video fragments to the packaging device through different transmission channels through a plurality of different transmission devices so that the packaging device can perform corresponding video data packaging processing.
102, acquiring the fragment serial number of each video fragment and the target frame identifier of the video frame to which the video fragment belongs;
in a specific implementation, after receiving the video segments transmitted by the multi-channel transmission channel, the group packaging device can obtain segment serial numbers of the video segments and target frame identifiers of video frames to which the video segments belong.
In an example, the acquisition device may include a central processing unit, at least one camera, a hardware encoder, and the like, and the central processing unit may receive video frames transmitted by the respective cameras in real time, perform H265 encoding on each video frame through the hardware encoder, and then fragment the video frames to meet requirements of UDP (User data Protocol) data transmission size. For a video slice, it may include fields such as a camera ID, a timestamp, a slice sequence number, a frame identification, a slice flag, and a video data length, and video slices from the same video frame may have the same frame identification.
The camera ID can be used for distinguishing different cameras corresponding to the acquisition device; the timestamp can be that when each video fragment is transmitted to the central processing unit, the central processing unit records corresponding time, the unit can be millisecond level, and the timestamp can be used for monitoring and counting the video fragments; the frame identifier (vFrameID) can be the only identifier of a video frame, for the same camera, every time a video frame is sent, the frame identifier is cumulatively added with 1 to realize the monitoring statistics of the video frame, and the time interval between two adjacent frames can be 5 ms; the fragment serial number can be the arrangement serial number of the video fragments, and for the same camera, the central processing unit cumulatively adds 1 to each video fragment generated by the central processing unit, so that the monitoring statistics of the video fragments is realized; the slice flag may be used to distinguish that a video slice belongs to the head, middle and end slices of a video frame, for example, "0" indicates no slice, i.e. one video frame is one video slice; "1" indicates that the video slice is the header slice of the video frame, i.e. the first video slice; "2" indicates that the video slice is the middle slice of the video frame; "3" indicates that the video slice is the last slice of the video frame, i.e. the last video slice, and in a general case, one video frame may be sliced into a header slice, at least one middle slice, an end slice, and the like.
103, adding video fragments which belong to the same frame identifier and are sent by different transmission channels to the same preset fragment set according to the fragment serial numbers;
after the grouping device acquires the fragment serial number of each video fragment and the target frame identifier of the video frame, the video fragments which belong to the same frame identifier and are sent by different transmission channels can be added to the same preset fragment set according to the fragment serial numbers, meanwhile, the video fragments of a plurality of fragment sets are grouped to generate video frames, then a plurality of video frames are adopted, the vehicle environment corresponding to the acquisition device is output, and a control instruction aiming at the vehicle environment is determined.
Optionally, for the video fragment, since the same piece of video data is transmitted by each transmission channel, the packaging device may receive the redundant video data based on the multiple transmission channels, so that the packaging device may package the redundant video data and restore the video frame sent by the acquisition device. In the process of packaging the video fragments, the packaging device needs to perform de-duplication processing on redundant video fragments, including video fragments that arrive after the video frame is packaged and video fragments that arrive before the video frame is packaged, and can perform corresponding video fragment processing according to specific conditions to restore corresponding video frames.
In the process of packaging the video fragments, the packaging device may preset a plurality of fragment sets, where the same fragment set is used for storing the video fragments of the same video frame, and the fragment set may be a buffer queue preset in the packaging device. Specifically, due to the existence of UDP transmission disorder, the last fragment of the previous video frame may arrive at the packaging device later than the head fragment of the next video frame, and a data transmission delay situation occurs, so that a plurality of fragment sets are set in the receiving and processing process for the same camera data, and a plurality of video frames are packaged at the same time. Optionally, 3 fragment sets may be set, allowing a packaging process of 3 video frames to be performed simultaneously, for a certain video fragment, a frame identifier modulo 3, that is, vFrameID% 3, may be used to determine which fragment set the video fragment should select for packaging, for example, when the frame identifier is 1, it indicates that the video fragment belongs to the first frame video frame, and then the video fragment may be stored in fragment set (i) for video packaging; when the frame mark is 2, the video fragment belongs to a second frame video frame, and the video fragment can be stored in a fragment set II to perform video packaging; when the frame mark is 3, the video fragment belongs to a third frame video frame and the like, so that a plurality of video frames are simultaneously packaged through a plurality of fragment sets, and the packaging efficiency of the video frames is effectively improved.
It should be noted that, in the process of packaging video frames, the packaging device may package video frames of the same camera at the same time through a plurality of fragment sets, and the process may be referred to in the video frame packaging manner corresponding to each camera, which is not described herein again, and the packaging device will be exemplarily described below for the video frame packaging process in one fragment set.
Before storing the video slices, the group packaging device may determine a slice set to be stored to determine whether residual data exists in the slice set to be stored, specifically, if the slice set to be stored is an empty set, obtain a first timestamp of a first video slice added to the slice set, use a time point corresponding to the first timestamp as a group packaging start time of the slice set, and use a target frame identifier as a set identifier of the slice set. If the preset fragment set is a non-empty set, acquiring a set identifier corresponding to the fragment set, and determining the time difference between the current video fragment and the fragment set; if the target frame identification and the set identification belong to different frame identifications and the time difference is greater than a first preset frame interval, removing historical video fragments of the fragment set; and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking a target frame identifier as the set identifier of the fragment set. If the target frame identification and the set identification belong to the same frame identification and the time difference is greater than a second preset frame interval, removing the historical video fragments of the fragment set; and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking a target frame identifier as the set identifier of the fragment set.
For a fragment set, a set identifier of the fragment set is a frame identifier corresponding to a video frame to which a currently stored video fragment belongs, and if the fragment set is an empty set, the fragment set indicates that no video fragment is stored, a frame identifier to which a first video fragment stored in the fragment set belongs can be recorded, the frame identifier is used as a set identifier vFrameID of the fragment set, and a timestamp of the video fragment is recorded and used as a package start time t _ start of the fragment set; if the fragment set is a non-empty set, video fragments belonging to the same video frame (namely vFrameID) as the current video fragment may be stored in the fragment set, and video fragments of a video frame of a certain previous unfinished group may also be stored and belong to residual data, so that if the frame identifiers are different, a second timestamp of the current video fragment and the group start time of the fragment set may be obtained, the time difference between the second timestamp and the group start time of the fragment set is calculated, and if the time difference is greater than a first preset frame interval (the first preset frame interval may be 3 frame intervals and corresponds to the number of the fragment sets), the video fragments in the fragment set may be determined as residual data, the residual data is cleared, the set identifier and the group start time of the fragment set are updated, and the current video fragment is stored; if the frame identifications are the same, the corresponding time difference can be calculated, if the time difference is larger than a second preset frame interval (the second preset frame interval can be 256 frame intervals and represents a loop), the video fragments in the fragment set can be judged to be residual data, the residual data is cleared, the set identification and the package starting time of the fragment set are updated, and the current video fragments are stored.
In the process of grouping the fragment sets, video fragments belonging to the same frame identifier and sent by different transmission channels can be added to the fragment set corresponding to the frame identifier according to the fragment numbers of the video fragments, for example, for a transmission channel (i) transmitting the video fragments A1, A2, A3 …, a transmission channel (ii) transmitting the video fragments B1, B2, B3 …, etc., the same video frame data is transmitted by the two, for the same fragment set, video fragments such as A1B1A2B2A3B3 …, etc. can be stored in sequence according to the sequence of fragment sequence numbers from small to large, optionally, only one video fragment is stored in the fragment set, for example, for the video fragment A1 and the video fragment B1, the subsequent video fragment can be discarded according to the sequence of the two, so that the redundancy of the video fragments can be utilized, and the integrity of data in the remote control process can be effectively ensured, meanwhile, redundant data are processed in time, and the expense of a storage space is reduced.
And step 104, packaging the video fragments of the fragment set simultaneously to generate a video frame.
In the embodiment of the present invention, the fragment sets may perform packet processing on the video fragments in a linked list or array manner, if the linked list manner is adopted, the packet packaging device may obtain the fragment marks of the video fragments in each fragment set and the fragment pointer information, and if the fragment marks of the video fragments include the end mark and the fragment pointer information points to the last video fragment, indicating that the video fragments corresponding to the video frames have been completely cached, the video fragments in the fragment sets are packaged to generate corresponding video frames. If the form of the array is adopted, the packaging device can obtain the fragment marks of each video fragment in each fragment set, and if the fragment marks comprise a head mark and a tail mark, the difference value between the maximum fragment serial number and the minimum fragment serial number in the video fragments is calculated; and if the difference value is the same as the number of the video fragments in the fragment set and indicates that the video fragments corresponding to the video frames are completely cached, packaging each video fragment in the fragment set to generate the video frames corresponding to the fragment set.
In addition, the packaging device can acquire the target fragment serial number of the last video fragment in the fragment set after packaging all the video fragments in the fragment set, and the target fragment serial number is used as the comparison fragment serial number of the fragment set, the video fragments in the fragment set are cleared, and returning to the step of receiving the video fragments sent by each transmission channel, continuously performing packet packaging processing on the next video frame, thereby, the acquisition device is in communication connection with at least two transmission channels, the acquisition device can fragment the video frames and transmit the video frames to the packaging device through the multipath channel, on one hand, the stability of the network is ensured through a plurality of transmission channels, on the other hand, the video frames are processed through the modes of fragmentation, multipath transmission, packaging and the like, under the condition of reducing the transmission data unit, the real-time performance and the reliability of data stream transmission are ensured.
In an example, the fragment set may buffer the video fragments in a linked list manner, and for the fragment set in the linked list manner, a ready pointer may be set to identify that a sequence from a first video fragment (linked list head) to the pointer position is already complete, that is, if a fragment sequence number of a newly arrived video fragment is not greater than a fragment sequence number of a position pointed by the ready pointer, it indicates that the newly arrived video fragment belongs to a duplicate fragment, and the newly arrived video fragment is directly discarded. Based on the linked list type video slice receiving process, the general situation may be that the video slices arrive in the ideal A1B1A2B2A3B3 … manner, i.e. ordered and adjacent repeats, so if not dropped quickly, each attempt to insert and check a repeat always takes precedence at the tail of the linked list before using the way of traversing from the beginning. And discarding when the duplication is detected, otherwise, inserting into the linked list from small to large according to the sequence number. In particular, during the process of receiving a video slice, if a header slice (slice flag 1) has been received, and the sequence number of the fragment currently inserted into the linked list is just connected with the sequence number of the previous node, the slave pointer is moved backward from the current position and checked, until a node with a non-consecutive sequence number is encountered, when the last slice (slice marked 3) has been received, and the ready pointer points to the last video slice, indicating that the video slices in the set of slices have been received completely, and that packaging of video frames can be performed, generating corresponding video frames, thereby on one hand, the stability of the network is ensured through a plurality of transmission channels, on the other hand, the video data is processed through the modes of fragmentation, multiplex transmission, package packaging and the like, under the condition of reducing the transmission data unit, the real-time performance and the reliability of data stream transmission are ensured.
In another example, the fragment set may cache video fragments in an array manner, and then, for a fragment set in an array form, an array length may be set according to a maximum fragment number of a video frame, and a constraint condition corresponding to the array length may be determined, where the constraint condition may be used to add a first received video fragment to a corresponding position in the array, and then, for a subsequent video fragment, a position of the video fragment in the array is determined according to a fragment sequence number of the video fragment, the constraint condition, and the array length, and is added to the corresponding position, so as to implement storage of the video fragment. If the video fragments are stored in the array, the video fragments can be determined to be repeated video fragments, and the repeated video fragments are discarded. For example, assuming that the maximum number of slices of a video frame is 100, the array set may be expressed by an array with a length of 200, and its index may be 0-199, where 100 may be a constraint condition, and when a first video slice is received, whether it is a header slice, a middle slice, or an end slice, it may be directly stored in a position with an index of 100 during the process of receiving the video slices. When a subsequent video fragment is received, which position in the array the video fragment should be stored in can be determined according to the offset between the subsequent video fragment and the fragment sequence number of the first video fragment, that is, the storage position of the current video fragment can be determined by "the fragment sequence number of the current video fragment +100 — the fragment sequence number of the first video fragment". When detecting that the head mark and the tail mark exist in the video fragments in the array and the number of the video fragments stored in the array is equal to the difference value between the subscripts of the last video fragment and the subscript of the first video fragment, the video fragments in the fragment set are completely received, and the video frames can be packaged to generate corresponding video frames, so that on one hand, the stability of a network is ensured through a plurality of transmission channels, on the other hand, the video data is processed through modes of fragmentation, multipath transmission, packaging and the like, and on the condition of reducing a transmission data unit, the real-time performance and the reliability of data stream transmission are ensured.
It should be noted that the embodiments of the present invention include, but are not limited to, the above examples, and it is understood that, under the guidance of the idea of the present invention, those skilled in the art may also set the embodiments according to actual needs, and the present invention is not limited to these.
In the embodiment of the present invention, in the process of processing video data, especially in the process of remote transmission of video data, video fragments simultaneously sent by multiple transmission channels may be obtained, and fragment serial numbers of the video fragments and target frame identifiers of the video frames that belong to the video fragments are obtained, then video fragments that belong to the same frame identifier and are sent by different transmission channels are added to the same preset fragment set according to the fragment serial numbers, and then video fragments of a plurality of fragment sets are packaged at the same time to generate a video frame.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 3, a block diagram of a processing apparatus for multiple paths of video data according to an embodiment of the present invention is shown, where the video stream processing system includes an acquisition apparatus, at least two transmission channels, and a packet packaging apparatus that is respectively connected to the transmission channels in a communication manner, and the apparatus may specifically include:
a video fragment acquiring module 301, configured to acquire video fragments simultaneously sent by multiple transmission channels;
a target frame identifier obtaining module 302, configured to obtain a fragment serial number of each video fragment and a target frame identifier of a video frame to which the video fragment belongs;
the video fragment processing module 303 is configured to add video fragments belonging to the same frame identifier and sent by different transmission channels to the same preset fragment set according to the fragment sequence number;
a video frame generating module 304, configured to group the video segments of the segment sets at the same time, so as to generate a video frame.
In an optional embodiment of the invention, the apparatus further comprises:
and the set identifier determining module is used for acquiring a first time stamp of a video slice added to the slice set if a preset slice set is an empty set, taking a time point corresponding to the first time stamp as the package starting time of the slice set, and taking the target frame identifier as the set identifier of the slice set.
In an optional embodiment of the invention, the apparatus further comprises:
the time difference determining module is used for acquiring a set identifier corresponding to a fragment set and determining the time difference between a current video fragment and the fragment set if the preset fragment set is a non-empty set;
the video fragment clearing module is used for clearing the historical video fragments of the fragment set if the target frame identification and the set identification belong to different frame identifications and the time difference is greater than a first preset frame interval;
and the set identifier determining module is used for acquiring a first timestamp of a first video segment added to the segment set, taking a time point corresponding to the first timestamp as a group package starting time of the segment set, and taking the target frame identifier as a set identifier of the segment set.
In an optional embodiment of the invention, the apparatus further comprises:
the video fragment clearing module is used for clearing the historical video fragments of the fragment set if the target frame identifier and the set identifier belong to the same frame identifier and the time difference is greater than a second preset frame interval;
and the set identifier determining module is used for acquiring a first timestamp of a first video segment added to the segment set, taking a time point corresponding to the first timestamp as a group package starting time of the segment set, and taking the target frame identifier as a set identifier of the segment set.
In an optional embodiment of the present invention, the time difference determining module is specifically configured to:
acquiring a second timestamp of the current video fragment and the package starting time of the fragment set;
and calculating the time difference between the video fragment and the fragment set by adopting the second time stamp of the current video fragment.
In an optional embodiment of the present invention, the video slice processing module 303 is specifically configured to:
and adding the video fragments which belong to the same frame identifier and are sent by different transmission channels to the fragment set corresponding to the frame identifier according to the sequence of the sequence numbers of the fragments.
In an optional embodiment of the invention, the apparatus further comprises:
and the video fragment discarding module is used for selecting one of the video fragments with the same fragment sequence number to discard if the video fragments with the same fragment sequence number exist in the same fragment set.
In an optional embodiment of the present invention, the video frame generation module 304 comprises:
the information acquisition submodule is used for acquiring the fragment marks of the video fragments in each fragment set and the fragment pointer information;
and the video frame generation sub-module is used for packing the video fragments in the fragment set to generate a corresponding video frame if the fragment marks of the video fragments comprise the tail mark and the fragment pointer information points to the last video fragment.
In an optional embodiment of the present invention, the video frame generation module 304 comprises:
a fragment mark acquisition sub-module, configured to acquire a fragment mark of each video fragment in each fragment set;
a difference value calculating submodule, configured to calculate a difference value between a maximum fragment sequence number and a minimum fragment sequence number in the video fragments if the fragment flag includes a head flag and a tail flag;
and the video frame generation sub-module is used for packaging each video fragment in the fragment set to generate a video frame corresponding to the fragment set if the difference value is the same as the number of the video fragments in the fragment set.
In an optional embodiment of the invention, the apparatus further comprises:
the fragment serial number determining module is used for acquiring a target fragment serial number of the last video fragment in the fragment set and taking the target fragment serial number as a comparison fragment serial number of the fragment set;
and the video fragment clearing module is used for clearing the video fragments in the fragment set and returning to the step of receiving the video fragments sent by each transmission channel.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an electronic device, including:
one or more processors; and
one or more machine-readable media having instructions stored thereon, which when executed by the one or more processors, cause the electronic device to perform methods as described in embodiments of the invention.
Embodiments of the present invention also provide a computer-readable storage medium having stored thereon instructions, which, when executed by one or more processors, cause the processors to perform a method according to embodiments of the present invention.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, EEPROM, Flash, eMMC, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The present invention provides a method for processing multiple channels of video data, a device for processing multiple channels of video data, an electronic device, and a computer-readable storage medium, which are described in detail above, and specific examples are applied herein to illustrate the principles and embodiments of the present invention, and the description of the above embodiments is only used to help understand the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (13)

1. A method for processing multiple channels of video data, comprising:
acquiring video fragments simultaneously transmitted by multiple transmission channels;
acquiring the fragment serial number of each video fragment and the target frame identification of the video frame to which the video fragment belongs;
according to the fragment serial numbers, video fragments which are sent by different transmission channels and belong to the same frame identifier are added to the same preset fragment set;
and simultaneously packaging the video fragments of the fragment set to generate a video frame.
2. The method according to claim 1, wherein before adding video slices belonging to the same frame identifier and sent by different transmission channels to the same preset slice set according to the slice sequence number, the method further comprises:
if the preset fragment set is an empty set, acquiring a first timestamp of a first video fragment added to the fragment set, taking a time point corresponding to the first timestamp as a package start time of the fragment set, and taking the target frame identifier as a set identifier of the fragment set.
3. The method according to claim 2, wherein before adding video slices belonging to the same frame identifier and sent by different transmission channels to the same preset slice set according to the slice sequence number, the method further comprises:
if the preset fragment set is a non-empty set, acquiring a set identifier corresponding to the fragment set, and determining the time difference between the current video fragment and the fragment set;
if the target frame identification and the set identification belong to different frame identifications and the time difference is larger than a first preset frame interval, removing historical video fragments of the fragment set;
and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking the target frame identifier as the set identifier of the fragment set.
4. The method of claim 3, further comprising:
if the target frame identification and the set identification belong to the same frame identification and the time difference is greater than a second preset frame interval, removing the historical video fragment of the fragment set;
and acquiring a first time stamp of a first video fragment added to the fragment set, taking a time point corresponding to the first time stamp as the package starting time of the fragment set, and taking the target frame identifier as the set identifier of the fragment set.
5. The method of claim 4, wherein determining the time difference between the current video slice and the slice set comprises:
acquiring a second timestamp of the current video fragment and the package starting time of the fragment set;
and calculating the time difference between the video fragment and the fragment set by adopting the second time stamp of the current video fragment.
6. The method according to claim 2, 3, 4 or 5, wherein the adding video slices belonging to the same frame identifier and sent by different transmission channels to the same preset slice set according to the slice sequence number comprises:
and adding the video fragments which belong to the same frame identifier and are sent by different transmission channels to the fragment set corresponding to the frame identifier according to the sequence of the sequence numbers of the fragments.
7. The method according to claim 1, wherein after adding video slices belonging to the same frame identifier and sent by different transmission channels to the same preset slice set according to the slice sequence number, the method further comprises:
and if the video fragments with the same fragment sequence number exist in the same fragment set, selecting one of the video fragments with the same fragment sequence number for discarding.
8. The method of claim 1, wherein said simultaneously packing video slices of said plurality of slice sets to generate video frames comprises:
acquiring fragment marks and fragment pointer information of video fragments in each fragment set;
and if the fragment mark of the video fragment contains the tail mark and the fragment pointer information points to the last video fragment, packaging the video fragments in the fragment set to generate a corresponding video frame.
9. The method of claim 1, wherein said simultaneously packing video slices of said plurality of slice sets to generate video frames comprises:
acquiring fragment marks of the video fragments in the fragment sets;
if the fragment mark comprises a head mark and an end mark, calculating a difference value between a maximum fragment serial number and a minimum fragment serial number in the video fragments;
and if the difference value is the same as the number of the video fragments in the fragment set, grouping each video fragment in the fragment set to generate a video frame corresponding to the fragment set.
10. The method of claim 1, wherein after the video slices of the slice set are simultaneously grouped to generate video frames, the method further comprises:
acquiring a target fragment serial number of the last video fragment in the fragment set, and taking the target fragment serial number as a comparison fragment serial number of the fragment set;
and clearing the video fragments in the fragment set, and returning to the step of receiving the video fragments sent by each transmission channel.
11. An apparatus for processing multiple video data, comprising:
the video fragment acquisition module is used for acquiring video fragments simultaneously sent by multiple transmission channels;
the target frame identification acquisition module is used for acquiring the fragment serial number of each video fragment and the target frame identification of the video frame to which the video fragment belongs;
the video fragment processing module is used for adding video fragments which belong to the same frame identifier and are sent by different transmission channels to the same preset fragment set according to the fragment serial numbers;
and the video frame generating module is used for packaging the video fragments of the fragment set at the same time to generate a video frame.
12. An electronic device, comprising:
one or more processors; and
one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-10.
13. A computer-readable storage medium having stored thereon instructions, which when executed by one or more processors, cause the processors to perform the method of any one of claims 1-10.
CN202110496334.2A 2021-05-07 2021-05-07 Method and device for processing multi-channel video data, electronic equipment and medium Pending CN113259715A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110496334.2A CN113259715A (en) 2021-05-07 2021-05-07 Method and device for processing multi-channel video data, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110496334.2A CN113259715A (en) 2021-05-07 2021-05-07 Method and device for processing multi-channel video data, electronic equipment and medium

Publications (1)

Publication Number Publication Date
CN113259715A true CN113259715A (en) 2021-08-13

Family

ID=77223926

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110496334.2A Pending CN113259715A (en) 2021-05-07 2021-05-07 Method and device for processing multi-channel video data, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN113259715A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113765786A (en) * 2021-09-23 2021-12-07 上海高骏科技有限公司 Multilink aggregation transmission method and device, computer equipment and storage medium thereof
CN114697301A (en) * 2022-04-11 2022-07-01 北京国基科技股份有限公司 Video stream transmission method and device
WO2023236666A1 (en) * 2022-06-08 2023-12-14 中兴通讯股份有限公司 Media information processing method and apparatus, and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101257441A (en) * 2008-02-29 2008-09-03 华为技术有限公司 Method, equipment and system for forwarding message as well as equipment for receiving message
CN101286945A (en) * 2008-05-22 2008-10-15 北京星网锐捷网络技术有限公司 Method and apparatus for processing of data fragmentation
CN101662461A (en) * 2008-08-27 2010-03-03 华为技术有限公司 Method, device and system for recombining multilink protocol fragment data
CN107071358A (en) * 2017-04-19 2017-08-18 中国电子科技集团公司电子科学研究院 Panorama live broadcast system under video-splicing server and mobile status
CN107342955A (en) * 2017-06-29 2017-11-10 迈普通信技术股份有限公司 A kind of data message sharding method and device, recombination method and device
CN108696773A (en) * 2017-04-11 2018-10-23 上海谦问万答吧云计算科技有限公司 A kind of transmission method and device of real-time video
CN109246036A (en) * 2017-07-10 2019-01-18 深圳市中兴微电子技术有限公司 A kind of method and apparatus handling fragment message
CN109257143A (en) * 2018-09-07 2019-01-22 武汉虹信通信技术有限责任公司 Method for there is sliced transmission data packet in the network transmission protocol of length limitation
CN111970215A (en) * 2020-10-21 2020-11-20 武汉中科通达高新技术股份有限公司 Data packet management method and device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101257441A (en) * 2008-02-29 2008-09-03 华为技术有限公司 Method, equipment and system for forwarding message as well as equipment for receiving message
CN101286945A (en) * 2008-05-22 2008-10-15 北京星网锐捷网络技术有限公司 Method and apparatus for processing of data fragmentation
CN101662461A (en) * 2008-08-27 2010-03-03 华为技术有限公司 Method, device and system for recombining multilink protocol fragment data
CN108696773A (en) * 2017-04-11 2018-10-23 上海谦问万答吧云计算科技有限公司 A kind of transmission method and device of real-time video
CN107071358A (en) * 2017-04-19 2017-08-18 中国电子科技集团公司电子科学研究院 Panorama live broadcast system under video-splicing server and mobile status
CN107342955A (en) * 2017-06-29 2017-11-10 迈普通信技术股份有限公司 A kind of data message sharding method and device, recombination method and device
CN109246036A (en) * 2017-07-10 2019-01-18 深圳市中兴微电子技术有限公司 A kind of method and apparatus handling fragment message
CN109257143A (en) * 2018-09-07 2019-01-22 武汉虹信通信技术有限责任公司 Method for there is sliced transmission data packet in the network transmission protocol of length limitation
CN111970215A (en) * 2020-10-21 2020-11-20 武汉中科通达高新技术股份有限公司 Data packet management method and device

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113765786A (en) * 2021-09-23 2021-12-07 上海高骏科技有限公司 Multilink aggregation transmission method and device, computer equipment and storage medium thereof
CN114697301A (en) * 2022-04-11 2022-07-01 北京国基科技股份有限公司 Video stream transmission method and device
CN114697301B (en) * 2022-04-11 2023-10-20 北京国基科技股份有限公司 Video streaming transmission method and device
WO2023236666A1 (en) * 2022-06-08 2023-12-14 中兴通讯股份有限公司 Media information processing method and apparatus, and storage medium

Similar Documents

Publication Publication Date Title
CN113259715A (en) Method and device for processing multi-channel video data, electronic equipment and medium
US11381625B2 (en) Apparatus and method for transmitting multimedia data in hybrid network
CN109691039B (en) Message transmission method and device
CN113994642A (en) On-line performance monitoring
EP2661028B1 (en) Duplicate Packet Suppression
CN113055127B (en) Data message duplicate removal and transmission method, electronic equipment and storage medium
US20130258843A1 (en) Network system and apparatis
CN109257143B (en) Method for fragmenting data packets for transmission in network transmission protocol with length limitation
CN109391627B (en) Method for identifying encrypted transmission YouTube DASH video of TLS protocol
US20030118047A1 (en) Fibre channel frame batching for IP transmission
JP2001285400A (en) Correcting method of traffic statistics information
CN110505123B (en) Packet loss rate calculation method, server and computer-readable storage medium
JP6631232B2 (en) System and method for determining routing information
KR101039550B1 (en) Method for calculating transfer rate and method for setting bandwidth by using the same
CN113141415B (en) Remote driving system and method for vehicle, electronic device and storage medium
CN106954103B (en) Transmission system and transmission method for downlink service in 100G EPON system
CN114157730A (en) Message duplicate removal method and device
CN109361613B (en) Communication method of transformer substation and server
CN108737287B (en) Repeated packet identification method and device and convergence and distribution equipment
CN111447645A (en) Method for sensing network state in real time under wireless sensor network scene
CN115348332B (en) Method for reorganizing HTTP data stream session in signaling analysis scene
CN116319468B (en) Network telemetry method, device, switch, network, electronic equipment and medium
CN115484202B (en) INT-based lightweight path detection method
JP2010141536A (en) Transmission terminal, reception terminal, and transmission system used for unidirectional transmission path
CN115967841A (en) Method and device for detecting media transmission quality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210813