CN116506412A - Video real-time encrypted transmission method and system compatible with H.264 and H.265 encoding standards - Google Patents

Abstract

The application belongs to the technical field of information encryption, and particularly relates to a video real-time transmission method and system compatible with H.264 and H.265 coding standards, wherein the video real-time encryption transmission method comprises the following steps: a streaming request; RTSP verification, wherein the verification process also comprises the steps of analyzing and encrypting the video stream and transmitting the coding parameter set to a video playing server; after passing the RTSP verification, the relevant video parameters in each frame of the video stream are removed, and only the reserved video data part is transmitted; the video playing server identifies and decrypts the encrypted encoding parameter set and restores the encoding parameter set plaintext. The video real-time encryption transmission method and system can protect the safety of video content on the premise of not changing the overall structure and transmission protocol of the video stream, are compatible with H.264 and H.265 coding standards, and in addition, because the parameter sets VPS, SPS and PPS are encrypted only once in the RTSP verification process, the ciphertext expansion rate is not more than 1%, the real-time transmission delay is lower, and the practicability is stronger.

Description

Video real-time encrypted transmission method and system compatible with H.264 and H.265 encoding standards

technical field

The application belongs to the technical field of information encryption, and specifically relates to a video real-time encrypted transmission method and system compatible with H.264 and H.265 encoding standards.

Background technique

With the continuous development of the Internet and communication technology, the sharing of multimedia information has become more convenient. Video applications such as video conferencing, video surveillance, and perimeter security have gradually become part of daily life. The openness of the Internet also exposes video information to the public environment, making it vulnerable to interception, theft, or tampering attacks by malicious organizations or hackers, which has caused great negative impacts and economic losses on industries and organizations. Therefore, how to ensure the security of video data in an open Internet environment is particularly important.

In addition, with the continuous development of high-definition video, video stitching and panoramic video technology, multiplexed and massive video data also brings new challenges to data transmission and storage. In order to deal with this problem, video coding standards have changed from the earliest H.261 and MPEG series coding standards to the epoch-making H.264. With the further development of IC chip technology, H.265 will gradually replace H.264 as the The mainstream standard of the future. Compared with H.264, the H.265 coding standard not only continues to use the original technology, but also improves and optimizes some technologies, increasing the video compression rate to about twice that of H.264, that is, transmitting the same quality video. Video requires half the bandwidth of before. Although H.265 will be the mainstream of future applications, a large part of the currently running systems still use the H.264 encoding standard. Therefore, it is of great significance to study how to be compatible with H.264 and H.265 coding standards under the premise of ensuring video data security.

One way to solve the above problem is to encrypt the video content itself. Because video information has the characteristics of large data volume and high correlation, in the field of real-time video communication, video often needs to be compressed before transmission to reduce the redundancy of video information, thereby reducing the amount of data to ensure real-time video transmission.

Video encryption schemes are mainly divided into three types: encryption before encoding, encryption during encoding, and encryption after encoding; among them, encryption after encoding is to directly perform encryption operations on the encoded video stream, because the encryption scheme is weakly related to the video encoding format. Therefore, it is currently widely used by major security vendors and video vendors.

However, most of the current encryption after encoding is full encryption, that is, all encoded video streams are encrypted. This method still needs to encrypt and process a large amount of video data, resulting in large delay overhead and bandwidth overhead.

Contents of the invention

In order to solve at least one technical problem in the prior art, the present application provides a video real-time encrypted transmission method and system compatible with H.264 and H.265 encoding standards.

In the first aspect, the present application discloses a video real-time encrypted transmission method compatible with H.264 and H.265 encoding standards, comprising the following steps:

Step 1: Make a streaming request between the video playback server and the video capture end;

Step 2, after the stream pull request is successful, RTSP verification is performed between the video playback server and the video capture end, and the verification process also includes parsing and encrypting the video stream, and analyzing and encrypting the obtained The encoding parameter set is transmitted to the video playback server; wherein, parsing the video stream includes:

Step 2.1, analyzing the video stream to determine its encoding standard, wherein the encoding standard includes H.264 and/or H.265;

Step 2.2, locating the predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

When the encoding standard is H.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

When the coding standard is H.265 or includes both H.264 and H.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS and a picture parameter set PPS;

Step 2.3, identifying the located predetermined parameter set as key content;

The encryption processing of the video stream includes:

Step 2.4, encrypting the key content identified in step 2.3 to obtain an encoding parameter set;

Step 3, after the RTSP verification is passed, in the video stream transmitted from the video shooting end to the video playback server, the relevant video parameters in each frame are removed, and only the video data part is reserved for transmission to the video playback server ;

Step 4: The video playing server identifies and decrypts the encrypted encoding parameter set, and recovers the plaintext of the encoding parameter set.

According to at least one embodiment of the present application, in the step 2.2, locating the predetermined parameter set in each of the network abstraction layer units NALU includes:

The determination of each parameter set in each network abstraction layer unit NALU and the positioning of the start and end positions of each parameter set.

According to at least one embodiment of the present application, the determination of the video parameter set VPS, the sequence parameter set SPS and the image parameter set PPS in each of the network abstraction layer units NALU includes the following steps:

Step 2.2.1, locate the network abstraction layer unit NALU by searching the start code in the video stream;

Step 2.2.2, then read the data type bit nal_unit_type of the network abstraction layer unit NALU header bit by bit, then convert its value into plastic and determine whether the corresponding parameter set is contained in the network abstraction layer unit NALU, wherein :

For the H.264 encoding standard:

When the reshaped value is 7, it is determined that the network abstraction layer unit NALU contains the sequence parameter set SPS;

When the reshaped value is 8, it is determined that the network abstraction layer unit NALU contains the image parameter set PPS;

For the H.265 encoding standard:

When the shaped value is 32, it is determined that the network abstraction layer unit NALU includes a video parameter set VPS;

When the shaped value is 33, it is determined that the network abstraction layer unit NALU contains a sequence parameter set SPS;

When the reshaped value is 34, it is determined that the network abstraction layer unit NALU includes a picture parameter set PPS.

According to at least one embodiment of the present application, the positioning of the start and end positions of each parameter set in each network abstraction layer unit NALU includes:

Step 2.2.3, skipping one or two bytes at the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set;

Step 2.2.4. Locate the next network abstraction layer unit NALU by searching the start code, so as to determine the end position of the current network abstraction layer unit NALU, that is, the end position of the corresponding parameter set.

According to at least one embodiment of the present application, in the step 2.4, encrypting the identified key content includes the following steps:

Step 2.4.1, using a symmetric block encryption algorithm to perform confidentiality encryption on the key content to obtain confidentiality-encrypted ciphertext data;

Step 2.4.2, use the key-based hash operation message authentication code algorithm to perform integrity encryption on the key content, and obtain the ciphertext data after integrity encryption;

Step 2.4.3, directly splicing the confidentiality-encrypted ciphertext data and the integrity-encrypted ciphertext data, and using the spliced ciphertext to replace the plaintext of the key content.

According to at least one embodiment of the present application, in the step 2.4, after encrypting the key content and before obtaining the encoding parameter set, it also includes performing packing processing on the ciphertext obtained after the encryption processing, Specifically include the following steps:

Step 2.4.4, packing the spliced ciphertext;

Step 2.4.5. Add a 0xffffffff check code at the end of the spliced ciphertext to mark the end of the ciphertext.

In the second aspect, the present application also discloses a video real-time encrypted transmission system compatible with H.264 and H.265 encoding standards, including a video playback server, a video capture terminal, and a video encryption client, wherein:

The video playback server is used to perform a streaming request with the video capture terminal, and performs RTSP verification with the video capture terminal after the stream pull request succeeds;

The video encryption client is used to analyze and encrypt the video stream in the RTSP verification process, and transmit the encoding parameter set obtained after the analysis and encryption to the video playback server; wherein, the video stream is performed Parsing includes the following steps:

Step 2.1, analyzing the video stream to determine its encoding standard, wherein the encoding standard includes H.264 and/or H.265;

Step 2.2, locating the predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

When the encoding standard is H.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

When the coding standard is H.265 or includes both H.264 and H.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS and a picture parameter set PPS;

Step 2.3, identifying the located predetermined parameter set as key content;

Described video flow is carried out encrypting and processing comprises the steps:

Step 2.4, encrypting the key content identified in step 2.3 to obtain an encoding parameter set;

Further, the video encryption client is also used to remove relevant video parameters in each frame from the video stream transmitted from the video shooting end to the video playback server after the RTSP verification is passed, and only keep the video The data part is transmitted to the video playing server;

Further, the video playing server is also used for identifying and decrypting the encrypted encoding parameter set, and recovering the plaintext of the encoding parameter set.

According to at least one embodiment of the present application, the video encryption client includes:

A first positioning module, configured to determine each parameter set in each of the network abstraction layer units NALU;

The second positioning module is configured to locate the start and end positions of each parameter set in each of the network abstraction layer units NALU.

According to at least one embodiment of the present application, the determination of each parameter set by the first positioning module includes:

Locate the network abstraction layer unit NALU by searching the start code in the video stream to be encrypted, then read the data type bit nal_unit_type of the network abstraction layer unit NALU header bit by bit, and then convert its value is shaping, where:

For the H.264 encoding standard:

When the value after shaping is 7, it is determined that the parameter set includes the sequence parameter set SPS; when the value after shaping is 8, it is determined that the parameter set includes the image parameter set PPS;

For the H.265 encoding standard:

When the value after shaping is 32, it is determined that the parameter set contains video parameter set VPS; when the value after shaping is 33, it is determined that the parameter set contains sequence parameter set SPS; when the value after shaping is 34, it is determined that the parameter set contains image parameter set PPS;

Further, the positioning of the start and end positions of each parameter set by the second positioning module includes:

Skip one or two bytes at the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set, and then locate the next network abstraction layer unit NALU by searching the start code to determine the current network abstraction layer The end position of the unit NALU, that is, the end position of the corresponding parameter set.

According to at least one embodiment of the present application, the video encryption client also includes:

The first encryption module is configured to perform confidentiality encryption on the key content through a symmetric block encryption algorithm to obtain confidentiality-encrypted ciphertext data;

The second encryption module is used to perform integrity encryption on the key content based on the hash operation message authentication code algorithm based on the key, and obtain the ciphertext data after the integrity encryption;

A splicing module, configured to directly splice the confidentiality-encrypted ciphertext data and the integrity-encrypted ciphertext data, and use the spliced ciphertext to replace the plaintext of the key content;

The packing module is used for packing the spliced ciphertext, and then adding a 0xffffffff check code at the end of the spliced ciphertext to mark the end of the ciphertext.

The present application at least has the following beneficial technical effects:

1) The video real-time encrypted transmission method and system compatible with the H.264 and H.265 coding standards of this application can protect the security of the video content without changing the overall structure of the video stream and the transmission protocol, and is compatible with the mainstream H. 264 and H.265 encoding standards;

In addition, this application only retains the encoding parameter set in the return RTSP 200OK step of RTSP verification, and then removes the video parameters in each frame during the video stream transmission process, and only retains the video data part to be transmitted to the video playback server, and finally in the video The playback server uses the decrypted encoding parameter set for decoding and playback, which can further save bandwidth resource consumption, and is better suitable for high-definition video, panoramic video and other video real-time transmission encryption scenarios that require high bandwidth resources;

2) In the video real-time encrypted transmission method and system compatible with H.264 and H.265 coding standards of the present application, because only once in the RTSP authentication process, encrypt the parameter set VPS, SPS and PPS, therefore make the ciphertext expansion rate not exceed 1%, which makes the real-time transmission delay lower and more practical.

Description of drawings

Fig. 1 is the block flow diagram of the video real-time encryption transmission method compatible with H.264 and H.265 encoding standards of the present application;

Fig. 2 is that the application is compatible with H.264 and H.265 encoding standard in the video real-time encrypted transmission method RTSP verification flow block diagram;

Fig. 3 is the flow diagram of analysis and encryption processing steps in the video real-time encrypted transmission method compatible with H.264 and H.265 encoding standards of the present application;

Fig. 4 is the structural block diagram of the video real-time encrypted transmission system compatible with H.264 and H.265 coding standards of the present application;

Fig. 5 is a composition diagram of positioning modules in the video real-time encryption transmission system compatible with H.264 and H.265 coding standards of the present application;

Fig. 6 is a composition diagram of an encryption module in the video real-time encrypted transmission system compatible with the H.264 and H.265 coding standards of the present application;

Fig. 7 is the NALU header format (frame format) parsing diagram of H.264;

Fig. 8 is the NALU header format (frame format) parsing diagram of H.265;

Fig. 9 is a diagram (table) of an H.265 code stream in an embodiment.

Detailed ways

In order to enable those skilled in the art to better understand the application, the technical terms mentioned in the application are first described:

1) Video coding is a method of compressing raw video data into code streams. In this application, H.264 and/or H.265 are used (that is, H.264 or H.265, or both H.264 and H.264 H.265) technology for encoding;

2) The code stream is composed of multiple network abstract layer units (Network Abstract Layer Unit, NALU). For example, the H.264 original code stream (also known as "naked stream") is composed of NALUs one by one, and each The NALUs are separated by startcode (start code), which is divided into two types: 0x000001 (3Byte) or 0x00000001 (4Byte);

3) The network abstraction layer unit NALU includes two parts: the header (Header) and the payload data (Payload Data). According to the different types of data carried, the NALU includes: a video coding layer network abstraction layer unit (Video CodeLayer NALU, VCLU ) and non-video coding layer network abstraction layer unit (non-Video Code Layer NALU, non-VCLU) two types;

Wherein, the non-VCLU is the NALU used to carry the parameter set. In this application, the parameter set includes: Video Parament Set (Video Paramenter Set, VPS), Sequence Parameter Set (Sequence Paramenter Set, SPS), Image Parameter Set (Picture Paramenter Set, Set, PPS) three types;

In addition, the video parameter set VPS is a parameter set unique to the H.265 coding standard, and the sequence parameter set SPS and the image parameter set PPS are common parameter sets of H.264 and H.265;

5) Video stream refers to the transmission of video data, for example, it can be processed as a stable and continuous stream through the network, that is, audio, video and other multimedia can be transmitted in real time on the Internet and Intranet without downloading The technology of playing in a waiting manner;

6) Video stream analysis refers to analyzing its basic unit NALU from the corresponding code stream, and can analyze the fields of the NALU header. For example, the analysis of the H.264 code stream first searches for 0x000001 and 0x00000001 from the code stream, and separates NALU, and then analyze the various fields of the NALU.

7) Plaintext refers to unencrypted text (or character strings). In a communication system, it may be a bit stream, such as text, bitmap, digitized voice, or digitized video image. A meaningful set of characters or bits, or a message that can be obtained through some public encoding standard.

In order to make the purpose, technical solution and advantages of the application more clear, the technical solution in the embodiment of the application will be described in more detail below in conjunction with the drawings in the embodiment of the application. The described embodiments are some, but not all, embodiments of the application. The embodiments described below by referring to the figures are exemplary, and are intended to explain the present application, and should not be construed as limiting the present application.

In the first aspect, as shown in Figure 1-3, the present application discloses a video real-time encrypted transmission method compatible with H.264 and H.265 encoding standards, including the following steps:

Step 1: A streaming request is made between the video playing server and the video shooting end (that is, the camera in FIG. 1 ).

It should be noted that in the field of plaintext video transmission, the streaming request, RTSP verification, and video streaming between the video playback server and the camera are relatively mature technologies, so no specific description of each step will be given here. .

Step 2: After the streaming request is successful, RTSP verification is performed between the video playback server and the video shooting end (see the process shown in Figure 2), and the verification process also includes parsing and encrypting the video stream (see Figure 3 process), and transmit the encoding parameter set obtained after parsing and encryption to the video playback server.

Among them, parsing the video stream includes:

Step 2.1, analyze the frequency stream to determine its encoding standard, wherein the encoding standard includes H.264 and/or H.265 (the default encoding standard other than H.265 in Figure 1 is currently more commonly used H .264).

Step 2.2, locate the predetermined parameter set in the network abstraction layer unit NALU of each (mentioned above, the original code stream is made up of a plurality of NALUs) in the video stream, wherein:

When the encoding standard is H.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

When the encoding standard is H.265 or includes both H.264 and H.265, the predetermined parameter set includes video parameter set VPS, sequence parameter set SPS and picture parameter set PPS.

Further, the positioning mentioned in this step includes the determination of each parameter set (i.e. video parameter set VPS, sequence parameter set SPS and image parameter set PPS) in each network abstraction layer unit NALU and the start and end positions of each parameter set positioning.

Specifically, the determination of the video parameter set VPS, the sequence parameter set SPS and the image parameter set PPS in each network abstraction layer unit NALU includes the following steps:

Step 2.2.1, locate the network abstraction layer unit NALU by searching the start code (0x000001) in the video stream.

Step 2.2.2, then read the data type bit nal_unit_type of the network abstraction layer unit NALU header bit by bit according to the coding standard, and then convert its value into shaping and then determine whether the network abstraction layer unit NALU contains the corresponding parameter set, wherein :

For the H.264 encoding standard:

When the reshaped value is 7, it is determined that the network abstraction layer unit NALU contains the sequence parameter set SPS;

When the reshaped value is 8, it is determined that the network abstraction layer unit NALU contains the image parameter set PPS.

For the H.265 encoding standard:

When the reshaped value is 32, it is determined that the network abstraction layer unit NALU contains the video parameter set VPS;

When the reshaped value is 33, it is determined that the sequence parameter set SPS is included in the network abstraction layer unit NALU;

When the reshaped value is 34, it is determined that the network abstraction layer unit NALU contains the image parameter set PPS.

Further, as shown in FIG. 7, the NALU header (Header) of H.264 only occupies one byte (that is, 1 byte, which is equal to 8 bits), wherein:

F: forbidden_zero_bit, which is the forbidden bit, occupying 1 bit, the forbidden bit should be 0, when the forbidden bit is 1, it means that there is an error in the NALU, and it needs to be discarded or corrected;

NRI: nal_ref_idc, which is an important indicator bit, occupies 2 bits, takes 00~11, and indicates the importance of this NALU; the larger the value, the more important the current NAL is, and it needs to be protected first; if the current NALU is a sequence parameter set or an image When parameter sets these important units, this syntax element must be greater than 0;

Type: nal_unit_type, which identifies the data type of the load in the NALU, accounting for 5 bits. The specific meanings of common nal_unit_type are shown in Table 1 below:

Table 1 - Common nal_unit_type meaning table in the NALU header of H.264

nal_unit_type Expressed content 0 Not defined 7 Sequence parameter set SPS 8 Image parameter set PPS twenty four Event Package 25-27 Time Pack 28-29 Fragmentation unit 30-31 reserved bit

Further, as shown in Figure 8, the NALU header of H.265 occupies two bytes, where:

F: forbidden_zero_bit, which is the forbidden bit, occupying 1 bit, the forbidden bit should be 0, when the forbidden bit is 1, it means that there is an error in the NALU, which needs to be discarded or corrected;

Type: nal_unit_type, occupying 6 bits, identifies the data type of the load in the NALU; the specific meaning of the common nal_unit_type is shown in Table 2 below;

LayerId: nuh_layer_id, occupying 6 bits, indicating the identifier of the layer to which the NALU belongs or the identifier of the layer to which the non-VCLU is applied;

TID: nuh_temporal_id_plus1, accounting for 3 bits, specifies the time identifier of the NALU;

Table 2 - Common nal_unit_type meaning table in the NALU header of H.265

Further, as shown in Figure 9, it is an example analysis of a H.265 code stream, where the start code in the rectangle a1 is the start code, the NALU header in the rectangle a2, and the nal_unit_type corresponding to "40 01" is 32 (VPS), the nal_unit_type corresponding to "42 01" is 33 (SPS), and the nal_unit_type corresponding to "44 01" is 34 (PPS).

Further, the positioning of the start and end positions of each parameter set in each network abstraction layer unit NALU in step 3 specifically includes:

Step 2.2.3. According to the recognized encoding standard, one or two bytes are skipped at the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set.

Step 2.2.4. Locate the next network abstraction layer unit NALU by searching the start code (0x000001), so as to determine the end position of the current network abstraction layer unit NALU, that is, the end position of the corresponding parameter set.

Step 2.3, identifying the located predetermined parameter set as key content.

Specifically, this step is to identify the parameter sets VPS, SPS and PPS located in step 2.2 as key content.

It should be noted that through the analysis of H.264 and H.265 encoding, it can be known that the information in the video stream is not evenly distributed, and some key content contains more information than other content. It plays a more important role in the decoding (that is, parsing) process. The reason for the division of important content will be briefly analyzed below:

The video parameter set VPS contains multiple syntax elements of the video coding sequence, which can be divided into three parts according to the content of the video parameter set: the first part is composed of syntax elements shared by sub-layers or operation points, mainly including the video parameter set identifier, The maximum number of layers of the video coding sequence, the maximum number of time-domain sub-layers of the video coding sequence, etc.; the second part is composed of information related to grades, layers and levels, mainly including the syntax element profile_tier_level(); profile_tier_level() will be based on the parameters The value is used to determine the relevant content of the grade, layer and level; the third part is composed of the specific operation point information of the video parameter set.

The content of the sequence parameter set SPS is mainly composed of the information shared by the images in the coded video sequence. According to the content of the sequence parameter set, it can be divided into four parts: the first part is composed of the syntax elements shared by the operation point, mainly including the video parameter set Reference identifier, sequence parameter set identifier, the maximum number of time-domain sub-layers contained in the video coding sequence, and restriction information for inter-frame prediction of the video coding sequence, where SPS will indicate the referenced video parameter set through the vps_id field, and use the sps_id field as The unique identifier enables it to be referenced by the image parameter set; the second part is composed of information related to grade, layer and level; the third part is composed of image format-related information, mainly including chroma sampling format, image resolution, and quantization depth , Time-domain sub-layer order mark; the fourth part is composed of coding parameter-related information, mainly including the size of the minimum brightness coding block and its difference with the maximum brightness coding block, the minimum transformation block size and its difference with the maximum transformation block , the maximum transformation level depth of the inter-frame prediction coding block and the intra-frame prediction coding block, whether to use a quantization table, etc.

The image parameter set PPS is mainly composed of various public parameters used in the process of encoding and decoding images. According to the content of the image parameter set, it can be divided into five parts: the first part is composed of the common syntax elements of the encoded image, mainly including the pps_id field , the referenced sps_id field; the second part consists of information about the slice header, mainly including whether the slice is a dependent slice, whether there is extra bit data in the slice header, the initialization method of the binary entropy coding algorithm, etc.; the third part consists of Composition of syntax elements related to quantization, mainly including syntax elements such as the initial value of quantization parameters, the offset of brightness and color quantization parameters; the fourth part is composed of information related to the availability of coding tools, mainly including P frame weighted prediction B frame weighted prediction, TILE mode, entropy coding synchronization mode, etc.; the fifth part is composed of extension information-related flag information, including a flag indicating the extension information of the slice header and a flag indicating the extension information of the image parameter set.

From the above analysis, we can see that the parameter sets VPS, SPS and PPS contain a large amount of key information needed in the video encoding and decoding process, such as identifier field, image size, video format, difference value, encoding level, etc., without parameter set will cause all video streams encoded using this parameter set to be unable to be decoded normally, nor can it be parsed and played by a general-purpose decoder. Therefore, the parameter sets VPS, SPS and PPS are identified as key content in this application. And because there are many parameter fields in the parameter set and the value range is wider, the plaintext space is sufficient to resist brute force attacks. Even if the encrypted video stream is intercepted or leaked, the video cannot be recovered without a specific decoder and key. content. In addition, since the encrypted part does not contain the actual encoded video content, the encryption and decryption processing delay and transmission delay are low, so it can be applied to video surveillance systems with high real-time requirements such as video surveillance and perimeter security.

Step 2.4, encrypting the key content identified in step 2.3.

Specifically, in this step, the identified key content parameter sets VPS, SPS and PPS are encrypted, specifically including the following steps:

Step 2.4.1. Use a symmetric block encryption algorithm to perform confidentiality encryption on the key content to obtain confidentially encrypted ciphertext data; wherein the symmetric block encryption algorithm includes AES, SM1, SM4, etc.

Step 2.4.2, use the key-based hash operation message authentication code algorithm (HMAC) to carry out integrity encryption on the key content, and obtain the ciphertext data after integrity encryption; wherein the key-based hash operation message authentication code Algorithms include SHA-256, SHA-512, SM3, etc.

Step 2.4.3. Directly concatenate the confidentiality-encrypted ciphertext data and the integrity-encrypted ciphertext data without adding any separator, and use the concatenated ciphertext to replace the plaintext of the key content. It should be noted that since the ciphertext output by integrity encryption has a fixed length, a specific decoder needs to first locate and intercept the integrity-encrypted ciphertext when decoding, and the rest is the confidentiality-encrypted ciphertext.

In a further embodiment, after encrypting the key content in step 2.4 and before obtaining the coding parameter set, it also includes packing the ciphertext obtained after the encryption, specifically including the following steps:

Step 2.4.4, performing a packing operation on the ciphertext spliced in the above step 2.4.3.

Step 2.4.5. Add a 0xffffffff check code at the end of the spliced ciphertext to mark the end of the ciphertext.

It should be noted that H.264 stipulates that there is a start code 0x000001 before each NALU. The decoder detects each start code as the start identifier of a NALU. When the next start code is detected, the current NALU ends . At the same time, H.264 stipulates that when 0x000000 is detected, it can also represent the end of the current NALU. Therefore, when 0 appears at the end of the encrypted ciphertext, H.264 cannot be decoded correctly by the decoder.

For this reason, the present application adds a 0xffffffff check code at the end of the spliced ciphertext to mark the end of the ciphertext, so as to be compatible with H.264 and H.265 encoding standards. In addition, when the information of this application is decoded by the decoder, when the 0xffffffff sequence is detected, it is the end position of the ciphertext, and then the encrypted data is read byte by byte, and then the ciphertext decryption and integrity verification are performed.

Step 3: After the RTSP verification is passed, in the video stream transmitted from the video shooting end to the video playback server, the relevant video parameters in each frame are removed, and only the video data part is reserved for transmission to the video playback server.

Step 4: The video playing server identifies and decrypts the encrypted encoding parameter set, and recovers the plaintext of the encoding parameter set.

It should be noted that, as shown in Figure 2, usually the RTSP 200OK returned by the video shooting end (that is, the camera) for the PLAY request will contain all the encoding parameter sets. During the RTSP verification process of plaintext video transmission, the video playback service The encoding parameter set in the RTSP 200OK at the end is in plain text; however, in this application, the video encryption client will encrypt the encoding parameter set in the RTSP 200OK returned by the camera, and then return the encrypted RTSP 200OK to the video playback service end. The video playback server receives the encrypted RTSP 200OK, uses the embedded decryption module to identify and decrypt the encrypted encoding parameter set, and restores the plaintext of the encoding parameter set.

In addition, in the process of video streaming of plaintext video, each frame encoded by the built-in encoder of the camera will add encoding parameter sets such as SPS, PPS or VPS, so some data packets will contain encoding parameters during video streaming. set; usually, the encoding parameters compiled by the same encoder are the same, which consumes a certain amount of broadband resources and cannot be well applied to scenarios where broadband resources are tight. Therefore, this application only retains the encoding in the returned RTSP 200OK Parameter set, during the subsequent video streaming process, the video encryption client will remove the video parameters in each frame, and only retain the video data part to transmit to the video playback server, where the decrypted encoding parameter set is used for decoding and playback.

To sum up, the real-time video transmission method compatible with the H.264 and H.265 encoding standards of this application can protect the security of the video content without changing the overall structure of the video stream and the transmission protocol, and is compatible with the mainstream H.264 and H.265 encoding standards. H.265 encoding standard; and, the encrypted transmission method of the present application can further save the consumption of broadband resources, so as to be better applicable to high-definition video, panoramic video and other video real-time transmission encryption scenarios that require higher bandwidth resources; in addition, the present application In the encrypted transmission method, because the parameter sets VPS, SPS and PPS are only encrypted once during the RTSP verification process, the ciphertext expansion rate does not exceed 1%, and the real-time transmission delay is low, and the practicability is stronger.

In the second aspect, as shown in Figures 1 and 4, this application also discloses a video real-time encrypted transmission system compatible with H.264 and H.265 encoding standards, which may include a video playback server 100, a video capture terminal 200, and a video Encrypt client 300.

Wherein, the video playing server 100 is used to perform a streaming request with the video shooting end 200, and perform RTSP verification with the video shooting end after the streaming request is successful.

The video encryption client 300 is used to analyze and encrypt the video stream during the RTSP verification process, and transmit the encoding parameter set obtained after the analysis and encryption to the video playback server 100 .

Wherein, parsing the video stream includes the following steps:

Step 2.1, analyzing the video stream to determine its encoding standard, wherein the encoding standard includes H.264 and/or H.265;

Step 2.2, locate the predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

When the encoding standard is H.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

When the encoding standard is H.265 or includes both H.264 and H.265, the predetermined parameter set includes video parameter set VPS, sequence parameter set SPS and picture parameter set PPS.

Specifically, as shown in FIG. 5 , the video encryption client 300 may further include a first positioning module 301 and a second positioning module 302 .

Wherein, the first positioning module 301 is used to determine each parameter set in each network abstraction layer unit NALU, specifically including:

In the video stream to be encrypted, the network abstraction layer unit NALU is positioned by searching the start code, and then the data type bit nal_unit_type of the network abstraction layer unit NALU header is read bit by bit, and then its value is converted into shaping;

in:

For the H.264 encoding standard:

When the reshaped value is 7, it is determined that the parameter set includes the sequence parameter set SPS; when the reshaped value is 8, it is determined that the parameter set includes the image parameter set PPS.

For the H.265 encoding standard:

When the value after shaping is 32, it is determined that the parameter set contains video parameter set VPS; when the value after shaping is 33, it is determined that the parameter set contains sequence parameter set SPS; when the value after shaping is 34, it is determined that the parameter set contains image Parameter set PPS.

The second positioning module 302 is used to locate the start and end positions of each parameter set in each network abstraction layer unit NALU, specifically including:

Skip one or two bytes at the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set, and then locate the next network abstraction layer unit NALU by searching the start code to determine the current network abstraction layer The end position of the unit NALU, that is, the end position of the corresponding parameter set.

Further, as shown in FIG. 6 , the video encryption client 300 also includes a first encryption module 303 , a second encryption module 304 , a splicing module 305 and a packing module 306 .

Wherein, the first encryption module 303 is configured to perform confidentiality encryption on the key content through a symmetric block encryption algorithm to obtain confidentiality-encrypted ciphertext data;

The second encryption module 304 is used to perform integrity encryption on the key content based on the hash operation message authentication code algorithm of the key, and obtain the ciphertext data after the integrity encryption;

The splicing module 305 is used to directly splice the confidentiality-encrypted ciphertext data and the integrity-encrypted ciphertext data, and use the spliced ciphertext to replace the plaintext of the key content.

Packing module 306 is used for carrying out packing operation to the ciphertext after splicing, then adds 0xffffffff check code at the end of ciphertext after splicing, in order to mark the end of this ciphertext, the ciphertext of packing after the final processing and The remaining video streams are combined to obtain encrypted video streams to be transmitted.

Similarly, the video real-time encrypted transmission system compatible with the H.264 and H.265 encoding standards of this application can protect the security of the video content without changing the overall structure of the video stream and the transmission protocol, and is compatible with the mainstream H.264 and H.265 encoding standards. H.265 encoding standard; and, the encrypted transmission system of the present application can further save the consumption of broadband resources, so as to be better applicable to high-definition video, panoramic video and other video real-time transmission encryption scenarios that require higher bandwidth resources; in addition, the present application In the encrypted transmission system, the parameter sets VPS, SPS and PPS are only encrypted once during the RTSP verification process, so the ciphertext expansion rate does not exceed 1%, and the real-time transmission delay is low, and the practicability is stronger.

Another aspect of the present application provides a computer device, including a processor, a memory, and a computer program stored on the memory and operable on the processor. The processor executes the computer program to implement the above-mentioned compatible H.264 and H. .265 encoding standard video real-time transmission method.

The computer device may include a central processing unit (CPU) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) or loaded from a storage section into a random access memory (RAM). In RAM, various programs and data necessary for the operation of the computer device are also stored. The CPU, ROM, and RAM are connected to each other through a bus. Input/output (I/O) interfaces are also connected to the bus.

The following parts are connected to the I/O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section including a hard disk, etc.; The communication part of the network interface card of LAN card, modem, etc. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the I/O interface as needed. A removable medium, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive as necessary so that a computer program read therefrom is installed into the storage section as necessary.

In particular, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication part, and/or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present application are performed. It should be noted that the computer storage medium in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The modules or units involved in the embodiments described in the present application may be implemented by means of software or by means of hardware. The described modules or units may also be set in the processor, and the names of these modules or units do not constitute limitations on the modules or units themselves in some cases.

As another aspect, the present application also provides a computer-readable storage medium, which may be included in the device described in the above-mentioned embodiments; or exist independently without being assembled into the device middle. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed by the device, the data is processed according to the above method.

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. All should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (10)

1. A video real-time encryption transmission method compatible with H.264 and H.265 coding standards is characterized by comprising the following steps:

step one, a streaming request is carried out between a video playing server and a video shooting end;

step two, after the streaming request is successful, RTSP verification is carried out between the video playing server and the video shooting end, and the verification process also comprises the steps of analyzing and encrypting the video stream and transmitting the coding parameter set obtained after the analysis and the encryption to the video playing server; wherein, the parsing the video stream includes:

step 2.1, analyzing the video stream to judge the coding standard, wherein the coding standard comprises H.264 and/or H.265;

step 2.2, locating a predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

when the coding standard is h.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

when the encoding standard is h.265 or includes both h.264 and h.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS;

step 2.3, marking the positioned preset parameter set as key content;

The encrypting the video stream comprises:

step 2.4, encrypting the key content identified in the step 2.3 to obtain a coding parameter set;

step three, after passing the RTSP verification, rejecting relevant video parameters in each frame in a video stream transmitted from the video shooting end to the video playing server, and only reserving a video data part for transmission to the video playing server;

and step four, the video playing server identifies and decrypts the encrypted coding parameter set and restores the plaintext of the coding parameter set.

2. The method according to claim 1, wherein in step 2.2, locating the predetermined parameter set in each network abstraction layer unit NALU comprises:

the determination of the parameter sets in each of said network abstraction layer units NALU and the positioning of the start and end positions of the parameter sets.

3. The method for video real-time encrypted transmission according to claim 2, wherein the determination of the video parameter set VPS, the sequence parameter set SPS and the picture parameter set PPS in each of the network abstraction layer units NALU comprises the steps of:

Step 2.2.1, locating the network abstraction layer unit NALU in the video stream by searching a start code;

step 2.2.2, reading the data type bit nal_unit_type of the network abstraction layer unit NALU header according to the bit, converting the value into a shaped value, and judging whether the network abstraction layer unit NALU contains a corresponding parameter set, wherein:

for the h.264 coding standard:

when the shaped value is 7, determining that a sequence parameter set SPS is contained in a network abstraction layer unit NALU;

when the shaped value is 8, determining that the network abstraction layer unit NALU contains an image parameter set PPS;

for the h.265 coding standard:

when the shaped value is 32, determining that the network abstraction layer unit NALU contains a video parameter set VPS;

when the shaped value is 33, determining that the network abstraction layer unit NALU contains a sequence parameter set SPS;

when the shaped value is 34, it is determined that the network abstraction layer unit NALU contains the picture parameter set PPS.

4. A method of video real-time encrypted transmission according to claim 3, wherein locating the start and end positions of the parameter sets in each network abstraction layer unit NALU comprises:

Step 2.2.3, skipping one or two bytes in the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set therein;

and 2.2.4, locating the next network abstraction layer unit NALU by searching the start code, thereby determining the end position of the current network abstraction layer unit NALU, namely the end position of the corresponding parameter set.

5. The method for real-time encrypted transmission of video according to claim 1, wherein in step 2.4, the step of encrypting the identified key content comprises the steps of:

2.4.1, carrying out confidentiality encryption on the key content by using a symmetric block encryption algorithm to obtain confidentiality encrypted ciphertext data;

step 2.4.2, carrying out integrity encryption on the key content by using a hash operation message authentication code algorithm based on a secret key to obtain encrypted ciphertext data;

and 2.4.3, directly splicing the encrypted ciphertext data with confidentiality and the encrypted ciphertext data with integrity, and replacing the plaintext of the key content by using the spliced ciphertext.

6. The method for real-time encrypted transmission of video according to claim 5, wherein after said encrypting said key content in step 2.4 and before said obtaining said encoding parameter set, further comprising the step of shell-adding ciphertext obtained after said encrypting, comprising the steps of:

Step 2.4.4, performing shell adding operation on the spliced ciphertext;

and 2.4.5, adding a 0xffffffff check code at the end of the spliced ciphertext to identify the end of the ciphertext.

7. The video real-time encryption transmission system compatible with the H.264 and H.265 coding standards is characterized by comprising a video playing service end (100), a video shooting end (200) and a video encryption client end (300), wherein:

the video playing server (100) is used for carrying out a streaming request with the video shooting end (200) and carrying out RTSP verification with the video shooting end after the streaming request is successful;

the video encryption client (300) is used for analyzing and encrypting the video stream in the RTSP verification process and transmitting the coding parameter set obtained after the analysis and encryption to the video playing server (100); the parsing the video stream includes the following steps:

step 2.1, analyzing the video stream to judge the coding standard, wherein the coding standard comprises H.264 and/or H.265;

step 2.2, locating a predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

when the coding standard is h.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

When the encoding standard is h.265 or includes both h.264 and h.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS;

step 2.3, marking the positioned preset parameter set as key content;

the encryption processing of the video stream comprises the following steps:

step 2.4, encrypting the key content identified in the step 2.3 to obtain a coding parameter set;

further, the video encryption client (300) is further configured to reject relevant video parameters in each frame from a video stream transmitted from the video capturing end (200) to the video playing server (100) after the RTSP verification is passed, and only a video data portion is reserved and transmitted to the video playing server (100);

further, the video playing server (100) is further configured to identify and decrypt the encrypted encoding parameter set, and restore the encoding parameter set plaintext.

8. The video real-time encrypted transmission system according to claim 7, wherein the video encryption client (300) comprises:

a first positioning module (301) configured to determine parameter sets in each network abstraction layer unit NALU;

And a second positioning module (302) configured to position a start and end position of each parameter set in each network abstraction layer unit NALU.

9. The video real-time encrypted transmission system according to claim 8, wherein said first positioning module (301) determining each parameter set comprises:

the network abstraction layer unit NALU is positioned in the video stream to be encrypted by searching a start code, then a data type bit nal_unit_type of the head of the network abstraction layer unit NALU is read according to the bit, and then the value is converted into a shaping, wherein:

for the h.264 coding standard:

when the shaped value is 7, determining that the parameter set comprises a sequence parameter set SPS; when the shaped value is 8, determining that the parameter set comprises an image parameter set PPS;

for the h.265 coding standard:

when the shaped value is 32, determining that the parameter set comprises a video parameter set VPS; when the shaped value is 33, determining that the parameter set comprises a sequence parameter set SPS; when the shaped value is 34, determining that the parameter set comprises the picture parameter set PPS;

further, the second positioning module (302) positioning the start and end positions of each parameter set includes:

One or two bytes are skipped over from the head of the NALU of the current network abstraction layer unit to locate the start position of the corresponding parameter set, and then the NALU of the next network abstraction layer unit is located by searching the start code, so that the end position of the NALU of the current network abstraction layer unit, namely the end position of the corresponding parameter set, is determined.

10. The video real-time encrypted transmission system according to claim 7, wherein said video encryption client (300) further comprises:

the first encryption module (303) is used for carrying out confidentiality encryption on the key content through a symmetric block encryption algorithm to obtain confidentiality encrypted ciphertext data;

the second encryption module (304) is used for carrying out integrity encryption on the key content based on a hash operation message authentication code algorithm of the key to obtain encrypted ciphertext data;

the splicing module (305) is used for directly splicing the secret encrypted ciphertext data and the integrity encrypted ciphertext data and replacing the plaintext of the key content by using the spliced ciphertext;

and the shell adding module (306) is used for carrying out shell adding operation on the spliced ciphertext, and then adding a 0 xffffffffff check code at the tail end of the spliced ciphertext to mark the end of the ciphertext.