CN116506412A

CN116506412A - Video real-time encryption transmission method and system compatible with H.264 and H.265 coding standards

Info

Publication number: CN116506412A
Application number: CN202310348907.6A
Authority: CN
Inventors: 吴敏; 张瑞庆; 龚丽; 刘翱; 张益�; 何龙; 张红莉
Original assignee: Second Research Institute of CAAC
Current assignee: Second Research Institute of CAAC
Priority date: 2023-04-03
Filing date: 2023-04-03
Publication date: 2023-07-28

Abstract

The application belongs to the technical field of information encryption, and particularly relates to a video real-time transmission method and system compatible with H.264 and H.265 coding standards, wherein the video real-time encryption transmission method comprises the following steps: a streaming request; RTSP verification, wherein the verification process also comprises the steps of analyzing and encrypting the video stream and transmitting the coding parameter set to a video playing server; after passing the RTSP verification, the relevant video parameters in each frame of the video stream are removed, and only the reserved video data part is transmitted; the video playing server identifies and decrypts the encrypted encoding parameter set and restores the encoding parameter set plaintext. The video real-time encryption transmission method and system can protect the safety of video content on the premise of not changing the overall structure and transmission protocol of the video stream, are compatible with H.264 and H.265 coding standards, and in addition, because the parameter sets VPS, SPS and PPS are encrypted only once in the RTSP verification process, the ciphertext expansion rate is not more than 1%, the real-time transmission delay is lower, and the practicability is stronger.

Description

Video real-time encryption transmission method and system compatible with H.264 and H.265 coding standards

Technical Field

The application belongs to the technical field of information encryption, and particularly relates to a video real-time encryption transmission method and system compatible with H.264 and H.265 coding standards.

Background

With the continuous development of the internet and communication technology, the sharing of multimedia information is also more convenient. Video applications such as video conferencing, video monitoring, perimeter security, etc. have gradually become part of everyday life. The openness of the internet also enables video information to be exposed in public environments, is easily intercepted, stolen or tampered by malicious organizations or hackers, and causes great negative influence and economic loss to industries and organizations. Therefore, how to secure video data in an open internet environment is important.

In addition, with the continuous high definition of video, video stitching and panoramic video technology are continuously developed, and multiplexing and sea-level video data also brings new challenges for data transmission and storage. To cope with this problem, the video coding standard is from the earliest h.261, MPEG series coding standard to the epoch-making h.264, and with the further development of IC chip technology, h.265 will gradually replace h.264 to become the mainstream standard in the future. Compared with H.264, the H.265 coding standard not only maintains the original technology, but also improves and optimizes partial technology, and improves the video compression rate to about 2 times of H.264, namely, only half of the bandwidth before the video with the same quality is transmitted. While h.265 is the mainstay of future applications, a significant portion of the systems now running use the h.264 coding standard. Therefore, it is important to study how to be compatible with the H.264 and H.265 coding standards on the premise of ensuring the safety of video data.

One way to solve the above problems is to encrypt the video content itself. Because the video information has the characteristics of large data volume, high correlation and the like, in the field of real-time video communication, the video is often required to be subjected to compression processing to reduce the redundancy of the video information before transmission, so that the data volume is reduced to ensure the real-time performance of video transmission.

The video encryption scheme is mainly divided into: encryption before encoding, encryption during encoding and encryption after encoding; the encryption after encoding is directly performed on the encoded video code stream, and because the encryption scheme is weakly related to the video encoding format, the encryption scheme is commonly adopted by various large security manufacturers and video manufacturers at present.

However, most of the current encryption after encoding belongs to full encryption, that is, all the video streams after encoding are encrypted, and the way still needs to encrypt and process a large amount of video data, so that delay cost and bandwidth cost are high.

Disclosure of Invention

In order to solve at least one technical problem in the prior art, the application provides a video real-time encryption transmission method and system compatible with H.264 and H.265 coding standards.

In a first aspect, the application discloses a video real-time encryption transmission method compatible with h.264 and h.265 coding standards, including the following steps:

Step one, a streaming request is carried out between a video playing server and a video shooting end;

step two, after the streaming request is successful, RTSP verification is carried out between the video playing server and the video shooting end, and the verification process also comprises the steps of analyzing and encrypting the video stream and transmitting the coding parameter set obtained after the analysis and the encryption to the video playing server; wherein, the parsing the video stream includes:

step 2.1, analyzing the video stream to judge the coding standard, wherein the coding standard comprises H.264 and/or H.265;

step 2.2, locating a predetermined parameter set in each network abstraction layer unit NALU in the video stream, wherein:

when the coding standard is h.264, the predetermined parameter set includes a sequence parameter set SPS and a picture parameter set PPS;

when the encoding standard is h.265 or includes both h.264 and h.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS;

step 2.3, marking the positioned preset parameter set as key content;

the encrypting the video stream comprises:

step 2.4, encrypting the key content identified in the step 2.3 to obtain a coding parameter set;

Step three, after passing the RTSP verification, rejecting relevant video parameters in each frame in a video stream transmitted from the video shooting end to the video playing server, and only reserving a video data part for transmission to the video playing server;

and step four, the video playing server identifies and decrypts the encrypted coding parameter set and restores the plaintext of the coding parameter set.

According to at least one embodiment of the present application, in the step 2.2, locating the predetermined parameter set in each network abstraction layer unit NALU includes:

the determination of the parameter sets in each of said network abstraction layer units NALU and the positioning of the start and end positions of the parameter sets.

In accordance with at least one embodiment of the present application, the determination of the video parameter set VPS, the sequence parameter set SPS and the picture parameter set PPS in each of the network abstraction layer units NALU comprises the steps of:

step 2.2.1, locating the network abstraction layer unit NALU in the video stream by searching a start code;

step 2.2.2, reading the data type bit nal_unit_type of the network abstraction layer unit NALU header according to the bit, converting the value into a shaped value, and judging whether the network abstraction layer unit NALU contains a corresponding parameter set, wherein:

For the h.264 coding standard:

when the shaped value is 7, determining that a sequence parameter set SPS is contained in a network abstraction layer unit NALU;

when the shaped value is 8, determining that the network abstraction layer unit NALU contains an image parameter set PPS;

for the h.265 coding standard:

when the shaped value is 32, determining that the network abstraction layer unit NALU contains a video parameter set VPS;

when the shaped value is 33, determining that the network abstraction layer unit NALU contains a sequence parameter set SPS;

when the shaped value is 34, it is determined that the network abstraction layer unit NALU contains the picture parameter set PPS.

According to at least one embodiment of the present application, the positioning of the start and end positions of the parameter sets in each network abstraction layer unit NALU includes:

step 2.2.3, skipping one or two bytes in the head of the current network abstraction layer unit NALU to locate the start position of the corresponding parameter set therein;

and 2.2.4, locating the next network abstraction layer unit NALU by searching the start code, thereby determining the end position of the current network abstraction layer unit NALU, namely the end position of the corresponding parameter set.

According to at least one embodiment of the present application, in the step 2.4, the encrypting the identified key content includes the following steps:

2.4.1, carrying out confidentiality encryption on the key content by using a symmetric block encryption algorithm to obtain confidentiality encrypted ciphertext data;

step 2.4.2, carrying out integrity encryption on the key content by using a hash operation message authentication code algorithm based on a secret key to obtain encrypted ciphertext data;

and 2.4.3, directly splicing the encrypted ciphertext data with confidentiality and the encrypted ciphertext data with integrity, and replacing the plaintext of the key content by using the spliced ciphertext.

According to at least one embodiment of the present application, in the step 2.4, before the key content is encrypted to obtain the encoding parameter set, the method further includes a step of shell-adding the ciphertext obtained after the encryption, and specifically includes the following steps:

step 2.4.4, performing shell adding operation on the spliced ciphertext;

and 2.4.5, adding a 0xffffffff check code at the end of the spliced ciphertext to identify the end of the ciphertext.

In a second aspect, the application further discloses a video real-time encryption transmission system compatible with h.264 and h.265 coding standards, including a video playing server, a video shooting end and a video encryption client, wherein:

The video playing server side is used for carrying out a streaming request with the video shooting end, and carrying out RTSP verification with the video shooting end after the streaming request is successful;

the video encryption client is used for analyzing and encrypting the video stream in the RTSP verification process and transmitting the coding parameter set obtained after the analysis and encryption to the video playing server; the parsing the video stream includes the following steps:

step 2.3, marking the positioned preset parameter set as key content;

the encryption processing of the video stream comprises the following steps:

further, the video encryption client is further configured to reject relevant video parameters in each frame from a video stream transmitted from the video capturing end to the video playing server after the RTSP verification is passed, and only a video data portion is reserved and transmitted to the video playing server;

further, the video playing server is further configured to identify and decrypt the encrypted encoding parameter set, and restore the plaintext of the encoding parameter set.

According to at least one embodiment of the present application, the video encryption client includes:

the first positioning module is used for determining each parameter set in each network abstraction layer unit NALU;

and the second positioning module is used for positioning the starting and ending positions of each parameter set in each network abstraction layer unit NALU.

According to at least one embodiment of the present application, the determining, by the first positioning module, each parameter set includes:

the network abstraction layer unit NALU is positioned in the video stream to be encrypted by searching a start code, then a data type bit nal_unit_type of the head of the network abstraction layer unit NALU is read according to the bit, and then the value is converted into a shaping, wherein:

For the h.264 coding standard:

when the shaped value is 7, determining that the parameter set comprises a sequence parameter set SPS; when the shaped value is 8, determining that the parameter set comprises an image parameter set PPS;

for the h.265 coding standard:

when the shaped value is 32, determining that the parameter set comprises a video parameter set VPS; when the shaped value is 33, determining that the parameter set comprises a sequence parameter set SPS; when the shaped value is 34, determining that the parameter set comprises the picture parameter set PPS;

further, the second positioning module positioning the start and end positions of each parameter set includes:

one or two bytes are skipped over from the head of the NALU of the current network abstraction layer unit to locate the start position of the corresponding parameter set, and then the NALU of the next network abstraction layer unit is located by searching the start code, so that the end position of the NALU of the current network abstraction layer unit, namely the end position of the corresponding parameter set, is determined.

According to at least one embodiment of the present application, the video encryption client further comprises:

the first encryption module is used for carrying out confidentiality encryption on the key content through a symmetrical block encryption algorithm to obtain ciphertext data after confidentiality encryption;

The second encryption module is used for carrying out integrity encryption on the key content by a hash operation message authentication code algorithm based on the key to obtain encrypted ciphertext data;

the splicing module is used for directly splicing the secret encrypted ciphertext data and the integrity encrypted ciphertext data, and replacing the plaintext of the key content by using the spliced ciphertext;

and the shell adding module is used for carrying out shell adding operation on the spliced ciphertext, and then adding a 0xffffffff check code at the tail end of the spliced ciphertext to identify the end of the ciphertext.

The application has at least the following beneficial technical effects:

1) The video real-time encryption transmission method and system compatible with the H.264 and H.265 coding standards can protect the safety of video content on the premise of not changing the overall structure and transmission protocol of the video stream, and simultaneously is compatible with the mainstream H.264 and H.265 coding standards;

in addition, the encoding parameter set is reserved only in the RTSP 200OK returning step of RTSP verification, video parameters in each frame are removed in the video streaming transmission process, only video data parts are reserved and transmitted to the video playing server, and finally decoding and playing are carried out by using the decrypted encoding parameter set in the video playing server, so that the broadband resource consumption can be further saved, and the method is better suitable for video real-time transmission encryption scenes with high bandwidth resource requirements, such as high-definition videos and panoramic videos;

2) In the video real-time encryption transmission method and system compatible with the H.264 and H.265 coding standards, the parameter sets VPS, SPS and PPS are encrypted only once in the RTSP verification process, so that the ciphertext expansion rate is not more than 1%, the real-time transmission delay is lower, and the practicability is stronger.

Drawings

FIG. 1 is a flow chart of a video real-time encryption transmission method compatible with H.264 and H.265 coding standards in the application;

FIG. 2 is a block diagram of the RTSP verification flow in the video real-time encryption transmission method compatible with the H.264 and H.265 coding standards;

FIG. 3 is a block flow diagram of parsing and encryption processing steps in the video real-time encryption transmission method compatible with H.264 and H.265 coding standards of the present application;

FIG. 4 is a block diagram of a video real-time encrypted transmission system compatible with the H.264 and H.265 coding standards in the present application;

FIG. 5 is a diagram of a positioning module in the video real-time encryption transmission system compatible with the H.264 and H.265 coding standards;

FIG. 6 is a diagram showing the configuration of an encryption module in the video real-time encryption transmission system compatible with the H.264 and H.265 coding standards;

fig. 7 is a NALU header format (frame format) parsing diagram of h.264;

fig. 8 is a NALU header format (frame format) parsing diagram of h.265;

Fig. 9 is a section of an h.265 code flow diagram (table) in one embodiment.

Detailed Description

For a better understanding of the present application, technical terms mentioned in the present application will first be described:

1) Video coding is a mode of compressing video original data into a code stream, and in the application, H.264 and/or H.265 (namely H.264 or H.265 or H.264 and H.265 are contained simultaneously) technology is adopted for coding;

2) The code stream is composed of a plurality of network abstraction layer units (Network Abstract Layer Unit, NALU), for example, the h.264 original code stream (also called "bare stream") is composed of one NALU, wherein each NALU is separated by a startcode (start code), and the start codes are divided into two types: 0x000001 (3 Byte) or 0x00000001 (4 Byte);

3) The network abstraction layer unit NALU includes a Header (Header) and Payload Data (Payload Data), and according to the type of Data carried, the NALU includes: a Video Code Layer network abstraction Layer unit (VCLU) and a non-Video Code Layer network abstraction Layer unit (non-Video Code Layer NALU, non-VCLU) for two types;

wherein, non-VCLU is NALU for carrying parameter set, in this application, parameter set includes: video parameter set (Video Parament Set, VPS), sequence parameter set (Sequence Paramenter Set, SPS), picture parameter set (Picture Paramenter Set, PPS) three types;

In addition, the video parameter set VPS is a parameter set specific to the h.265 coding standard, and the sequence parameter set SPS and the picture parameter set PPS are parameter sets common to h.264 and h.265;

5) Video streaming, which refers to the transmission of video data, for example, can be handled as a steady and continuous stream over the network, i.e., a technique that enables audio, video and other multimedia to be played over the Internet and Intranet in real-time, without waiting for download;

6) The parsing of the video stream refers to parsing the NALU of its basic unit from the corresponding code stream, and may parse the fields of the NALU header, for example, the parsing of the h.264 code stream is first searching for 0x000001 and 0x00000001 from the code stream, separating the NALU, and then parsing the fields of the NALU.

7) Plaintext, meaning text (or string of characters) that is not encrypted, may be a bit stream in a communication system, such as text, a bit map, digitized speech or digitized video images, etc., and may be generally simply considered to be a meaningful character or set of bits, or a message that is available through some sort of published encoding standard.

In order to make the purposes, technical solutions and advantages of the implementation of the present application more clear, the technical solutions in the embodiments of the present application will be described in more detail below with reference to the accompanying drawings in the embodiments of the present application. The described embodiments are some, but not all, of the embodiments of the present application. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.

1-3, the application discloses a video real-time encryption transmission method compatible with H.264 and H.265 coding standards, which comprises the following steps:

step one, a streaming request is performed between a video playing server and a video shooting end (i.e. a camera in fig. 1).

It should be noted that, in the field of plaintext video transmission, a streaming request, RTSP verification and video streaming are sequentially performed between a video playing server and a camera, which belong to a relatively mature technology, so that each step is not described in detail herein.

Step two, after the streaming request is successful, RTSP verification is carried out between the video playing server and the video shooting end (see the flow shown in fig. 2), and the verification process also comprises analysis and encryption processing of the video stream (see the flow shown in fig. 3), and the coding parameter set obtained after the analysis and encryption processing is transmitted to the video playing server.

Wherein parsing the video stream includes:

step 2.1, parsing the frequency stream to determine the coding standard, wherein the coding standard includes h.264 and/or h.265 (the coding standard except h.265 is default in fig. 1 to be h.264 which is commonly used at present).

Step 2.2, locating a predetermined set of parameters in each of the video streams (the original code stream is composed of a plurality of NALUs) in the network abstraction layer unit NALUs, wherein:

when the encoding standard is h.265 or includes both h.264 and h.265, the predetermined parameter set includes a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS.

Further, the positioning described in this step includes determining the respective parameter sets (i.e., the video parameter set VPS, the sequence parameter set SPS, and the picture parameter set PPS) in each network abstraction layer unit NALU and positioning the start and end positions of the respective parameter sets.

Specifically, the determination of the video parameter set VPS, the sequence parameter set SPS and the picture parameter set PPS in each network abstraction layer unit NALU includes the following steps:

step 2.2.1, locating the network abstraction layer unit NALU in the video stream by searching for a start code (0 x 000001).

Step 2.2.2, respectively reading the data type bit nal_unit_type of the head of the network abstraction layer unit NALU according to the coding standard, converting the value into a shaped value, and judging whether the network abstraction layer unit NALU contains a corresponding parameter set, wherein:

For the h.264 coding standard:

when the value after shaping is 8, the network abstraction layer unit NALU is determined to contain the picture parameter set PPS.

For the h.265 coding standard:

when the value after shaping is 33, determining that a sequence parameter set SPS is contained in a network abstraction layer unit NALU;

when the reshaped value is 34, it is determined that the network abstraction layer unit NALU contains the picture parameter set PPS.

Further, referring to fig. 7, the NALU Header (Header) of h.264 occupies only one byte (i.e., 1byte, which is equal to 8 bits), wherein:

f: for the forbidden_zero_bit, to be a forbidden bit, the forbidden bit should be 0, and when the forbidden bit is 1, it indicates that there is an error in NALU, and it needs to be discarded or corrected;

NRI: the nal_ref_idc is an important indication bit, occupies 2 bits, takes 00 to 11, and indicates the importance of the NALU; the larger the value is, the more important the current NAL is, and the priority is required to be protected; if the current NALU is the important unit of a sequence parameter set or an image parameter set, the syntax element must be greater than 0;

Type: the specific meaning of nal_unit_type, which identifies the data type of the payload in NALU, occupies 5 bits, is shown in table 1 below:

table 1-nalu_unit_type meaning table commonly found in NALU header of h.264

nal_unit_type	Content of the representation
		0	Is not defined
7	Sequence parameter set SPS
		8	Picture parameter set PPS
24	Event portfolio
		25-27	Time combination bag
28-29	Slicing unit
		30-31	Reserved bits

Further, referring to fig. 8, the NALU header of h.265 occupies two bytes, wherein:

type: nal_unit_type, occupying 6 bits, identifies the data type of the payload in NALU; the specific meaning of the common nal_unit_type is shown in table 2 below;

layerld: nuh_layer_id, 6 bits, represents an identifier of a layer to which NALU belongs or an identifier of a layer to which non-VCLU is applied;

TID: nuh_temporal_id_plus1, 3 bits, specifies the time identifier of NALU;

table 2-NALU header of h.265 common nal_unit_type meaning table

Further, referring to fig. 9, an example analysis is performed by using a code stream of h.265, wherein the start code is in the rectangular frame a1, the NALU header is in the rectangular frame a2, the nal_unit_type corresponding to "40" is 32 (VPS), the nal_unit_type corresponding to "42" is 33 (SPS), and the nal_unit_type corresponding to "44" is 34 (PPS).

Further, in the third step, the positioning of the start and end positions of each parameter set in each network abstraction layer unit NALU specifically includes:

step 2.2.3, skipping one or two bytes in the current network abstraction layer unit NALU header according to the identified coding standard to locate the start position of the corresponding parameter set therein.

Step 2.2.4, locating the next network abstraction layer unit NALU by searching for a start code (0 x 000001), thereby determining the end position of the current network abstraction layer unit NALU, i.e. the end position of the corresponding parameter set.

And 2.3, identifying the located preset parameter set as key content.

Specifically, this step is to identify the parameter sets VPS, SPS and PPS located in step 2.2 as key contents.

It should be noted that, through analysis of the h.264 and h.265 codes, it can be known that the information in the video stream is not uniformly distributed, and that some of the key content contains more information than other content, which plays a more important role in the video encoding and decoding (i.e. parsing) process, and the following will briefly analyze the reason for dividing the important content:

the video parameter set VPS contains a plurality of syntax elements of the video coding sequence, which can be divided into three parts according to the content of the video parameter set: the first part is composed of syntax elements shared by sublayers or operation points and mainly comprises a video parameter set identifier, the maximum layer number of a video coding sequence, the maximum time domain sublayer number of the video coding sequence and the like; the second part is composed of information related to the level, the layer and the level, and mainly comprises a syntax element profile_tier_level (); profile_tier_level () determines the level, layer and level related content according to the value of the parameter; the third part is constituted by the operation point information specific to the video parameter set.

The content of the sequence parameter set SPS is mainly composed of information common to the pictures in the coded video sequence, which can be divided into four parts according to the content of the sequence parameter set: the first part is composed of syntax elements shared by operation points and mainly comprises a video parameter set reference identifier, a sequence parameter set identifier, a maximum time domain sub-layer number contained in a video coding sequence and limit information of inter-frame prediction of the video coding sequence, wherein SPS indicates the referenced video parameter set through a vps_id field, and the SPS can be referenced by the image parameter set through taking the sps_id field as a unique identifier; the second part is composed of information related to the grade, the layer and the level; the third part is composed of image format related information and mainly comprises a chroma sampling format, an image resolution, a quantization depth and a time domain sub-layer sequence mark; the fourth part is composed of coding parameter related information, and mainly comprises the size of the minimum brightness coding block and the difference value between the minimum brightness coding block and the maximum brightness coding block, the size of the minimum transformation block and the difference value between the minimum transformation block and the maximum transformation block, the maximum transformation hierarchical depth between the inter-frame prediction coding block and the intra-frame prediction coding block, whether a quantization table is used or not, and the like.

The picture parameter set PPS is mainly composed of various common parameters used in the process of encoding and decoding a picture, and may be divided into five parts according to the content of the picture parameter set: the first part is composed of a common syntax element of the coded picture, and mainly comprises a pps_id field and a referenced sps_id field; the second part is composed of related information of the slice header and mainly comprises information such as whether the slice is a dependent slice, whether extra bit data exists in the slice header, a binary entropy coding algorithm initialization method and the like; the third part is composed of syntax elements related to quantization and mainly comprises syntax elements such as initial values of quantization parameters, offsets of luminance and chrominance quantization parameters and the like; the fourth part is composed of the relevant information of the availability of the coding tool and mainly comprises a P frame weighted prediction B frame weighted prediction, a TILE mode, an entropy coding synchronous mode and the like; the fifth section is composed of extension information related flag information including a flag indicating the slice header extension information and a flag indicating the image parameter set extension information.

From the above analysis, it can be seen that the parameter sets VPS, SPS and PPS contain a lot of key information required in the video codec process, such as identifier field, image size, video format, difference value, encoded level, etc., and the absence of the parameter sets would not result in that all video streams encoded using the parameter sets cannot be decoded normally nor parsed and played by the general decoder, so the parameter sets VPS, SPS and PPS are identified as key contents in the present application. And because the parameter fields in the parameter set are numerous, the value range is wider, so that the plaintext space is enough to resist violent exhaustive attack, even if the encrypted video stream is intercepted or leaked, the video content can not be recovered without a specific decoder and a specific secret key. In addition, the encryption part does not contain the video content after actual encoding, so the encryption and decryption processing time delay and the transmission time delay are lower, and the method is suitable for video monitoring systems with higher requirements on real-time performance, such as video monitoring, surrounding security and the like.

And 2.4, encrypting the key content identified in the step 2.3.

Specifically, the encryption of the identified key content parameter sets VPS, SPS and PPS includes the following steps:

2.4.1, carrying out confidentiality encryption on the key content by using a symmetric block encryption algorithm to obtain confidentiality encrypted ciphertext data; the symmetric block encryption algorithm comprises AES, SM1, SM4 and the like.

2.4.2, performing integrity encryption on the key content by using a hash operation message authentication code algorithm (HMAC) based on a secret key to obtain encrypted ciphertext data; the key-based hash operation message authentication code algorithm comprises SHA-256, SHA-512, SM3 and the like.

And 2.4.3, directly splicing the confidentiality encrypted ciphertext data and the integrity encrypted ciphertext data, adding no separator in the middle, and replacing the plaintext of the key content by using the spliced ciphertext. It should be noted that, because the ciphertext output by the integrity encryption has a fixed length, when decoding, a specific decoder needs to first locate and intercept the ciphertext of the integrity encryption, and the rest is the ciphertext of the confidentiality encryption.

In a further embodiment, after the encryption processing is performed on the key content in step 2.4 and before the encoding parameter set is obtained, the method further includes the step of shell-adding the ciphertext obtained after the encryption processing, and specifically includes the following steps:

And 2.4.4, performing shell adding operation on the ciphertext spliced in the step 2.4.3.

It should be noted that h.264 specifies that each NALU is preceded by a start code of 0x000001, and the decoder detects each start code as a start identifier of one NALU, and when the next start code is detected, the current NALU ends. Meanwhile, h.264 specifies that when 0x000000 is detected, the end of the current NALU can also be characterized. So when 0 appears at the end of the encrypted ciphertext, it may result in h.264 not being decoded correctly by the decoder.

Therefore, the 0xffffffff check code is added at the end of the spliced ciphertext to identify the end of the ciphertext, so that the H.264 and H.265 coding standards can be compatible. In addition, when the decoder decodes the information of the application, when the 0xffffffff sequence is detected, the end position of the ciphertext is obtained, then the encrypted data is read byte by byte, and then the ciphertext decryption and the integrity check are carried out.

And thirdly, after passing the RTSP verification, rejecting relevant video parameters in each frame in a video stream transmitted from a video shooting end to the video playing server, and only reserving a video data part and transmitting the video data part to the video playing server.

And step four, the video playing server identifies and decrypts the encrypted coding parameter set, and recovers the plaintext of the coding parameter set.

It should be noted that, referring to fig. 2, in general, the RTSP 200OK returned by the video capturing end (i.e., the camera) for the PLAY request will include all the encoding parameter sets, and in the RTSP verification process of the plaintext video transmission, the encoding parameter set in the RTSP 200OK returned to the video playing server is in the plaintext state; however, in the present application, the video encryption client encrypts the encoding parameter set in the RTSP 200OK returned by the camera, and then returns the encrypted RTSP 200OK to the video playing server. The video playing server receives the encrypted RTSP 200OK, and the embedded decryption module is used for identifying and decrypting the encrypted coding parameter set to restore the plaintext of the coding parameter set.

In addition, in the video stream transmission process of the plaintext video, each frame encoded by a built-in encoder of a general camera is added with encoding parameter sets such as SPS, PPS or VPS, so that a part of data packets contain the encoding parameter sets in the video stream transmission process; in general, the coding parameters encoded by the same encoder are the same, and have a certain broadband resource consumption, and cannot be well suitable for scenes with broadband resource shortage, so that the application only reserves a coding parameter set in the returned RTSP 200OK, and then in the video streaming transmission process, the video encryption client rejects the video parameters in each frame, only reserves a video data part and transmits the video parameters to the video playing server, and the video playing server uses the decrypted coding parameter set to perform decoding playing.

In summary, the video real-time transmission method compatible with the H.264 and H.265 coding standards can protect the safety of video content on the premise of not changing the overall structure and transmission protocol of the video stream, and is compatible with the mainstream H.264 and H.265 coding standards; in addition, the encryption transmission method can further save broadband resource consumption, so that the method is better suitable for transmitting the encryption scene in real time for high-definition videos, panoramic videos and other videos with high requirements on bandwidth resources; in addition, in the encryption transmission method, the parameter sets VPS, SPS and PPS are encrypted only once in the RTSP verification process, so that the ciphertext expansion rate is not more than 1%, the real-time transmission delay is lower, and the practicability is higher.

In a second aspect, as shown in fig. 1 and 4, the application further discloses a video real-time encryption transmission system compatible with h.264 and h.265 coding standards, which may include a video playing server 100, a video capturing end 200, and a video encryption client 300.

The video playing server 100 is configured to perform a streaming request with the video capturing end 200, and perform RTSP verification with the video capturing end after the streaming request is successful.

The video encryption client 300 is configured to parse and encrypt the video stream in the RTSP verification process, and transmit the encoding parameter set obtained after the parsing and the encryption process to the video playing server 100.

The parsing of the video stream comprises the following steps:

Specifically, as shown in fig. 5, the video encryption client 300 may further include a first positioning module 301 and a second positioning module 302.

The first positioning module 301 is configured to determine each parameter set in each network abstraction layer unit NALU, and specifically includes:

positioning a network abstraction layer unit NALU in a video stream to be encrypted by searching a start code, reading a data type bit nal_unit_type of the head of the network abstraction layer unit NALU according to the bit, and converting the value into shaping;

wherein:

for the h.264 coding standard:

when the shaped value is 7, determining that the parameter set comprises a sequence parameter set SPS; when the reshaped value is 8, it is determined that the parameter set contains the picture parameter set PPS.

For the h.265 coding standard:

when the value after shaping is 32, the parameter set is determined to contain the video parameter set VPS, when the value after shaping is 33, the parameter set is determined to contain the sequence parameter set SPS, and when the value after shaping is 34, the parameter set is determined to contain the picture parameter set PPS.

The second positioning module 302 is configured to position a start position and an end position of each parameter set in each network abstraction layer unit NALU, and specifically includes:

Further, as shown in fig. 6, the video encryption client 300 further includes a first encryption module 303, a second encryption module 304, a splicing module 305, and a shell adding module 306.

The first encryption module 303 is configured to perform confidentiality encryption on the key content through a symmetric packet encryption algorithm, so as to obtain ciphertext data after confidentiality encryption;

the second encryption module 304 is configured to perform integrity encryption on the key content based on a hash operation message authentication code algorithm of the key, so as to obtain encrypted ciphertext data after integrity encryption;

The splicing module 305 is configured to directly splice the ciphertext data after confidentiality encryption and the ciphertext data after integrity encryption, and replace the plaintext of the key content with the ciphertext after splicing.

The shell adding module 306 is configured to perform a shell adding operation on the spliced ciphertext, and then add a 0xffffffff check code at the end of the spliced ciphertext to identify the end of the ciphertext, and finally combine the processed and shelled ciphertext with the remaining video stream, thereby obtaining the encrypted video stream to be transmitted.

Likewise, the video real-time encryption transmission system compatible with the H.264 and H.265 coding standards can protect the safety of video content on the premise of not changing the overall structure and transmission protocol of the video stream, and is compatible with the mainstream H.264 and H.265 coding standards; in addition, the encryption transmission system can further save broadband resource consumption, so that the encryption transmission system is better suitable for transmitting the encryption scene in real time for high-definition videos, panoramic videos and other videos with high requirements on bandwidth resources; in addition, in the encryption transmission system, the parameter sets VPS, SPS and PPS are encrypted only once in the RTSP verification process, so that the ciphertext expansion rate is not more than 1%, the real-time transmission delay is lower, and the practicability is higher.

In another aspect of the present application, a computer device is provided, including a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor executing the computer program for implementing the video real-time transmission method compatible with the h.264 and h.265 coding standards.

The computer device may include a Central Processing Unit (CPU) that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) or a program loaded from a storage section into a Random Access Memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The CPU, ROM and RAM are connected to each other by a bus. An input/output (I/O) interface is also connected to the bus.

The following components are connected to the I/O interface: an input section including a keyboard, a mouse, etc.; an output section including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section including a hard disk or the like; and a communication section including a network interface card such as a LAN card, a modem, and the like. The communication section performs communication processing via a network such as the internet. The drives are also connected to the I/O interfaces as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and the like are mounted on the drive as needed so that a computer program read therefrom is mounted into the storage section as needed.

In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU). It should be noted that, the computer storage medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The modules or units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware. The modules or units described may also be provided in a processor, the names of which do not in some cases constitute a limitation of the module or unit itself.

As another aspect, the present application also provides a computer-readable storage medium, which may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable storage medium carries one or more programs which, when executed by the apparatus, process data as described above.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions easily conceivable by those skilled in the art within the technical scope of the present application should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video real-time encryption transmission method compatible with H.264 and H.265 coding standards is characterized by comprising the following steps:

step 2.3, marking the positioned preset parameter set as key content;

The encrypting the video stream comprises:

2. The method according to claim 1, wherein in step 2.2, locating the predetermined parameter set in each network abstraction layer unit NALU comprises:

3. The method for video real-time encrypted transmission according to claim 2, wherein the determination of the video parameter set VPS, the sequence parameter set SPS and the picture parameter set PPS in each of the network abstraction layer units NALU comprises the steps of:

for the h.264 coding standard:

for the h.265 coding standard:

4. A method of video real-time encrypted transmission according to claim 3, wherein locating the start and end positions of the parameter sets in each network abstraction layer unit NALU comprises:

5. The method for real-time encrypted transmission of video according to claim 1, wherein in step 2.4, the step of encrypting the identified key content comprises the steps of:

6. The method for real-time encrypted transmission of video according to claim 5, wherein after said encrypting said key content in step 2.4 and before said obtaining said encoding parameter set, further comprising the step of shell-adding ciphertext obtained after said encrypting, comprising the steps of:

Step 2.4.4, performing shell adding operation on the spliced ciphertext;

7. The video real-time encryption transmission system compatible with the H.264 and H.265 coding standards is characterized by comprising a video playing service end (100), a video shooting end (200) and a video encryption client end (300), wherein:

the video playing server (100) is used for carrying out a streaming request with the video shooting end (200) and carrying out RTSP verification with the video shooting end after the streaming request is successful;

the video encryption client (300) is used for analyzing and encrypting the video stream in the RTSP verification process and transmitting the coding parameter set obtained after the analysis and encryption to the video playing server (100); the parsing the video stream includes the following steps:

step 2.3, marking the positioned preset parameter set as key content;

the encryption processing of the video stream comprises the following steps:

further, the video encryption client (300) is further configured to reject relevant video parameters in each frame from a video stream transmitted from the video capturing end (200) to the video playing server (100) after the RTSP verification is passed, and only a video data portion is reserved and transmitted to the video playing server (100);

further, the video playing server (100) is further configured to identify and decrypt the encrypted encoding parameter set, and restore the encoding parameter set plaintext.

8. The video real-time encrypted transmission system according to claim 7, wherein the video encryption client (300) comprises:

a first positioning module (301) configured to determine parameter sets in each network abstraction layer unit NALU;

And a second positioning module (302) configured to position a start and end position of each parameter set in each network abstraction layer unit NALU.

9. The video real-time encrypted transmission system according to claim 8, wherein said first positioning module (301) determining each parameter set comprises:

for the h.264 coding standard:

for the h.265 coding standard:

further, the second positioning module (302) positioning the start and end positions of each parameter set includes:

10. The video real-time encrypted transmission system according to claim 7, wherein said video encryption client (300) further comprises:

the first encryption module (303) is used for carrying out confidentiality encryption on the key content through a symmetric block encryption algorithm to obtain confidentiality encrypted ciphertext data;

the second encryption module (304) is used for carrying out integrity encryption on the key content based on a hash operation message authentication code algorithm of the key to obtain encrypted ciphertext data;

the splicing module (305) is used for directly splicing the secret encrypted ciphertext data and the integrity encrypted ciphertext data and replacing the plaintext of the key content by using the spliced ciphertext;

and the shell adding module (306) is used for carrying out shell adding operation on the spliced ciphertext, and then adding a 0 xffffffffff check code at the tail end of the spliced ciphertext to mark the end of the ciphertext.