EP2297951A1

EP2297951A1 - Method and system making it possible to visually encrypt the mobile objects within a compressed video stream

Info

Publication number: EP2297951A1
Application number: EP09757564A
Authority: EP
Inventors: Cedric Le Barz; Marc Leny; Erwann Renan
Original assignee: Thales SA
Current assignee: Thales SA
Priority date: 2008-06-03
Filing date: 2009-06-03
Publication date: 2011-03-23
Also published as: FR2932046B1; WO2009147183A1; BRPI0913390A2; MX2010013317A; US20110075842A1; FR2932046A1; MA32377B1

Abstract

Method of protecting at least one part of a video stream or of a compressed video sequence against a violation of the information contained in said stream, said stream being able to be decomposed into at least one first type of objects and one second type of objects, the method being applied to a selection of images contained in said video sequence, characterized in that it comprises at least the following steps: analyzing the video sequence in the compressed domain so as to define for a given image N at least one first type of objects to be protected by encryption and a second type of objects (1, 2), the analysis module which identifies the portions of the image belonging to the first type of objects to be protected sends the motion estimation vectors and the transformed coefficients to the encryption step, determining, according to the partitioning into portions or into groups of portions of the existing image N (3b, 3a), those corresponding respectively to the first type of objects to be protected (3b) and to the other types (3a), encrypting at least a part of the first type of objects (5a), reconstructing on the basis of the outputs of the previous steps a stream composed of at least one first type of encrypted objects and of other types of objects, to which is added information indicating which objects are the ones that have been encrypted. System making it possible to implement the method.

Description

METHOD AND SYSTEM FOR VISUALLY CRYPTING MOBILE OBJECTS WITHIN A COMPRESSED VIDEO STREAM

The invention relates to a method and a system for transmitting a compressed video stream by providing visual encryption of moving objects. The method includes a step of encrypting the selected areas in a given image of the compressed stream. The invention applies to transmit compressed video streams in a transmission context where the latter are likely to be intercepted and viewed. It is also used to prevent access to the content of certain protected areas in the image. Thus, any malicious visualization of the flow can be avoided. It allows to protect, for example, the privacy of people in places equipped with video surveillance, allowing a partial or total decoding of the image according to a particular level of accreditation or authorization. In the remainder of the description, the term "foreground" designates the mobile object or objects in a video sequence, for example, a pedestrian, a vehicle, a molecule in medical imaging. In contrast, the designation "background" is used with reference to the environment as well as to fixed objects. This includes, for example, soil, buildings, trees that are not perfectly immobile or parked cars. More generally, it is possible in an image to define several areas that will be treated differently depending on the degree of confidentiality that must be associated. The words encryption or encryption are used interchangeably.

The invention applies, for example, at the output of a video encoder. The invention can, inter alia, be applied in applications implementing the standard defined jointly by the ISO MPEG and the video coding group of I ¹ ITU-T called H.264 or MPEG-4 AVC (advanced video coding) and SVC (scalable video coding), which is a video standard that provides more efficient compression than previous video standards while providing Reasonable implementation complexity and oriented towards networked applications.

In the remainder of the description, the Applicant uses the expression "compressed stream" or "compressed video sequence" to designate the same object.

The expression "slices" or "portions" more known in the field under the term "slices" corresponds to a sub-part of the image consisting of macro blocks that belong to the same set defined by the user. . These terms are well known to those skilled in the art of compression, for example, in MPEG standards. The concept of network abstraction layer, or NAL (Network Abstraction Layer), used in the rest of the description exists in the H.264 standard. It is a network transport unit that can contain either a "slice" portion for the Video Coding Layer NALs, or a data packet (parameter sets known to those skilled in the art - SPS, PPS -, user data, etc.) for NON-VCL NALs.

Video surveillance is increasingly used, raising the issue of privacy. In some applications, only images or zones of video stream images must be accessible, accessibility being associated with authorized persons. The publication Cyril Bergeron and Catherine Lamy's Compilant Selective Encryption for H.264 / AVC video streams discloses a method for encrypting a video stream to secure transmission. information contained in this stream while preserving compatibility with the H.264 standard. If only a few bits inside the stream are modified to obtain this result, the video sequence produced is visually completely encrypted, and can not be displayed without a decoding key. One of the objects of the present invention is to provide a method for protecting the image content or certain areas of an image capable of targeting only the moving objects within a compressed video stream in order to preserve the image. data integrity and confidentiality.

The invention relates to a method for protecting at least a part of a video stream or a compressed video sequence against a violation of the information contained in said stream, said stream being able to be decomposed into at least a first type of objects and a second type of object, the method applying to a selection of images contained in said video sequence, characterized in that it comprises at least the following steps: a) analyzing the video sequence in the compressed domain in order to define for a given image N, at least one first type of object to be protected by encryption and a second type of object, b) to determine, according to the division into portions or groups of portions of the existing image N, the portions or groups portions respectively corresponding to the first type of objects to be protected and the other types of objects, c) encrypting at least a portion of the first type of objects, and d) reconstructing from the outputs of the preceding steps a stream consisting of at least a first type of encrypted objects and other types of objects, to which are added information indicating which objects have been encrypted.

The invention also relates to a method for protecting a compressed video stream with an H.264 standard, characterized in that it comprises, during step c), at least the following steps:

> analyze the video stream in the compressed domain,

define at least one first group of objects containing zones or objects to be encrypted in said stream, and identify the portions of the image containing them, the analysis module that identifies the portions of the image belonging to the first type from objects to protect sends the motion estimation vectors and the transformed coefficients to the encryption step,

> encrypt the portions identified by the previous step,

> for a given image or group of images, create an undefined NAL type network transport unit (described in the standard as "undefined NAL", which will convey the identification of the encrypted image portions,

> reconstitute a single compressed stream comprising encrypted and unencrypted zones, as well as the new NAL units identifying the portions of images that have been encrypted.

The first group of objects corresponds, for example, to a first plane comprising moving objects in an image.

It can use an encryption method for modifying within groups of portions or "groups groups" corresponding to the foreground bits allowing decoding with a standard decoder.

It uses, for example, an encryption or encryption method for modifying inside groups of portions or "groups groups" corresponding to the foreground bits that will allow a total decoding in case of use of the encryption key or encryption. The method uses, for example, a function adapted to determine a mask for identifying the blocks of an image or group of images comprising one or more moving objects defined as a plurality of regions of the mask and the other blocks belonging to the background following an analysis in the compressed domain. Since the video sequence is produced by an MPEG-4 part 10 / H.264 standard, the method for defining the portions or "slices groups" uses, for example, the flexible scheduling technique or Flexible Macrocbloc Ordering (FMO) allowing the definition macroblock macroblock groups. The invention also relates to a system for visually encrypting a video sequence characterized in that it comprises at least one unit of video processing adapted to perform the steps of the method according to the method described above comprising at least the following elements:

a means adapted to identify the portions of the video sequence that will be encrypted, a module for encrypting the identified portions,

a buffer memory that receives the other parts of the unencrypted stream,

a module adapted to merge the portion of the compressed encrypted stream and the portion of the unencrypted and compressed stream

> said identification and visual encryption modules communicating with each other.

Other features and advantages of the device according to the invention will appear better on reading the description which follows of an example of embodiment given by way of illustration and in no way limiting attached to the figures which represent:

FIGS. 1 to 4, the results obtained by an analysis in the compressed domain,

FIG. 5, an example describing the steps implemented to add visual encryption to a compressed stream, and FIG. 6, an exemplary diagram for a video processing unit according to the invention.

In order to better understand the operation of the method according to the invention, the description includes a reminder on how to perform an analysis in the compressed domain, as described for example in the referenced US patent application 2006 188013. in Figures 1, 2, 3 and 4 and also in the following two references:

Leny, Nicholson, Loaners, "Motion Estimation for Real-Time Video Analysis in the Compressed Domain", GRETSI, 2007. Leny, Loaner, Nicholson, SPIE Electronic Imaging, "Statistical motion vector analysis for object tracking in compressed video!", San Jose, 2008.

In summary the techniques used inter alia in the MPEG standards and exposed in these articles consist in dividing the video compression in two steps. The first step is to compress a still image. The image is divided into blocks of pixels (of 4x4 or 8x8 according to the MPEG-1/2/4 standards), which are subsequently subjected to a transform allowing a passage in the frequency domain and then a quantization makes it possible to approximate or eliminate the high frequencies to which the eye is less sensitive. Finally, these quantified data are coded entropically. The second step is to reduce temporal redundancy. For this purpose, it makes it possible to predict an image from one or more other previously decoded images within the same sequence (motion prediction). For this, the process searches in these reference images the block that best corresponds to the desired prediction. Only one vector (Vector Motion Estimation, also known as Motion Vector), corresponding to the displacement of the block between the two images, as well as a residual error allowing to refine the visual rendering are preserved.

These vectors, however, do not necessarily correspond to a real movement of an object in the video sequence but can be similar to noise. Different steps are needed to use this information to identify mobile objects. The work described in the aforementioned Leny et al publication, "Motion Estimation for Real-Time Analysis of Videos in the Compressed Domain", and in the aforementioned US patent application have delineated five functions making the analysis in the compressed domain possible, these functions and the means of implementation corresponding to them being represented in FIG. 1: 1) a low-resolution decoder (LRD) allows to reconstruct the entirety of a sequence at the resolution of the block, removing on this scale the motion prediction;

2) a motion estimation vector generator (MEG - Motion Estimation Generator) determines vectors for all the blocks encoded by the encoder in "Intra" mode (within Intra or predicted images);

3) a Low-Res Object Segmentation (LROS) module relies on an estimation of the background in the compressed domain thanks to the sequences reconstructed by the LRD and thus gives a first estimate of the objects mobile;

4) Object Motion Filtering (OMF) uses MEG output vectors to determine moving areas from motion estimation; 5) finally a cooperative decision module (CD - Cooperative Decision) allows to establish the final result from these two segmentations, taking into account the specificities of each module according to the type of analyzed image (Intra or predicted).

The main interest of the analysis in the compressed domain concerns computation times and memory requirements which are considerably reduced compared to conventional analysis tools. Based on the work done at the time of video compression, analysis times are now 10 to 20 times the real time (250 to 500 images processed per second) for images 720x576 4: 2: 0 .

One of the drawbacks of the analysis in the compressed domain as described in the aforementioned documents is that the work is performed on the equivalent of low resolution images by manipulating blocks composed of groups of pixels. As a result, the image is analyzed with less precision than by implementing the usual algorithms used in the uncompressed domain. In addition, objects that are too small in relation to block cutting can go undetected.

The results obtained by the analysis in the compressed domain are illustrated in FIG. 2 which show the identification of zones containing moving objects. Figure 3 schematizes the extraction of specific data such as vector motion estimates and Figure 4 obtained low resolution confidence maps corresponding to the contours of the image.

FIG. 5 schematizes an example of implementation of the method according to the invention during which zones of an image contained in a compressed video stream will be visually encrypted. This step, performed by encrypting certain bits using, for example, the method described in the article by Bergeron et al supra, will be executed on the compressed video stream. Some areas in the image with a higher level of privacy will be selected and encrypted to avoid direct access to the data they contain.

The compressed video stream 1 at the output of an encoder is transmitted to a first analysis step 2 whose function is to identify moving objects. Thus, the method generates a sequence of masks comprising regions having received an identical label or blobs related to moving objects.

This analysis in the compressed domain made it possible to define for each image or for a group of GoP defined images, on the one hand, different zones Z1 i belonging to the first plane P1 and other zones Z2i belonging to the second plane P2 of a video image. The analysis can be carried out by implementing the method described in the aforementioned US patent application. However, any method making it possible to obtain an output of the analysis step in the form of masks per image, or any other format or parameters associated with the compressed video sequence analyzed, may also be implemented in place of the analysis step in the proposed compressed domain. At the end of the analysis step, the method has, for example, bit masks for each image (block resolution or macroblock). An example of a convention used may be the following: "1" corresponds to a block of the image belonging to the foreground and "O" corresponds to a block of the image belonging to the background. From these masks, the groups of portions of the image, better known by the English name "slice group", which comprise one or more blocks identified as belonging to the foreground are selected in view of the encryption step. This corresponds to step 2 of FIG. 5, the results being the groups of identified portions (3a and 3b). The image is thus segmented according to its semantic content. In a more general application framework, it will be possible to define not two zones, but several types of objects which will give rise to a visual encryption application according to their importance and their sensitivity, for example by the use of different keys. According to an implementation variant as indicated above, it is also possible to process the enclosing boxes of moving objects. The bounding box coordinates correspond to the moving objects and are calculated using the mask. These boxes can be defined by two extreme points, a central point associated with the size of the box, etc. One can also have a set of image coordinates or a set for the entire sequence with trajectory information (date and point of entry, curve described, date and exit point). We will search for the slices of the image at least partially covered by one of the bounding boxes corresponding to the foreground. Masks can be binary masks. The portions of the image 3a that do not require visual encryption or encryption are simply copied back into the temporary buffer 4. On the other hand, the method processes, 5a and 5b, the zones of the image comprising these mobile objects in order to encrypt them. visually by means of an encryption key, for example. The encrypted and unencrypted portions are then merged, 6. Information indicating which parts of the stream have been encrypted are added to differentiate them at the decoder. This addition is carried out using techniques known to those skilled in the art. In this implementation example, the encryption or encryption step applies, on the groups of slots identified in step 2, the method described in the Bergeron et al article or in the patent application WO 2006/067172. . The latter being designed to encrypt the data of a compressed stream, the visual encryption can be done directly within the compressed stream without going through other steps than that of interpretation of the flow, or "parsing" according to the English expression -Saxonne, which is already done prior to the analysis in the compressed domain. This makes it possible to obtain results with very little computing time and memory resources.

In the case of the H.264 standard, a NAL, type 30 or 31 undefined within which it is possible to transmit any type of information, will receive the identification of the portion groups that have been visually encrypted. Unlike other types of NALs, NALs 30 and 31 are not reserved either for the flow itself or the RTP-RTSP type network protocols. A standard decoder will simply set aside this information whereas a specific decoder, developed to take into account this type of NALs, may choose to use this information to detect and possibly decrypt the protected areas. In the case of a stream that would have been encoded according to the recommendations contained in the Applicant's patent application filed on the same day and entitled "construction of a new image structure based on semantic objects within a video stream compressed and the associated device ", the nature of the objects or groups of objects (encrypted or not) can be included directly in the corresponding header. This new structure is based on a hierarchy of the image in: Image, Group of images or Images, portions or "slices". The two buffers 4, 5b containing the parts of the encrypted and unencrypted streams are then merged, together with the determined NALs, to reconstruct a single stream 6 which is now visually partially encrypted.

It should be noted that the encryption method used preserves the structure of the flow from the various headers to the possibility of decoding the data via the variable length code (VLC). present in H.264. Thus, a conventional decoder will therefore consider a normal stream, without encryption particularity, and thus provide a decoded sequence presenting a part of the image without specificity (by default the background) and another encrypted part (groups of portions or "slices" Including moving objects). In comparison, a specific decoder will identify the "undefined NAL" including identification of the portion groups that have been encrypted. The decryption key will allow a visualization of the totally intelligible sequence. This allows, for example, in the context of video surveillance to transmit the encrypted stream to the guard station where security officers can identify the presence of a person, a car, a truck, etc. through the silhouettes without being able to directly identify the person's face or the vehicle's license plate, which can be viewed by a security officer, police department, etc. This feature allows a wider diffusion of surveillance flows while respecting the constraints of privacy.

This method is implemented within a video processing unit shown schematically in FIG. 6. The processing unit 10 comprises a tool for detecting the portions of the video stream or the image to be encrypted 1 1 adapted to determine the different areas of interest of the image, ie the areas of the image that must be encrypted. The unit 10 comprises a module 13 for encrypting the identified portions, and a buffer memory 12 which receives the portion of the encrypted stream combined with the other parts of the unencrypted stream into a partially encrypted stream and another module 14 for merging the two parts of the stream, the encrypted and compressed part and the unencrypted and compressed part.

Two main seemingly independent steps constitute the present invention: visual analysis and encryption. In concrete terms, these different modules can communicate with each other to optimize the entire processing chain:

- For the analysis in the compressed domain, it is necessary to unencapsulate the flow, to format the data (the parser) and finally to perform an entropy decoding. The motion estimation vectors and the transformed coefficients are thus obtained.

These modules are also needed for encryption but will not have to be reiterated.

- The analysis module that identifies the portions of the image belonging to the regions of interest sends these parameters to the encryption brick accompanied by the previously obtained data.

- For the visual encryption part itself, once again the transformed coefficients and motion estimation vectors are necessary. The proposed method also makes it possible to dispense with the step of desencapsulation and entropy decoding since the information flows from module to module.

- Once these steps are processed, only then the new entropy coding (which can concretely start during encryption) and the encapsulation of the stream take place

The invention thus allows more than a simple juxtaposition of functions processing a video stream in series: feedback loops are possible and all the redundant steps between the intervening modules are only present once.

The division into portions and groups of portions is not managed by the method, which will have to apply the proposed treatments on the partition of the existing image within the compressed video stream 1. Therefore, the results may vary depending on the type of standard and encoder involved in the compression of the sequence. If the image is for example simply vertically divided into three portions, the process will then encrypt one, two or even three thirds of the image. Although the method works in this case, the results that result are unattractive. On the other hand, with an encoder implementing H.264 in which a group of slices is assigned to each mobile object, the results correspond to an encryption of the mobile objects only, even allowing an identification of the type of object by its shape and contour ( car, pedestrian, etc.). For example, a video stream having a structure such as that described in the applicant's patent application the Applicant's patent application filed on the same day as the present application and entitled "building a new image structure based on objects" semantics within a compressed video stream and the associated device "provides optimal results. The independent reconstruction of objects ensures a visual encryption closer to the object, without impact on the intelligibility of the rest of the scene.

Without departing from the scope of the invention, other techniques for encrypting zones in an image to ensure the confidentiality of the information they contain can be used, as long as they do not call into question the fact that the stream can be decoded by an implementation according to the standard considered.

This operation can also be performed in parallel with a target bit rate order of the moving objects in the sequence. It will then be possible to add robustness to errors (which increases the bit rates) coupled to an adaptation to a target bit rate by optimization on the fly on the already compressed stream. This coupling will for example maintain a fixed rate while protecting the content related to the foreground.

Claims

1 - Method for protecting at least part of a video stream or a compressed video sequence against a violation of the information contained in said stream, said stream being able to be decomposed into at least a first type of objects and a second type of object, the method applying to a selection of images contained in said video sequence characterized in that it comprises at least the following steps: a) analyzing the video sequence in the compressed domain in order to define for an image datum N at least one first type of objects to be protected by encryption and a second type of objects, (1, 2), the analysis module which identifies the portions of the image belonging to the first type of objects to be protected sends the motion estimation vectors and the transformed coefficients to the encryption step; b) determining, according to the division into portions or groups of portions of the existing image N (3b, 3a), the portions or groups of Corr portions respectively responding to the first type of objects to be protected (3b) and the other types (3a), c) encrypting at least a portion of the first type of objects (5a), and d) reconstructing from the outputs of the previous steps a stream consisting of at least a first type of encrypted objects (5a, 5b) and other types of objects (4), to which are added information indicating which objects have been encrypted.

2 - Process according to claim 1 for a stream compressed with an H.264 standard characterized in that it comprises during step c) at least the following steps:

> analyze the video stream in the compressed domain (2), defining (2, 3) at least one first group of objects containing zones or objects to be encrypted in said stream, and identifying the portions of the image containing them,

> encrypt the portions identified by the previous step,> for a given image or group of images, create an undefined NAL type network transport unit (described in the standard as "undefined NAL", which will convey the identification of encrypted image portions,

3 - Method according to one of claims 1 or 2 characterized in that the first object group corresponds to a first plane comprising moving objects in an image.

4 - Process according to claim 3 characterized in that it uses an encryption method for modifying inside groups of portions or "groups groups" corresponding to the foreground bits for decoding with a standard decoder.

5 - Process according to claim 4 characterized in that it uses an encryption method for modifying inside the groups of portions or "groups groups" corresponding to the foreground bits that will allow a total decoding in case of use the encryption or encryption key.

6 - Process according to claim 1 or 2 characterized in that it uses a function adapted to determine a mask for the identification of the blocks of an image or group of images comprising one or more mobile objects defined as a plurality of regions of the mask and the other blocks belonging to the background following an analysis in the compressed domain. 7 - Method according to one of the preceding claims characterized in that the video sequence being produced by a standard MPEG-4 part 10 / H.264, the method for defining the portions or "slices groups" using the flexible scheduling technique or FMO (Flexible Macrobloc Ordering) allowing the definition of groups macroblock groups by macroblock.

8 - System for visually encrypting a video sequence characterized in that it comprises at least one video processing unit (10) adapted to perform the steps of the method according to one of claims 1 to 7 and comprising at least the following elements :

means (12) adapted to identify the portions of the video sequence that will be encrypted,

a module (13) for encrypting the identified portions, a buffer memory (12) which receives the other parts of the unencrypted stream,

a module (14) adapted to merge the portion of the compressed encrypted stream and the portion of the unencrypted and compressed stream,