KR102575244B1 - Apparatus and method for encrypting privacy areas from compressed video - Google Patents

Abstract

The present invention relates to a masking technique for encrypting only compressed data corresponding to a partial region derived by image analysis without re-encoding the entire image.
A privacy area encryption apparatus for receiving a compressed bitstream and generating a secure bitstream by applying encryption to at least a portion of an image included in the compressed bitstream decodes the received compressed bitstream to restore A video decoder that generates a video; an image analysis unit that analyzes the restored video to set a privacy area; an encryption unit that encrypts blocks belonging to the privacy area in the restored video; and a bitstream regenerator for generating the secure bitstream by replacing blocks corresponding to encrypted blocks with the encrypted blocks.

Description

Apparatus and method for encrypting privacy areas from compressed video

The present invention relates to a technology for masking a security area in a captured video image, and more particularly, to a masking technology for encrypting only compressed data corresponding to a partial area derived by image analysis without newly encoding the entire image. will be.

In general, since CCTV cameras target a wide range of areas such as buildings, public places, outdoors, and roads, there is a high risk of invasion of privacy due to the captured images. In particular, if high-resolution cameras such as FHD and UHD are installed in places that handle sensitive information such as passwords, addresses, and names, such as banks and post offices, there is a high risk of personal information being exposed. Therefore, technologies to prevent invasion of personal life by deleting information related to personal information or encrypting the part in CCTV images have been developed and used.

As described above, conventional methods of protecting privacy in captured images are largely divided into a privacy masking method and a method of encrypting only a privacy area. First, as a privacy masking method, a method of overlaying an original image in a player using Privacy Masking region information to prevent viewing of the corresponding region, a method of overwriting the privacy region on the original image and removing the corresponding region, and a method of removing the corresponding region from the original image There is a method of overwriting the privacy area and separately providing information about the area.

The method of making a corresponding region invisible through overlay processing has a limitation in that a dedicated player must be used to implement a masking effect because the overlay is processed in a player. In addition, the method of removing the privacy area from the original video has a problem in that the corresponding area cannot be identified even when necessary, such as for a criminal investigation, because the privacy area is completely removed. In addition, the method of providing separate information about the area removed by overwriting has a limitation in that only a predefined area can be used because the privacy area must be determined before image compression is performed by the camera. If it is desired to detect a dynamically changing privacy area, an excessive computational load will be generated in the camera because image analysis must be performed first.

On the other hand, as a method of encrypting the privacy area, a method of encrypting the header part of the compressed video so that the image cannot be viewed, a method of encrypting the entire compressed video so that the image itself cannot be viewed, and only part of the frequency data of the compressed video are encrypted how to, etc. However, since all of these methods require a password to view encrypted data and a dedicated player, there is a limitation that no video can be viewed under normal circumstances.

Since the privacy area corresponds to a part of the video image, it is necessary for general users to be able to check the corresponding video image without a dedicated player and password. In general, it is not realistically possible to perform image analysis for determining the privacy area at the camera end, and it is a task that can be performed only in a high-performance device such as a server. This is because it is much more advantageous to use a general-purpose infrastructure such as a PC or server than to mount a high-performance chip on the camera stage from the perspective of cost versus computational load.

Therefore, there is a post-processing request for detecting type information by OCR (Optical Character Recognition) processing at the server side and dynamically masking only the corresponding area. However, even with this post-processing method, if an image needs to be re-encoded, system load increases and image quality deteriorates.

Under these constraints, a general user can easily view the rest of the video image except for the privacy area, and if an authorized person needs to check the privacy area, it is possible to view the entire area of the video image through a dedicated player and password. You need a one-dimensional structure. In other words, it is necessary to separate general use and special authority so that there is no inconvenience in general use, and those who have special authority can use it completely.

Korean Registered Patent Publication No. 1559647

In view of the above needs, the problem to be solved by the present invention is to achieve the purpose of masking by encrypting only compressed data corresponding to a partial region derived by image analysis without newly encoding the entire image.

Another problem to be solved by the present invention is to compress video by applying privacy masking to a specific block area of already compressed video data in consideration of the properties of a typical video compression technique that performs video compression in units of predetermined-sized blocks. It is intended to overcome the limitations of the prior art in which encryption and encryption are separately applied.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention for achieving the above technical problem, a privacy area for receiving a compressed bitstream and generating a secure bitstream by applying encryption to at least a part of an image included in the compressed bitstream The encryption device may include: a video decoder for decoding the received compressed bitstream to generate a reconstructed video; an image analyzer configured to analyze the restored video and set a privacy area; an encryption unit encrypting blocks belonging to the privacy area in the restored video; and a bitstream regenerator generating the security bitstream by replacing blocks corresponding to the encrypted blocks in the compressed bitstream with the encrypted blocks.

According to the present invention, there is an advantage in that a general-purpose player can view the masked image and at the same time, use a password to view the entire unmasked image through a dedicated player if necessary.

In addition, according to the present invention, there is no need to determine the masking area in the camera before image compression, masking can be applied even after image compression, and there is no need to re-encode the entire image even after masking is applied. Therefore, through the elimination of this re-encoding step, it is possible to improve overall processing speed and maintain image quality.

In addition, according to the present invention, there is an advantage that a high-end camera capable of processing image analysis is not required, and at the same time, the reliability of image analysis is increased at the server end, so that dynamic masking can be applied.

1 is a block diagram showing an entire system according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a video encoder according to an embodiment of the present invention.
3 is a block diagram showing the configuration of a video decoder according to an embodiment of the present invention.
4 is a diagram showing an example in which a video image is divided into blocks.
5 is a diagram showing an intra prediction method for a 4x4 block used in H.264.
6 is a diagram showing an example in which encryption is selectively applied to an 8x8 block included in a privacy area.
7 is a diagram showing the concept of regenerating a compressed bitstream into a secure bitstream.
8 is a diagram showing a reconstructed image in which a privacy area is blinded.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing an entire system according to an embodiment of the present invention. The system basically consists of a network camera 190 and a device 290 that encrypts a privacy zone from compressed video.

The network camera 190 encodes an image signal obtained by photographing a subject, generates a compressed video (compressed bitstream), and provides the compressed video (compressed bitstream) to the privacy area encryption device 290 .

Specifically, the network camera 190 includes an imaging device 105 that acquires an image signal from a subject, a video encoder 100 that encodes the image signal to generate a compressed bitstream, and transmits the compressed bitstream to a network It may be configured to include a transceiver 115 for transmitting to the privacy area encryption device 290 or receiving a control command related to photographing from the privacy area encryption device 290.

The imaging device 105 may include an image capturing device such as a Charge Coupled Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) that receives an image signal from a subject. Also, the image signal may be a digital signal represented by three color components such as RGB and YUV.

The video encoder 100 generates a compressed bitstream from the video signal according to various video coding standards such as MPEG-2, MPEG-4, H.264, and HEVC (H.265). 2 is a block diagram showing the configuration of a video encoder 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the video encoder 100 includes a picture division unit 110, a subtractor 111, a transform unit 120, a quantization unit 130, a scanning unit 131, an entropy encoding unit 140, a picture It includes a restoration unit 150 and a prediction unit 160.

The picture divider 110 analyzes the input video signal and divides the picture into blocks of a predetermined size. The unit of this division may be a variable block size including 16x16, 8x8, and 4x4, as in H.264, but may have a larger and more diverse block size as in HEVC.

The subtraction unit 111 generates a residual block by subtracting the prediction block provided from the prediction unit 160 from the divided original block.

The transform unit 120 spatially transforms the residual block to generate transform coefficients having frequency components. For the spatial transformation, discrete cosine transform (DCT), discrete sine transform (DST), wavelet transform (WT), or the like may be used.

The quantization unit 130 determines a quantization step size for quantizing the transform coefficients for each coding unit. Then, quantization coefficients are generated by quantizing the coefficients of the transform block according to the determined quantization step size.

The scanning unit 131 scans the quantization coefficients (two-dimensional array) in a predetermined manner (zigzag, horizontal, vertical scan, etc.) and converts them into one-dimensional quantization coefficients.

The entropy encoding unit 140 generates a compressed bitstream by entropy encoding (lossless encoding) the one-dimensional quantization coefficients scanned by the scanning unit 131 and the prediction information provided by the prediction unit 160. The prediction information means information according to intra prediction or inter prediction, and specifically means mode information in intra prediction or motion vector and reference picture information in inter prediction.

On the other hand, according to a normal closed-loop coding method, a picture is reconstructed through transformation and quantization, and then inverse quantization and inverse transformation without using the original picture itself as a reference picture, and the reconstructed picture is converted into another picture. Or, it is used as a reference for the same picture. Using another part of the same picture as a reference is called intra prediction, and using another picture as a reference is called inter prediction.

The picture reconstruction unit 150 performs inverse quantization and inverse transformation again on the 2-dimensional quantization coefficients obtained through the transformation and quantization to obtain a reconstructed picture (or part of the picture). The reconstructed picture is provided to the prediction unit 160, and the prediction unit 160 generates a reference picture by a prediction method that is advantageous between intra prediction and inter prediction in terms of rate-distortion (R-D) cost, and uses the subtractor 111 to generate a reference picture. provided to

Referring back to FIG. 1 , the compressed bitstream generated by the video encoder 100 is transmitted to the privacy area encryption device 290 by the transceiver 115 . The transceiver 115 may be a wired adapter supporting wired communication methods such as Ethernet, universal serial bus (USB), and power line communication (PLC), wireless LAN (WLAN), long term evolution (LTE) , it may be a wireless adapter supporting a wireless communication method such as Bluetooth.

Meanwhile, the privacy area encryption apparatus 290 receiving the compressed bitstream includes a transceiver 205, a video decoder 200, an image analysis unit 215, an encryption unit 235, and a bitstream regeneration unit 245. ).

The transceiver 205 receives the compressed bitstream from the network camera 190 through a network or transmits a control command related to capturing to the network camera 190 . The transceiver 205, like the transceiver 115 of the network camera 190, is a wired adapter supporting a wired communication method such as Ethernet, USB, or PLC, or a wired adapter supporting a wireless communication method such as WLAN, LTE, or Bluetooth. It could be a wireless adapter.

The video decoder 200 decodes the compressed bitstream provided through the transceiver 205 to generate a restored video. To this end, as shown in FIG. 3, the video decoder 200 includes an entropy decoding unit 210, an inverse scanning unit 220, an inverse quantization unit 230, an inverse transform unit 240, an adder 241, and It may include a processing unit 250 and a prediction unit 260.

The entropy decoding unit 210 decodes the compressed bitstream and separates it into prediction information and quantization coefficient information. The entropy decoding unit 210 supplies the decoded prediction information to the prediction unit 260 and supplies the decoded quantization coefficient information to the inverse scanning unit 220 . The prediction information means information according to intra prediction or inter prediction, and specifically means mode information in intra prediction or motion vector and reference picture information in inter prediction.

The inverse scanning unit 220 converts the quantization coefficient information into a 2D array of inverse quantization blocks. At this time, one of a plurality of inverse scanning patterns (eg, zigzag, vertical, horizontal scan) is selected for the conversion, which corresponds to the scanning pattern performed by the video encoder 100.

The inverse quantization unit 230 determines the quantization step size of the current coding unit. The step size is the same as the quantization step size determined by the quantization unit 130 of the video encoder 100.

The inverse quantization unit 230 restores an inverse quantization coefficient, that is, a transform coefficient, using the determined quantization step size, and the inverse transform unit 240 inverse spatially transforms the transform coefficient to restore a residual block.

The prediction unit 260 generates a prediction block according to an intra prediction mode or an inter prediction mode of the current block based on the prediction information received from the entropy decoding unit 210 .

The adder 241 restores an image block by adding the residual block reconstructed by the inverse transform unit 240 and the predicted block generated by the predictor 260.

Finally, the picture storage unit 250 combines the image blocks reconstructed by the adder 241 to generate completed pictures, and these completed pictures compose a restored video.

Again, referring to FIG. 1, the video image restored by the video decoder 200 is provided to the image analyzer 215, and the image analyzer 215 analyzes the video image to set a privacy area. . The selection of such a privacy area may be performed by a user directly checking an image, but it is preferable to automatically perform the selection through image analysis. In order to automatically perform image analysis, image-based text recognition such as OCR should be performed, and based on the performed text recognition, it should be determined whether it is privacy information such as vehicle number, phone number, or resident registration number. An algorithm for determining whether a specific area includes privacy information through image analysis may be applied with various techniques known in the related art, so a detailed description thereof will be omitted.

However, the video analyzer 215 uses information on blocks divided by the existing video encoder 100 in determining such a privacy area. The image analyzer 215 may obtain information on block division from the header of the compressed bitstream 195 .

4 shows an example in which a video image is divided into blocks. In a video coding standard using a variable block size such as H.264 or HEVC, a technique in which the size of a block is varied according to the complexity of an image is adopted. That is, in FIG. 4, an area with a monotonous image, such as the background, is assigned a slightly larger block size, and an area with a complex image, such as the edge of an object, is assigned a slightly smaller block size. In general, since the privacy area is likely to be a complex area rather than a monotonous area, it may be possible to set the area in more detailed units.

Anyway, since the present invention does not go through the process of re-encoding after restoring the compressed bitstream, it is necessary to use the block information used for video encoding as it is when the video analyzer 215 determines the privacy area. . Therefore, in the present invention, the privacy area can be regarded as including at least one or more coding blocks.

As described above, when the image analyzer 215 determines a privacy area from the restored video image, images (pixel values) of blocks corresponding to the privacy area are provided to the encryption unit 235. The encryption unit 235 applies encryption only to images of blocks corresponding to the privacy area. As such an encryption algorithm, various known schemes may be used. For example, symmetric key encryption algorithms such as DES, 3DES, RC4, AES, and SEED may be used, or asymmetric key encryption algorithms such as RSA, ECC, DSA, KCDSA, and PKI may be used.

However, due to the characteristics of video coding, there are additional considerations in encryption of a specific block. If an entire specific block is encrypted as it is, if there is another intra-prediction block to which the block must be referenced, errors may accumulate during video reconstruction. In this case, not only the area set as the privacy area but also the area unrelated to the area is not properly restored. That is, it is necessary to solve the problem of error propagation due to inter-block dependency destruction.

5 is a diagram showing an intra prediction method for a 4x4 block used in H.264. Here, a prediction block (region including a to p) for the current block is generated using reference samples (A to M) belonging to previously reconstructed adjacent blocks (left block, upper block, and upper left block). H.264 provides a total of nine intra prediction modes, and reference samples are referenced in different ways according to the nine directions to generate prediction blocks. If the current block is encoded using the intra prediction method, if there are no reference samples around the current block, the current block cannot be properly restored even if it does not belong to the privacy area.

Therefore, in the present invention, even if encryption is applied to a specific block, it is preferable not to apply encryption to pixel values adjacent to the right and lower boundaries belonging to the block rather than to encrypt the entire block.

6 is a diagram showing an example in which encryption is selectively applied to an 8x8 block included in a privacy area. As shown in FIG. 6, when it is assumed that a block includes a total of 64 pixel values, the encryption unit 235 excludes 15 pixels 65 adjacent to the right and lower borders of the block, and the remaining 49 pixel values. Encryption can be applied only to pixels 60 . At this time, in order to maintain the original values of the border pixels, the 8x8 block is replaced with the PCM mode, the remaining pixel values 60 are encrypted, and the adjacent pixel values 65 of the right and lower borders are stored as original values. do. The PCM mode refers to data that has not been subjected to a spatial conversion process of converting into frequency data in the conversion unit 120 of the video encoder 100, that is, data in the spatial domain. Here, while adjacent pixel values 65 are stored as original values, only encryption may be applied to the remaining pixel values 60, but compression (video encoding) may be applied before such encryption.

Referring back to FIG. 1 , the bitstream regeneration unit 245 converts blocks encrypted by the encryption unit 235 into blocks set as a privacy area by the image analysis unit 215, among the compressed bitstream, to the compressed bitstream. A secure bitstream 255 is generated by replacing and inserting the corresponding position of the bitstream. Referring to Figure 7, the original compressed bitstream 190 is composed of a number of coding blocks (310 to 380). Here, if the coding blocks 320, 350, and 370 belong to the privacy area, the coding blocks 320, 350, and 370 are encrypted by the encryption unit 235, and then the encrypted coding blocks 325, 355, 375) will be converted to At this time, the bitstream regenerator 245 generates a secure bitstream 255 by replacing and inserting the encrypted coding blocks 325, 355, and 375 into the existing coding blocks 320, 350, and 370. do.

The security bitstream 255 generated in this way may be restored as different images depending on whether there is an authority to decrypt encryption. If an ordinary user without such permission restores an image from the secure bitstream 255, the encrypted coding blocks 325, 355, and 375 in FIG. 7 will be displayed as broken blocks that cannot be restored. Accordingly, a general user will be able to view an image in which the privacy area is blinded as shown in FIG. 8 . However, since all areas other than the privacy area are properly displayed, even a general user has no difficulty in grasping the content of the restored video image.

On the other hand, if an administrator has decryption authority, since the encrypted coding blocks 325, 355, and 375 in the secure bitstream 255 can be restored normally, a complete image with the blinds removed shown in FIG. 8 can be viewed.

Up to now, each component of FIGS. 1 to 3 is software such as a task, class, subroutine, process, object, execution thread, or program performed in a predetermined area of memory, or a field-programmable gate array (FPGA). It may be implemented in hardware such as an application-specific integrated circuit (ASIC) or a combination of the above software and hardware. The components may be included in a computer-readable storage medium, or some of them may be distributed and distributed to a plurality of computers.

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Also, in some alternative implementations, it is possible for the functions mentioned in the blocks to occur out of order. For example, two blocks shown in succession may in fact be executed substantially concurrently, and it is also possible that the blocks are sometimes executed in reverse order depending on their function.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art can realize that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: video encoder 105: imaging device
110: picture divider 111: subtractor
120: conversion unit 130: quantization unit
131: scanning unit 140: entropy encoding unit
150: picture restoration unit 160, 260: prediction unit
115, 205: transceiver 190: network camera
195 compressed bitstream 200 video decoder
210: entropy decoding unit 220: inverse scanning unit
230: inverse quantization unit 240: inverse transformation unit
241: adder 250: picture storage unit
215: image analysis unit 235: encryption unit
245: bitstream regeneration unit 255: secure bitstream

Claims (6)

A privacy area encryption apparatus for receiving a compressed bitstream and generating a secure bitstream by applying encryption to at least a portion of an image included in the compressed bitstream,
a video decoder for decoding the received compressed bitstream to generate a reconstructed video;
an image analyzer configured to analyze the restored video and set a privacy area;
an encryption unit encrypting blocks belonging to the privacy area in the restored video; and
A bitstream regenerator for generating the secure bitstream by replacing blocks corresponding to the encrypted blocks in the compressed bitstream with the encrypted blocks;
The video analyzer obtains information about division of the blocks from a header of the compressed bitstream, and sets the privacy region so that the privacy region includes the blocks using the information;
Wherein the secure bitstream is composed of the encrypted blocks and some of the blocks included in the compressed bitstream. According to claim 1,
A size of blocks belonging to the privacy region is the same as a size of compressed blocks included in the compressed bitstream. According to claim 1,
wherein the encryption unit encrypts pixel values other than pixel values adjacent to right and lower boundaries of the blocks when encrypting the blocks belonging to the privacy region. According to claim 3,
Pixel values adjacent to the right and lower boundaries are pixel values to which spatial transformation is not applied according to a PCM mode. The method of claim 1, wherein the security bitstream,
Encryption so that a user without encryption decryption permission can view a video image to which privacy masking is applied from the security bitstream, and a user with encryption decryption permission to view a secure video image to which privacy masking is not applied from the security bitstream An apparatus for encrypting a privacy area, including encrypted blocks and unencrypted blocks together. A privacy region encryption method for receiving a compressed bitstream and generating a secure bitstream by applying encryption to at least a portion of an image included in the compressed bitstream,
decoding the received compressed bitstream to generate a reconstructed video;
setting a privacy area by analyzing the restored video;
encrypting blocks belonging to the privacy area in the restored video; and
Generating the secure bitstream by replacing blocks corresponding to the encrypted blocks in the compressed bitstream with the encrypted blocks,
The setting of the privacy region may include obtaining information about partitioning of the blocks from a header of the compressed bitstream, and setting the privacy region so that the privacy region includes the blocks using the information. including steps,
The privacy area encryption method of claim 1 , wherein the secure bitstream is composed of the encrypted blocks and some of the blocks included in the compressed bitstream.