KR20130114037A - Masking and recovering method of privacy region - Google Patents

Abstract

PURPOSE: A masking and restoring method of a privacy domain is provided to minimize the amount of data which is stored or transmitted. CONSTITUTION: Masked image data (MID) is generated by decoding encoded masked image data (EMID) (S21). Privacy data is generated by decoding encrypted privacy data (EPD) with a key (S22). In case a user does not have the key, masked image is displayed on a screen using the MID. In case the user has the key, an original image having a restored privacy domain is displayed on the screen using the information of the privacy domain, the MID and the PD. [Reference numerals] (AA) Start; (BB) End; (S21) Decode encoded masked image data; (S22) Decode encrypted privacy data using a key; (S23) Recover the original image of a privacy area by deducting the decoded privacy data from a saturation value; (S24) Obtain overall data about the original image data based on information about the privacy area's coordinates, size, and time; (S25) Display the masked image data or the original image data

Description

Masking and Recovering Method of Privacy Region

The present invention relates to a method for masking a face or license plate area corresponding to a privacy part in a captured video and restoring an original image.

When playing video recorded on video cameras such as CCTVs, network cameras, or black boxes (including the black box function of smartphones), anyone can see the privacy area such as the face or vehicle number of others. It is becoming a social issue.

In order to solve this problem, a privacy area such as a face or a vehicle number is often masked and reproduced in a captured image so that it cannot be seen during playback.

However, when an accident occurs, a specific user, such as a police officer or a security officer, needs to check an identity of a relevant person's face or vehicle number through the original image where the privacy zone is restored. Since the masking process is crushed, it is impossible to restore the privacy area during playback and the original image cannot be viewed.

In order to solve the problems of the related art, another conventional technology inserts QR codes related to privacy areas for each frame of a video (the privacy area related information such as the location, size, and time information of the privacy areas is inserted into the code). The data of the privacy area is scrambled to store the scrambled image and the QR code in the internal memory card or to the playback device via a network.

In addition, during playback, unscrambled scrambled privacy data is used to restore the privacy data, and by using the privacy region information present in the QR code, the privacy region of the image whose privacy region is scrambled is filled with the privacy data to restore the original image. In this case, the restored original image is shown to the specific user and the image including the scrambled privacy area is shown to the general user.

By the way, this other conventional technique is simple, but since the scrambled data and the QR code are not encoded and restored, the data cannot be compressed and the data volume is large. In the case of network transmission, however, the network is heavily loaded.

Therefore, in order to reduce the amount of data, such other conventional techniques have to shoot at a low resolution and a low frame rate, which is a fatal drawback for recent video recording apparatuses that require high resolution and a high frame rate. It is also difficult to identify yourself with a video.

SUMMARY OF THE INVENTION The present invention provides a privacy masking and restoring method that minimizes the amount of data to be stored or transmitted in view of the problems of the related art.

In order to achieve the above technical problem, this invention,

In the privacy masking and restoration method of the captured image,

Extracting a privacy region from the captured original image and generating information on the extracted privacy region;

Filling the extracted privacy region with a predetermined masking RGB value and performing compression encoding on the filled image to form encoded and masked image data (EMID);

Encrypting the extracted privacy area with a key to form encrypted privacy data (EPD);

Storing information about the privacy area, the encoded and masked image data (EMID), and the encrypted privacy data (EPD) in an internal storage device or transmitting to an external device through a network;

Decoding the encoded and masked image data (EMID) to form masked image data (MID);

Decrypting the encrypted privacy data (EPD) using the key to form privacy data (PD);

If the user does not have the key, the masked image is displayed on the screen using the masked image data (MID), and if the user has the key, information on the privacy area and the masked image are displayed. And displaying the original image on which the privacy area is restored using the image data MID and the privacy data PD.

By using the present invention as described above, it is possible to minimize the amount of data to be stored or transmitted at the time of masking and restoration, it is possible to store a long time video or to significantly reduce the load on the transmission network.

Figure 1 shows the overall configuration of the present invention.
Figure 2 shows a specific configuration of the video recording apparatus in the present invention.
Figure 3 shows a specific configuration of the video playback apparatus in the present invention.
4 shows a general method of detecting a face or license plate.
5 is a flowchart showing a masking method of the present invention.
6 is a flowchart showing a reproducing method of the present invention.

First, the overall configuration of the present invention will be described with reference to FIG.

In the video photographing apparatus 1, each frame image is photographed by a camera (S1), masking a portion requiring privacy protection such as a face or a license plate (S2) and compression encoding (S4) (e.g., an H.264 encoder Use) to form encoded and masked image data (EMID).

Then, the privacy region is extracted from the captured image (S3), and the extracted privacy portion is encrypted using a key (S5) to form encrypted privacy data (EPD: Encrypted Privacy Data).

Now, the encoded and masked image data (EMID) and encrypted privacy data (EPD) formed in the photographing apparatus 1 are stored in a memory card (e.g., an SD card) embedded in the photographing apparatus 1, and then stored in a user. This memory card is attached to the playback device 2 or transferred to the playback device 2 by a network 16 such as the Internet as shown in FIG.

Then, in the playback device 2, if the user is a general user without a key, the encoded and masked video data EMID is decoded (S6) (e.g., using an H.264 decoder) and the masked video data MID. To form a masked image on the screen.

In addition, if the user is a specific user with a key, the encrypted privacy data (EPD) is decrypted with the key (S7) to form the privacy data PD, and the privacy image is synthesized with the masked image to show the original image. .

At this time, since the encoded and masked image data (EMID) is compressed and encoded, the amount of data can be greatly reduced, thereby greatly reducing the data capacity stored in the memory card or significantly reducing the network load when transmitting data through a network.

In addition, since privacy areas such as faces and license plates generally occupy only a small portion of the entire screen, the amount of data in the privacy area is small, so that the privacy data (EPD) encrypted by only encryption without compression encoding for the privacy area as in the present invention is used. Even if the data is formed and stored or transmitted, the increase in storage capacity or increase in network load due to the encrypted privacy data (EPD) becomes very small.

When the masked image data (MID) is compressed and encoded, there is a slight data loss during decoding, so that the masked image provided to the general user appears to be slightly blurred compared to the original image, which is not detectable by the general user. This is not a problem.

On the other hand, since there is no compression encoding on the privacy area, during playback, a certain user sees the privacy area as it is without loss and the rest is slightly blurry, so that a particular user has no difficulty in seeing the entire image. In particular, the privacy area can be viewed as it is, so that the face or license plate corresponding to the privacy area can be accurately recognized.

On the other hand, by modifying the embodiment of Fig. 1, a compression encoding step is added between the privacy partial extraction step S1 and the encryption step S5, and a decoding step is added immediately after the privacy partial decryption step S7. The amount of data in the area can be greatly reduced, but since the privacy area occupies only a small portion of the entire screen and the original data amount is small, such a large reduction in the amount of data in the privacy area itself does not make a big difference. The processing load increases by the step.

Next, the configuration of the video recording apparatus 1 shown in FIG. 1 will be described in detail with reference to FIG.

The image detector 11 of the moving image photographing apparatus 1 detects an original image by a camera.

In addition, the masking processing unit 12 extracts an area (privacy area) requiring privacy protection such as a face or a license plate by using various known algorithms, and performs masking processing on the original image.

The encoder 13 receives the masked image from the masking processor 12 and compressively encodes the masked image in the same manner as in H.264 to form encoded and masked image data EMID.

In addition, the encryption unit 14 encrypts the privacy region extracted by the masking processing unit 12 using a key to form encrypted privacy data (EPD).

Finally, such encoded and masked video data (EMID) and encrypted privacy data (EPD) are stored in a memory card 15 (e.g., an SD card) embedded in the video spectator 1, or a network such as the Internet. The data is transmitted to the external playback device 2 (e.g. PC) by 16 and stored when necessary.

Now, the configuration of the video player 2 shown in FIG. 1 will be described in detail with reference to FIG.

The playback device 2 (e.g. PC) is encoded and masked by the user detaching the memory card 15 in the video recording device 1 and mounting it on the playback device 2 or receiving it via the network 16. Image data EMID and encrypted privacy data EPD are obtained.

Thus, the decoder 21 decodes the encoded image data EMID in the same manner as H.264 to form the masked image data MID, and the decoder 22 encrypts the image photographing apparatus 1. The private data PD is decrypted using the key used for encryption in the unit 14 to form the privacy data PD.

In addition, the masking release / maintenance processing unit 23 processes to maintain masking when the user is a general user who does not have a key, and outputs masked image data (MID). Or a security officer), the original image data is output by synthesizing the privacy data PD and the masked image data MID.

The image output unit 24 outputs the related image on the screen through the masked image data MID or the original image data output from the masking release / maintenance processor 23.

Next, although known to those skilled in the art, the privacy region extraction method used in the present invention will be briefly described with reference to FIG.

As shown in the upper part of FIG. 4, for face detection, a driving environment element may be removed from a given image, and pattern matching may be performed based on facial image features.

Then, as shown in the lower part of FIG. 4, for detecting the license plate, the moving object and the background are classified, and the license plate is detected by detecting features of the license plate image.

In this case, in order to implement such detection in real time, it is desirable to reduce the calculation amount by applying a method of making a high resolution image with a low resolution that can distinguish the face and the license plate.

Now, a method of masking in the image photographing apparatus 1 of the present invention will be described in detail with reference to the flowchart of FIG.

1. Obtain an image from the camera (S11).

2. From the obtained image, a face or a vehicle license plate area (privacy area) is extracted using the extraction method as shown in FIG. 4 (S12).

3. The extracted privacy region (ROI: Region Of Interest) includes coordinate information (x, y) of the reference point of the privacy region, the size of the privacy region (size), and time information (t) corresponding to the frame number. The privacy zone information is stored in a file File1 of the SD card 15 (S13). In this way, it is possible to know where the privacy areas exist in each frame and how large each privacy area is.

4. If the pixel value of each pixel of the privacy area is filled with a saturation value (eg, 255), the privacy area is masked with bright white (S14). Of course, if you want to be masked with a different color (for example, light blue in Figure 1), you can store the desired color-related RGB values in the file (File1), and fill the pixel values of the privacy area with the RGB values of the desired color.

5. In this way, the masked video is compressed and encoded in the same manner as in H.264 to form encoded and masked video data EMID and stored in the SD card 15 as a file File2 (S15).

6. Subtract the pixel values of each pixel from the saturation value or the desired color value with respect to the privacy area (S16). That is, 255- (RGB value of ROI (x, y)) or (RGB value of masking color)-(RGB value of ROI (x, y)).

7. By encrypting the deducted values using a key, an encrypted privacy data (EPD) is formed and stored as a file (File3) in the SD card 15 (S17).

Of course, in the above, the case where the files (File1, File2, File3) are stored in the SD card 15 has been described. However, it is also possible to transmit the data to an external device (eg, a PC) via a network.

Next, a method of playing back in the video reproducing apparatus 2 of the present invention will be described in detail with reference to the flowchart of FIG.

1. The user removes the SD card 15 from the video recording device 1 and mounts it on the video playback device 2 or receives data received from the video recording device 1 through the network 16 (the file (File1, File2 and File3), the encoded and masked video data EMID (stored in the file File2) is decoded by H.264 to form masked video data MID (S21).

2. The encrypted privacy data (EPD) (File3) for the privacy area is decrypted using the encryption key to form the privacy data PD (S22).

3. For the privacy zone, restore the original image data of the privacy zone by subtracting the privacy data PD from the saturation value 255 (if the masking color is bright white) or from the RGB value of the other masking color desired. (S23). 255- (255-ROI (x, y)) = ROI (x, y) or (RGB value of masking color)-((RGB value of masking color)-(RGB value of ROI (x, y)) ) = ROI (x, y).

4. Deriving the privacy area information including the coordinate information (x, y) of the privacy area, the size of the masking area and the time information (t) corresponding to the frame number from the file File1 of the SD card 15 Based on these, the masking portion of the masked image data MID is replaced with the original image data of the privacy area to obtain the overall original image data (S24).

5. In the case of a user who has a key, the user can see the original image without the masked part by outputting it to the screen using the entire original image data, and in the case of a user who does not have a key, the masked image data (MID) Output to the screen to see only the masked image (S25).

Although the preferred embodiment of the present invention has been described above, those skilled in the art should be noted that various modifications and changes can be made without departing from the spirit of the present invention.

For example, in the above, the use of an external PC as the playback device 2 has been described, but a playback module built in the video image pickup device 1 may be used.

Further, in the above description, the pixel values of each pixel are subtracted from the saturation value or the desired color value with respect to the privacy area, and the encrypted values are encrypted using a key to form the encrypted privacy data EPD, and the privacy data PD ) To restore the original image data of the masking area by subtracting the saturation value or the RGB value of the masking color, but encrypts the privacy area directly to form the encrypted privacy data (EPD), and the encrypted privacy data ( It is also possible to restore the original image data of the privacy area by decoding the EPD).

Claims (5)

In the privacy masking and restoration method of the captured image,
Extracting a privacy region from the captured original image and generating information on the extracted privacy region;
Filling the extracted privacy region with a predetermined masking RGB value and performing compression encoding on the filled image to form encoded and masked image data (EMID);
Encrypting the extracted privacy area with a key to form encrypted privacy data (EPD);
Storing information about the privacy area, the encoded and masked image data (EMID), and the encrypted privacy data (EPD) in an internal storage device or transmitting to an external device through a network;
Decoding the encoded and masked image data (EMID) to form masked image data (MID);
Decrypting the encrypted privacy data (EPD) using the key to form privacy data (PD);
If the user does not have the key, the masked image is displayed on the screen using the masked image data (MID), and if the user has the key, information on the privacy area and the masked image are displayed. And displaying the original image on which the privacy area is restored using the image data (MID) and the privacy data (PD) on the screen. The method of claim 1, wherein the forming of the encrypted privacy data (EPD) is achieved by encrypting the masking RGB value by subtracting the privacy data (PD) from the masking RGB value. . The method of claim 1, wherein the encryption and decryption is H.264. The method of claim 1, wherein the external device is a personal computer. The method of claim 1, wherein the built-in storage device is an SD card.

Images