CN116208833A - Privacy protection method and device for camera and storage medium - Google Patents

Abstract

The invention provides a privacy protection method, a privacy protection device and a privacy protection storage medium for a camera, wherein a mask body is arranged at the front end of the camera, so that the covered camera can only capture the internal picture of the mask body and cannot directly capture the surrounding environment, such as a living room or a bedroom, and the like, thereby preventing the camera from capturing privacy information from a physical aspect; meanwhile, a visual sensor is arranged outside the shade body and used for shooting pictures on the front surface of the covered camera, so that the normal shooting function is ensured; and the video recorded by the visual sensor is transmitted to the monitoring terminal through the network connection of the camera based on the visible light communication technology after being partially or completely encrypted, so that the video is isolated from the camera and the network thereof in logic and physics, the privacy information of a user is effectively protected, and meanwhile, the original function of the camera is reserved, so that consistent user experience is provided.

Description

Privacy protection method and device for camera and storage medium

Technical Field

The invention relates to the field of intelligent security, in particular to a privacy protection method and device for a camera and a storage medium.

Background

Cameras are now common in both public and private locations. People tend to be concerned about whether their sensitive information, such as face, behaviour, etc., will be captured and misused by the owner or operator of the camera. The rich information in videos or images is a necessary condition to make them "more intelligent" on the one hand, but also implies a higher risk of privacy disclosure on the other hand. Especially smart home devices, such as home security cameras, smart televisions, etc., are equipped with visual sensors for monitoring or improving user experience. Their visual perceptibility is more likely to cause privacy and security problems due to the sensitivity of the home environment.

Disclosure of Invention

In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a method, an apparatus and a storage medium for protecting privacy of a camera, so as to reduce risk of privacy disclosure of the camera.

To achieve the above and other related objects, a first aspect of the present invention provides a privacy protection method for a camera, including: installing a shade body at the front end of the camera so that the camera only shoots images in the shade body, wherein a visual sensor is arranged on the shade body to acquire original monitoring data of a monitoring area; encrypting the original monitoring data to obtain encrypted monitoring data; transmitting the encrypted monitoring data to a monitoring terminal through the camera based on a visible light communication technology; and the monitoring terminal decrypts the encrypted monitoring data to obtain the original monitoring data.

In some embodiments of the first aspect of the present invention, the transmission manner of the encrypted monitoring data includes: encoding the encrypted monitoring data to generate a sequence of visible light communication frames; sequentially displaying the visible light communication frame sequence on a display screen in the mask body; the camera focuses on the display screen through a lens to capture a visible light communication frame sequence on the display screen and transmit the visible light communication frame sequence to the monitoring terminal; and the monitoring terminal decodes the received visible light communication frame sequence to acquire the encrypted monitoring data.

In some embodiments of the first aspect of the present invention, the encryption method of the original monitoring data includes: extracting a region to be encrypted of a video frame based on the configured security policy and the original monitoring data; and encrypting the area to be encrypted based on a symmetric key shared in advance by the masking equipment and the monitoring terminal to acquire the encrypted monitoring data.

In some embodiments of the first aspect of the present invention, the configuration items of the security policy include: index item, area item to be encrypted, algorithm item, inversion item, time item and/or effective item; wherein the index item and the region item to be encrypted uniquely identify the security policy; the algorithm item configures an extraction algorithm of the region of interest; the inversion item configures the region to be encrypted to include the extracted region of interest or a complementary region opposite thereto; the valid item configures whether the security policy is enabled; the time item configures an enablement period for the security policy.

In some embodiments of the first aspect of the present invention, the privacy protection method of the camera includes: under the condition that the mask equipment detects a motion event, the warning LED lamp or the display screen in the mask body simulates a virtual motion event so as to trigger the motion detection function of the camera.

In some embodiments of the first aspect of the present invention, the privacy preserving method includes: a misalignment calibration preamble is inserted before the visible light communication frame to perform color misalignment and/or spatial misalignment calibration of a sequence of visible light communication frames displayed on the display screen.

In some embodiments of the first aspect of the present invention, the privacy preserving method includes: a distortion calibration preamble frame is inserted before the visible light communication frame to perform image distortion calibration on a sequence of visible light communication frames displayed on the display screen.

To achieve the above and other related objects, a second aspect of the present invention provides a privacy protecting apparatus for a camera, comprising: the shade body is arranged at the front end of the camera; the visual sensor is arranged on the shade body and used for acquiring original monitoring data of the monitoring area; the processing unit is connected with the visual sensor and used for receiving the original monitoring data and encrypting the original monitoring data to obtain encrypted monitoring data; encoding the encrypted monitoring data based on a visible light communication technology to obtain a visible light communication frame sequence; the display screen is arranged in the shade body and connected with the processing unit, and is used for displaying the visible light communication frame sequence on the display screen in sequence by the processing unit; and the lens is arranged between the display screen and the camera, and focuses the camera on the display screen to capture the visible light communication frame sequence on the display screen and transmit the visible light communication frame sequence to the monitoring terminal.

In some embodiments of the second aspect of the present invention, the privacy preserving apparatus includes: the warning LED lamp is connected with the processing unit and used for triggering the movement detection function of the camera; and the message LED lamp is connected with the processing unit and used for displaying the working state of the privacy protection device.

To achieve the above and other related objects, a third aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the privacy preserving method of a camera.

As described above, the privacy protection method, device and storage medium for the camera provided by the invention have the following beneficial effects: by installing the shade body at the front end of the camera, the covered camera can only capture the internal picture of the shade body and cannot directly shoot the surrounding environment, such as a living room or a bedroom, so that the camera is prevented from capturing privacy information from the physical aspect; meanwhile, a visual sensor is arranged outside the shade body and used for shooting pictures on the front surface of the covered camera, so that the normal shooting function is ensured; and the video recorded by the visual sensor is transmitted to the monitoring terminal through the network connection of the camera based on the visible light communication technology after being partially or completely encrypted, so that the video is isolated from the camera and the network thereof in logic and physics, the privacy information of a user is effectively protected, and meanwhile, the original function of the camera is reserved, so that consistent user experience is provided.

Drawings

Fig. 1 is a flowchart illustrating a privacy protection method of a camera according to an embodiment of the invention.

Fig. 2 is a schematic structural diagram of a privacy protection apparatus of a camera according to an embodiment of the invention.

Fig. 3 is a schematic diagram of a software architecture of a privacy preserving apparatus according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a visible light communication frame according to an embodiment of the invention.

Fig. 5 is a schematic diagram of an ROI setting protocol according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of an operation scenario of a privacy preserving apparatus according to an embodiment of the present invention.

Fig. 7 is a top view showing a hardware structure of a privacy preserving apparatus according to an embodiment of the present invention.

Fig. 8 is a left side view showing a hardware structure of a privacy preserving apparatus in one embodiment of the present invention.

Fig. 9 is a bottom view showing a hardware structure of a privacy preserving apparatus according to an embodiment of the present invention.

Fig. 10 is a schematic diagram of ROI-based video encryption according to an embodiment of the present invention.

Detailed Description

Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

In the following description, reference is made to the accompanying drawings, which illustrate several embodiments of the invention. It is to be understood that other embodiments may be utilized and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description is not to be taken in a limiting sense, and the scope of embodiments of the present invention is defined only by the claims of the issued patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Spatially relative terms, such as "upper," "lower," "left," "right," "lower," "upper," and the like, may be used herein to facilitate a description of one element or feature as illustrated in the figures as being related to another element or feature.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," "held," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.

Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including" specify the presence of stated features, operations, elements, components, items, categories, and/or groups, but do not preclude the presence, presence or addition of one or more other features, operations, elements, components, items, categories, and/or groups. The terms "or" and/or "as used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a, A is as follows; b, a step of preparing a composite material; c, performing operation; a and B; a and C; b and C; A. b and C). An exception to this definition will occur only when a combination of elements, functions or operations are in some way inherently mutually exclusive.

The invention provides a privacy protection method, a device and a storage medium for a camera, which are used for solving the problem that the existing camera is easy to cause privacy disclosure, and are particularly suitable for home environments with high requirements on privacy security.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention are further described in detail by the following embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Example 1

As shown in fig. 1, an embodiment of the present invention provides a flowchart of a privacy protection method for a camera, where the privacy protection method includes the following steps:

s11, installing a shade body at the front end of the camera so that the camera only shoots images in the shade body, wherein a visual sensor is arranged on the shade body to acquire original monitoring data of a monitoring area. By installing the shade body at the front end of the camera, the covered camera can only capture the internal picture of the shade body and cannot directly shoot the surrounding environment, such as a living room or a bedroom, so that the camera is prevented from capturing privacy information from the physical aspect; meanwhile, a visual sensor is arranged outside the shade body and used for shooting the picture on the front face of the covered camera, namely the monitoring data of the camera when the shade body is not installed, namely the original monitoring data of the monitoring area, so that the normal shooting function is ensured.

Further, under the condition that the mask equipment detects a motion event, the warning LED lamp or the display screen in the mask body simulates a virtual motion event so as to trigger the motion detection function of the camera. The warning LED lamp in the shade body simulates a virtual movement event through flashing. The design of the original monitoring function of the camera is guaranteed, if shot monitoring data are monitoring data of the camera when the mask body is not installed, and the motion detection function of the camera and the like are reserved.

And S12, encrypting the original monitoring data to obtain encrypted monitoring data. One straightforward way to protect visual privacy is to blur the entire video stream, but this can lead to information loss and reduce the practical value of the video. This embodiment is preferably selectively protected, i.e. only the sensitive area is protected and the remaining areas are not affected. The first step in the selective protection is to identify a region of interest (Region of Interest, ROI), which represents a sensitive region of the video/image, typically possibly a facial region. ROI detection relies on Computer Vision (CV) techniques, such as filter-based detection techniques, deep neural network detection-based detection techniques.

In some examples, ROI detection uses an on-demand processing scheme to reduce detection overhead. In other examples, ROI detection utilizes additional sensors to assist the ROI detection algorithm, e.g., a thermal image sensor and a bio-signal sensor to distinguish facial regions from other regions.

After the ROI area is acquired through ROI detection, the ROI area is processed to protect privacy. Alternative processing methods are blurring processing, pixelation processing, replacement object processing, and the like. In some examples, the processing of the ROI area may be done prior to image capture, e.g., by processing the ROI area through an optical device. In some examples, the ROI area may be protected directly by encryption, which retains the original information and allows full access by the authorized entity.

And S13, transmitting the encrypted monitoring data to a monitoring terminal through the camera based on a visible light communication (Visible Light Communication, VLC) technology.

In a preferred implementation manner of this embodiment, the transmission manner of the encrypted monitoring data includes: encoding the encrypted monitoring data to generate a sequence of visible light communication frames; sequentially displaying the visible light communication frame sequence on a display screen in the mask body; the camera focuses on the display screen through a lens to capture a visible light communication frame sequence on the display screen and transmit the visible light communication frame sequence to the monitoring terminal; and the monitoring terminal decodes the received visible light communication frame sequence to acquire the encrypted monitoring data.

In this embodiment, the display screen and the camera (i.e. screen-to-camera) communicate with each other through visible light to transmit data, so as to construct a unidirectional data transmission link. Preferably, based on a dynamic two-dimensional code technology, optical content in a two-dimensional code form is displayed on the display screen, and the camera captures a data stream corresponding to the optical content. In this embodiment, the unique use scene of two-dimensional code makes it have fixed shooting view angle, so more conventional two-dimensional code design is simpler.

Further, a deviation calibration preamble frame is inserted before the visible light communication frame to perform color deviation and/or spatial deviation calibration on a sequence of visible light communication frames displayed on the display screen. A distortion calibration preamble frame is inserted before the visible light communication frame to perform image distortion calibration on a sequence of visible light communication frames displayed on the display screen.

And S14, the monitoring terminal decrypts the encrypted monitoring data to obtain original monitoring data. The monitoring terminal can be a user mobile terminal, such as a mobile phone, ipad, smart watch, notebook computer and the like. And, the monitoring terminal shares a key in advance with an encryption end (masking device). For example, the encryption key is set in the device firmware and printed on a disposable sticker attached to the new device, and the device owner enters the key into the application program of the monitoring terminal to authorize it to access the encrypted information.

In a preferred implementation manner of this embodiment, the encryption manner of the original monitoring data includes: extracting a region to be encrypted (i.e., a sensitive region) of a video frame based on the configured security policy and the original monitoring data; and encrypting the area to be encrypted based on a symmetric key shared in advance by the masking equipment and the monitoring terminal to acquire the encrypted monitoring data. Optionally, the symmetric key includes DES (Data Encryption Standard), 3DES (Triple Data Encryption Standard), IDEA, RC5, RC6, etc.

Specifically, the configuration items of the security policy include: index item, area item to be encrypted, algorithm item, inversion item, time item and/or effective item; wherein the index item and the region item to be encrypted uniquely identify the security policy; the algorithm item configures an extraction algorithm of the region of interest; the inversion item configures the region to be encrypted to include the extracted region of interest or a complementary region opposite thereto; the valid item configures whether the security policy is enabled; the time item configures an enablement period for the security policy.

Further, the security policy supports enabling multiple ROI policies simultaneously, and the finally obtained region to be encrypted is the sum of all the ROI regions extracted by the enabled ROI policies.

In a preferred implementation manner of this embodiment, the method for obtaining the encrypted monitoring data includes: the visible light communication frame is cut into a plurality of tiles, and tiles to be encrypted, which are overlapped with the area to be encrypted, are obtained; the tile to be encrypted is encrypted by a key and a randomly generated key stream, and a random number used for encryption is inserted between a frame header and a data portion of a network abstraction layer unit, and data of the entire frame and a time stamp thereof are hashed and appended to encrypted content to acquire the encrypted monitoring data.

In some embodiments, the method may be applied to a controller, such as a ARM (Advanced RISC Machines) controller, FPGA (Field Programmable Gate Array) controller, soC (System on Chip) controller, DSP (Digital Signal Processing) controller, or MCU (Microcontroller Unit) controller, among others. In some embodiments, the method may also be applied to a computer including components such as memory, a memory controller, one or more processing units (CPUs), peripheral interfaces, RF circuitry, audio circuitry, speakers, microphones, input/output (I/O) subsystems, display screens, other output or control devices, and external ports; the computer includes, but is not limited to, a personal computer such as a desktop computer, a notebook computer, a tablet computer, a smart phone, a smart television, a personal digital assistant (Personal Digital Assistant, PDA for short), and the like. In other embodiments, the method may also be applied to servers, which may be disposed on one or more physical servers according to various factors such as functions, loads, etc., or may be composed of a distributed or centralized server cluster.

Example two

As shown in fig. 2, an embodiment of the present invention proposes a schematic structural diagram of a privacy protecting apparatus (mask device) of a camera, including: a mask body 21, a vision sensor 22, a processing unit 23, a display screen 24 and a lens 25.

The mask body 21 is provided at the front end of the camera (receiving camera) 20. In some examples, the mask body 21 is connected with the camera 20 through an adjustable clamping sleeve 26, and has the characteristics of convenient connection, attractive appearance, adjustability and flexible use. The vision sensor 22 is disposed on the mask body 21, and is used for acquiring original monitoring data of the monitoring area. The processing unit 23 is connected with the vision sensor 22 and is used for receiving the original monitoring data and encrypting the original monitoring data to obtain encrypted monitoring data; and encoding the encrypted monitoring data based on a visible light communication technology to obtain a visible light communication frame sequence. The display screen 24 is disposed in the mask body 21 and connected to the processing unit 23, so that the processing unit 23 can display the visible light communication frame sequence thereon in sequence. A lens 25 is provided between the display screen 24 and the camera 20 to focus the camera 20 on the display screen 24 to capture and transmit the sequence of visible light communication frames on the display screen 24 to a monitoring terminal.

In some examples, vision sensor 22 captures video and stores it in the memory of processing unit 23. The processing unit 23 runs an embedded operating system to manage the vision sensor 22 and the display screen 24. Masking software in the operating system is responsible for executing image/video processing algorithms, encryption and communication protocols. The display screen 24 serves as an output interface for transmitting the encrypted video stream from the masking device to the camera head 20. A suitable lens 25 is provided in front of the camera 20 to adjust the focal length of the camera 20 to accommodate the display screen 24.

The invention retains the function of the covered camera through the function compatibility design, and mainly comprises video monitoring preview and motion detection. The invention is realized by sharing the video preview clip from the camera matched application program to the mask application program (the program used for decryption and decoding by the monitoring terminal), and the monitoring preview picture is obtained after the mask application program decodes the video clip. For the motion detection function, firstly, a masking device (privacy preserving means) detects motion by analyzing the captured video stream, and secondly, triggers the camera using simulated motion events, thereby generating motion detection alerts to be sent to the camera companion application.

In the preferred implementation of the present embodiment, since the VLC stream displayed on the screen (display screen 24) may also be regarded as motion and cause false alarm, the screen displaying the VLC stream is excluded from the motion detection area by the detection area setting function of the camera.

In a preferred implementation of this embodiment, the privacy protection apparatus further includes: and the warning LED lamp 27 is connected with the processing unit 23 and is used for triggering the movement detection function of the camera. The alarm LED lamp 27 is provided in the mask body 21 to simulate a virtual movement event by flashing, thereby triggering the movement detection function of the camera 20.

In a preferred implementation of this embodiment, the privacy protection apparatus further includes: and a message LED lamp 28 connected with the processing unit 23 for displaying the working state of the privacy protection device. A message LED lamp 28 is provided outside the mask body 21 for informing the owner about the operating state of the mask device, for example, whether the face ROI is enabled. This mechanism is used to avoid protection vulnerabilities that may result from unknowing the owner, e.g., the owner forgets to enable the ROI.

The processing unit 23 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), and the like; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), field-programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.

As shown in fig. 3, an embodiment of the present invention provides a software architecture diagram of a mask device (privacy preserving apparatus), which can be expressed as follows: first, ROI-based privacy protection is implemented on a captured video stream: detecting the ROI area of the original video by adopting a standard computer vision module; the encryption module encrypts video content belonging to the ROI area while compressing the original video. The encrypted and compressed video stream is then encoded into VLC frames (visible light communication frames) that are filled into a frame buffer and displayed on a display screen inside the mask body. VLC frames are captured by the camera (i.e., the receiving camera) and transmitted over the network into the masking application. Finally, the owner uses the keyed mask application to retrieve the decoded and decrypted captured original video stream, resulting in a surveillance preview video.

The embodiment of the invention constructs a unidirectional data transmission path (unidirectional VLC data path) from the mask equipment to the camera by utilizing the visual sensing capability of the camera, and relays the video stream to the Internet by utilizing the camera so as to transmit the video stream to a user, thereby realizing the isolation of the mask equipment and the network and guaranteeing the privacy security of the data transmission process.

As shown in fig. 4, an embodiment of the present invention proposes a schematic diagram of a visible light communication frame, and the VLC frame looks like a color two-dimensional code. The masking device encodes the photographed video stream in a sequence of VLC frames, which are sequentially displayed on an in-mask display screen and captured by a camera, and the monitoring terminal extracts the video photographed by the masking device by decoding the captured VLC frames.

To describe VLC designs, embodiments of the present invention define VLC frames for transmission as a matrix of pixels txFR, each matrix element representing a color of a pixel. While rxFR is taken to represent the VLC frame captured by the camera. Each VLC frame represents a matrix element using a 10x10 pixel square. The pixels in the same block have the same color. In a preferred implementation of this embodiment, the VLC frame sequence is acquired using eight-color modulation, whose mapping is shown in table 1, i.e., 3-bits are modulated per matrix element, to thereby increase the throughput of the data path.

TABLE 1 mapping relationship of eight color modulation

The design of the lens brings serious lens distortion, so that the picture captured by the camera presents the distortion phenomenon of barrel-shaped expansion. Another problem with lenses is chromatic distortion, different colors have different degrees of distortion, because lenses have different refractive ratios for different colors of light, and the farther from the center, the larger the offset between the colors that can be observed, increasing the difficulty in locating the exact position of the block. Because the different color component positions of the data block are different when different colors are displayed.

In a preferred implementation of the present embodiment, in order to solve the effect of lens distortion, the embodiment of the present invention corrects the distortion by inserting a distortion correction preamble frame before the VLC data frame, as shown in fig. 4. The distortion calibration preamble frame includes a black and white checkerboard pattern having a grid size equal to the data block size. Each black and white block in rxFR was accurately located using computer vision methods and their positions recorded. To handle chromatic distortion, the above process is performed independently in the red, green and blue channels. The acquired position information will then be used to index the blocks in the VLC data frame to extract the color, thereby decoding the data bits. It should be noted that, since the mask device is fixed with the camera, the position of each data block is stable, so that the distortion calibration preamble frame only needs to be used in the initialization process after the device is started, and the data transmission is not repeated.

In addition, the receiving camera employs a variety of non-configurable image processing schemes to automatically adjust the contrast, brightness, saturation, etc. of the captured video. These bring about color deviations to rxFR. Meanwhile, part of intelligent security cameras have vignetting effect, so that the central area of rxFR is higher in brightness than the peripheral area, and the color of the data block is inconsistent in space.

In a preferred implementation of the present embodiment, the color deviation due to the auto-adjustment and/or vignetting effect is calibrated by introducing a deviation calibration preamble frame, as shown in fig. 4. The offset calibration leading frame includes a palette (palette), and is composed of the same palette for displaying colors used by txFR. These palettes are used as references for decoding colors. I.e. the closest color in the palette is selected as the color of the block in rxFR when decoding. Since palettes are piled up over the whole VLC frame, spatial deviations caused by vignetting effects can also be compensated for.

Embodiments of the present invention use digital encryption techniques to protect information between a covering device and an authorized party (e.g., a covering application) in a ROI-based selective encryption manner, rather than encrypting the entire frame, which allows owners flexibility in specifying the areas they want to protect. Further, by executing different security policies, the privacy security is ensured while the functionality of the camera is ensured.

In the preferred implementation of the embodiment of the present invention, the user (device owner) can flexibly configure the security policy of the device and specify the encryption working mode based on the ROI. The security policy entry in the mask device is shown for example in table 2 (security policy configuration table, ROI setting table entry) (similar to access control list rules for firewalls).

TABLE 2 Security policy configuration Table

1 haar cascades

2 histograms of oriented gradients

3 scene text detection

Each row of table 2 defines the protection behavior of the mask device for the relevant ROI. The "index" and "ROI" items uniquely identify the strategy of the ROI (where index 1 represents the haar cascades classifier, index 2 represents the histograms of oriented gradients gradient direction histogram, and index 3 represents the scene text detection Scene text detection). ROI detection may be performed by a variety of methods, which are specified by the "algorithm" term. The "inversion" item designates the protection region as the detected ROI or a corresponding supplementary region, e.g. the body or a background outside the body. The "time" and "valid" items determine when and if this policy entry should be enabled. The policy support enables multiple ROI policies simultaneously, and the final obtained region to be encrypted is the sum of all ROI regions extracted by the enabled ROI policies.

In a preferred implementation of the embodiment of the invention, the invention retains a unique input interface for retrieving messages from its owner in order to allow the mask device owner to configure the ROI policy. The present invention sets the ROI policy by scanning a two-dimensional code containing configuration information with a visual sensor of a mask apparatus and sharing between the mask application and the mask apparatus. The owner uses the mask application to generate a two-dimensional code containing the configuration message and displays it to the mask device. The configuration message example of the ROI policy "time" item is as follows:

further, since the configuration interface involves sensitive processing, the present invention uses the ROI setup protocol shown in fig. 5 to protect it from spoofing and replay attacks. The mask device uses a typical challenge and response protocol to verify the application displaying the two-dimensional code. When the configuration is started, the mask application program firstly displays a two-dimensional code containing HELLO information. After the masking device receives the HELLO, it sends a challenge message REQ, which contains a random number nonce, to the paired application via the VLC data path. Once the application receives the challenge, it replies to the masking device with another two-dimensional code named RES, containing a random number and a hash value of the shared key. The masking device can verify the legitimacy of the application by verifying the hash, since the key is shared only with authorized masking applications. Once authentication is successfully completed, the masking device will accept the encrypted configuration message, i.e. the configuration information ConfigMsg appended in RES.

Example III

As shown in fig. 6, an embodiment of the present invention proposes a schematic view of a working scenario of a privacy protection apparatus. The privacy protection device (camera mask, shown in dotted line) masks the camera with its mask body and acquires the real picture with its own vision sensor to ensure the monitoring capability. The video is captured through the encryption channel and is transmitted to the monitoring terminal through the camera, the Internet and the cloud server, and the monitoring terminal decrypts the video data through the loaded camera mask application program to realize remote monitoring preview. The camera masking device is physically and logically isolated from the network and the infected camera, so that the trustworthiness of the camera masking device can be guaranteed as long as the factory firmware of the camera masking device is trusted, and the masking system can protect the privacy and safety of video content even if the camera is completely controlled by malicious programs.

As shown in fig. 7 to 9, the embodiment of the invention provides a hardware structure schematic of a privacy protection device, wherein fig. 7 is a top view of the device, fig. 8 is a left side view of the device, and fig. 9 is a bottom view of the device. The embodiment uses commercial products and 3D printing shells to realize hardware of the shade equipment, and specifically: nvidia Jetson Nano development boards are selected for the processing unit 71; the visual sensor 72 is Raspberry Camera V2.1.1, 60-fps@720 p, and the visual sensor 72 is directly connected to a CMOS serial interface of the Nano development board 71; the display screen 73 is shirp LS029B3SX02 1440x1440@60fps, and is driven by the HDMI driving board 74; a 60-mm focal length lens 75 is used to focus a receiving camera 76 onto the display screen 73. Two LED lights (message LED light 77 and alert LED light 78) are connected to GPIO pins of Nano development board 71. And attaches the above components to a 3D printed housing 79. Further, the display screen is placed in a removable slot so that it can be adjusted to fit the focal plane of the smart camera.

In some examples, the mask apparatus leverages the characteristics of modern video codecs to enable video ROI encryption and transmission. The characteristics are as follows: modern video codecs (e.g. HEVC, or h.265) can divide video frames into uniform rectangular areas, tiles, and pack tiles into NAL units (network abstraction layer units, network Abstraction Layer Unit) for general network transmission.

Based on this characteristic, the masking device packages a plurality of video frames after HEVC compression into one VLC frame, each video frame containing a plurality of encrypted and unencrypted tiles; encrypting any tile overlapped with the ROI through a key and a randomly generated key stream, wherein a random number used for encryption is inserted between the NALU header and NALU data; the encrypted tile region can still be rendered for a normal HEVC player that does not know the key, but the picture information will be lost (as shown in fig. 10 as an example of ROI-based video encryption); for the unencrypted tile, the player can decode normally, so that the visibility of the non-privacy area is ensured; the data of the whole frame and the time stamp thereof are hashed and added to the encrypted content, ensuring the integrity and authenticity of the data frame.

Example IV

The embodiment of the invention provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, realizes the privacy protection method of a camera.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by computer program related hardware. The aforementioned computer program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

In summary, according to the privacy protection method, device and storage medium for the camera, the mask body is installed at the front end of the camera, so that the covered camera can only capture the internal picture of the mask body, but cannot directly capture the surrounding environment, such as a living room or bedroom, and the like, thereby preventing the camera from capturing privacy information from physical aspects; meanwhile, a visual sensor is arranged outside the shade body and used for shooting pictures on the front surface of the covered camera, so that the normal shooting function is ensured; and the video recorded by the visual sensor is transmitted to the monitoring terminal through the network connection of the camera based on the visible light communication technology after being partially or completely encrypted, so that the separation of the video from the camera and the network thereof in logic and physical aspects is realized, the privacy information of a user is effectively protected, and the privacy protection method of the camera using function copying and physical separation is realized. The privacy protection device (camera shade system) of the camera is designed to protect the privacy safety of the camera, can be directly applied to commercial household intelligent security cameras, and keeps key intelligent functions of the intelligent security cameras, such as monitoring picture preview and motion detection, so that consistent user experience is provided for users. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims (10)

1. A privacy protection method for a camera, comprising:

installing a shade body at the front end of the camera so that the camera only shoots images in the shade body, wherein a visual sensor is arranged on the shade body to acquire original monitoring data of a monitoring area;

encrypting the original monitoring data to obtain encrypted monitoring data;

transmitting the encrypted monitoring data to a monitoring terminal through the camera based on a visible light communication technology;

and the monitoring terminal decrypts the encrypted monitoring data to obtain the original monitoring data.

2. The method for protecting privacy of a camera according to claim 1, wherein the transmission mode of the encrypted monitoring data comprises:

encoding the encrypted monitoring data to generate a sequence of visible light communication frames;

sequentially displaying the visible light communication frame sequence on a display screen in the mask body;

the camera focuses on the display screen through a lens to capture a visible light communication frame sequence on the display screen and transmit the visible light communication frame sequence to the monitoring terminal;

and the monitoring terminal decodes the received visible light communication frame sequence to acquire the encrypted monitoring data.

3. The method for protecting privacy of a camera according to claim 1, wherein the encryption method of the original monitoring data comprises:

extracting a region to be encrypted of a video frame based on the configured security policy and the original monitoring data;

and encrypting the area to be encrypted based on a symmetric key shared in advance by the masking equipment and the monitoring terminal to acquire the encrypted monitoring data.

4. A method of protecting privacy of a camera according to claim 3, wherein the configuration items of the security policy include: index item, area item to be encrypted, algorithm item, inversion item, time item and/or effective item;

wherein the index item and the region item to be encrypted uniquely identify the security policy; the algorithm item configures an extraction algorithm of the region of interest; the inversion item configures the region to be encrypted to include the extracted region of interest or a complementary region opposite thereto; the valid item configures whether the security policy is enabled; the time item configures an enablement period for the security policy.

5. The privacy protection method of a camera according to claim 1, comprising:

under the condition that the mask equipment detects a motion event, the warning LED lamp or the display screen in the mask body simulates a virtual motion event so as to trigger the motion detection function of the camera.

6. The privacy protection method of a camera according to claim 2, comprising:

a misalignment calibration preamble is inserted before the visible light communication frame to perform color misalignment and/or spatial misalignment calibration of a sequence of visible light communication frames displayed on the display screen.

7. The privacy protection method of a camera according to claim 2, comprising:

a distortion calibration preamble frame is inserted before the visible light communication frame to perform image distortion calibration on a sequence of visible light communication frames displayed on the display screen.

8. A privacy protection apparatus for a camera, comprising:

the shade body is arranged at the front end of the camera;

the visual sensor is arranged on the shade body and used for acquiring original monitoring data of the monitoring area;

the processing unit is connected with the visual sensor and used for receiving the original monitoring data and encrypting the original monitoring data to obtain encrypted monitoring data; encoding the encrypted monitoring data based on a visible light communication technology to obtain a visible light communication frame sequence;

the display screen is arranged in the shade body and connected with the processing unit, and is used for displaying the visible light communication frame sequence on the display screen in sequence by the processing unit;

and the lens is arranged between the display screen and the camera, and focuses the camera on the display screen to capture the visible light communication frame sequence on the display screen and transmit the visible light communication frame sequence to the monitoring terminal.

9. The privacy securing apparatus of claim 8, comprising:

the warning LED lamp is connected with the processing unit and used for triggering the movement detection function of the camera;

and the message LED lamp is connected with the processing unit and used for displaying the working state of the privacy protection device.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the privacy protection method of a camera according to any one of claims 1 to 7.

Images