WO2013137534A1

WO2013137534A1 - System and method for processing image to protect privacy

Info

Publication number: WO2013137534A1
Application number: PCT/KR2012/009124
Authority: WO
Inventors: Joon Kyu Kim
Original assignee: Samsung Techwin Co.,Ltd.
Priority date: 2012-03-12
Filing date: 2012-11-01
Publication date: 2013-09-19

Abstract

Provided is a system for processing an image to protect privacy. The system includes: an image pickup unit generating first image data; an information setting unit setting privacy information of the first image data; and an image editing unit generating second image data by generating or removing a mask region which masks a subject by editing the first image data based on the privacy information, wherein when an event occurs, the image editing unit generates or removes the mask region for the subject.

Description

SYSTEM AND METHOD FOR PROCESSING IMAGE TO PROTECT PRIVACY

The present invention relates to a system and method for processing an image to protect privacy, and more particularly, to a system and method for processing an image to protect privacy, the system and method being employed to precisely set privacy masks in a protection area of a captured image.

A surveillance camera used for crime prevention and security purposes is attached to and installed in an area or place to be monitored. Images captured by the surveillance camera are output to a digital video recorder (DVR) and then displayed on the monitor screen or recorded on a recording medium such as a hard disk.

The surveillance camera may be rotated in a horizontal or vertical direction to monitor an area in a different direction, and an enlarged or reduced image of a subject may be captured by adjusting a zoom lens included in the surveillance camera.

The surveillance camera has a masking function. Thus, the surveillance camera generates a privacy mask for a subject whose privacy needs to be protected and displays the privacy mask on the subject to prevent the subject from being exposed in a captured image.

That is, a mask generation mode may be set in order to prevent a subject in a certain area of a camera image from being exposed without permission. In the mask generation mode, the surveillance camera generates a mask of a size corresponding to the start coordinates (Xs, Ys) and end coordinates (Xe, Ye) of a certain area.

In a conventional masking technique for protecting a subject’s privacy, a captured image is treated as a 2D plane. Thus, a mask is placed at the position of a subject to overlap the subject in order to prevent the exposure of the subject.

However, such a 2D masking technique does not take actual 3D spaces and distances in the captured image into consideration, making it difficult to smoothly mask dynamic subjects.

In addition, it may be desirable to divide an area being photographed by a surveillance camera into a plurality of sections and generate masks for subjects in some of the zones and not for subjects in the other zones. However, such active and real utilization of the 2D masking technique cannot be expected. Even if the area is divided two-dimensionally, when the surveillance camera changes its view angle, axes of boundaries that divide the area two-dimensionally also change. Thus, the axes have to be newly set every time the surveillance camera changes its view angle.

In particular, a subject (i.e., a prime target of surveillance) that once entered a surveillance area needs to be monitored continuously even after the subject leaves the surveillance area, and an image of the subject before the subject enters the surveillance area also needs to be monitored for more systematic surveillance.

Aspects of the present invention provide a system and method for processing an image to protect privacy, the system and method being employed to precisely set a privacy mask for a subject in image data captured by a surveillance camera.

Aspects of the present invention also provide a system and method for processing an image to protect privacy, the system and method being employed to adaptively obtain information about a surveillance area, which is an actual 3D space, by determining the positions of subjects using coordinates of reference axes changed by an angle at which image data was actually captured by a surveillance camera.

Aspects of the present invention also provide a system and method for processing an image to protect privacy, the system and method being employed to continuously monitor a subject (i.e., a prime target of surveillance such as a criminal) that once entered a surveillance area by removing a mask for the subject even after the subject leaves the surveillance area and to obtain a maskless image of the subject before the subject enters the surveillance area using a buffer unit which stores raw data without masks.

However, aspects of the present invention are not restricted to the one set forth herein. The above and other aspects of the present invention will become more apparent to one of ordinary skill in the art to which the present invention pertains by referencing the detailed description of the present invention given below.

According to an aspect of the present invention, there is provided a system for processing an image to protect privacy. The system includes: an image pickup unit generating first image data; an information setting unit setting privacy information of the first image data; and an image editing unit generating second image data by generating or removing a mask region which masks a subject by editing the first image data based on the privacy information, wherein when an event occurs, the image editing unit generates or removes the mask region for the subject.

According to another aspect of the present invention, there is provided a method of processing an image to protect privacy. The method includes: generating first image data; setting privacy information of the first image data; and generating second image data by generating or removing a mask region which masks a subject by editing the first image data based on the privacy information, wherein the generating of the second image data further includes generating or removing the mask region for the subject when an event occurs.

The above and other aspects and features of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings, in which:

FIG. 1 is a block diagram of a system for processing an image to protect privacy according to an embodiment of the present invention;

FIG. 2 shows an example of image data obtained by the system of FIG. 1;

FIG. 3 shows the image data of FIG. 2 to which coordinate data has been applied;

FIG. 4 shows privacy masks set for subjects existing within a predetermined range of coordinates based on the coordinate data of FIG. 3;

FIG. 5 shows the image data of FIG. 2 divided into a plurality of zones;

FIG. 6 shows picture-in-picture (PIP) data of a subject existing in a first zone of the image data of FIG. 5;

FIG. 7 shows a privacy mask set for a subject existing in a third zone of the image data of FIG. 5;

FIGS. 8A and 8B show a privacy mask set for a subject existing within a predetermined range in a second zone of the image data of FIG. 5;

FIG. 9 is a flowchart illustrating a method of processing an image to protect privacy according to an embodiment of the present invention; and

FIG. 10 is a detailed flowchart illustrating a privacy information setting process of FIG. 9.

Advantages and features of the present invention and methods of accomplishing the same may be understood more readily by reference to the following detailed description of exemplary embodiments and the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of the invention to those skilled in the art, and the present invention will only be defined by the appended claims. In the drawings, the sizes and relative sizes of layers and regions are exaggerated for clarity.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "made of," when used in this specification, specify the presence of stated components, steps, operations, and/or elements, but do not preclude the presence or addition of one or more other components, steps, operations, elements, and/or groups thereof.

Spatially relative terms, such as "below," “beneath,” “lower,” “above,” “upper,” and the like, may be used herein for ease of description to describe one device or element's relationship to another device(s) or element(s) as illustrated in the drawings. Like reference numerals refer to like elements throughout the specification.

A system for processing an image to protect privacy according to an embodiment of the present invention will now be described with reference to FIGS. 1 through 4. FIG. 1 is a block diagram of a system for processing an image to protect privacy according to an embodiment of the present invention. FIG. 2 shows an example of image data obtained by the system of FIG. 1. FIG. 3 shows the image data of FIG. 2 to which coordinate data has been applied. FIG. 4 shows privacy masks set for subjects existing within a predetermined range of coordinates based on the coordinate data of FIG. 3.

Referring to FIG. 1, the system for processing an image to protect privacy according to the current embodiment includes an image pickup unit 110 which generates image data, an information setting unit 120 which sets privacy information of the image data, and an image editing unit 130 which generates a mask region for masking a subject by editing the image data based on the privacy information.

In some other embodiments, the privacy information may include coordinate data set as a range in the image data, and the coordinate data may include 3D coordinate or distance information of subjects existing in the image data.

The image pickup unit 110 captures an image of the inside or outside of a building or a specific area and generates image data from the captured image. The image pickup unit 110 may include a camera unit which actually captures an image, an encoder unit which is connected to the camera unit and generates analog or digital image data, and a transceiver unit which transmits the image data to, e.g., a surveillance monitor.

The image pickup unit 110 may be, for example, a closed-circuit television (CCTV). However, the image pickup unit 110 is not limited to the CCTV. The image pickup unit 110 can be any conventional camera unit consisting of a plurality of optical lenses or pickup devices such as charge coupled devices (CCDs).

Referring to FIG. 2, the image pickup unit 110 may generate image data 200 of an area at a predetermined angle from a position at which the image pickup unit 110 is installed. Various subjects may be contained in the image data 200. The subjects may include various objects as well as people. In addition, the image pickup unit 110 may include a driver such as an actuator. The driver enables the image pickup unit 110 to rotate along a predetermined axis of rotation to capture images at different angles.

Generally, the image pickup unit 110 is installed at a certain position for crime prevention and security purposes. The image pickup unit 110 may be used to capture an image of an intruder who illegally intrudes into a private land or a personal area or a suspicious person and prepare for possible intrusion in advance or obtain personal information of the intruder who escaped.

However, the image pickup unit 110 may also capture an image of other’s privacy area depending on its photographing angle. In this case, there is a risk of invasion of privacy. For example, the image pickup unit 110 may be installed at a certain position to capture an image of the inside of a user’s house and an area around a fence of the house. In this case, an image of an area outside the house beyond the fence can also be captured. In particular, if an angle formed by the image pick-up unit 110 and the ground is small, a long-range image is captured. Thus, an image of the inside of other’s house may also be obtained.

Referring back to FIG. 1, to avoid an unintended invasion of privacy, the information setting unit 120 may set privacy information in advance, and the image editing unit 130 may edit image data obtained by the image pickup unit 110 based on the set privacy information and provide the image data having set privacy masks to a user.

For example, referring to FIG. 4, the image editing unit 130 may edit the image data 200 by taking into consideration the privacy of subjects P1 through P3 existing outside a target surveillance area and set privacy masks M1 through M3 at positions of the subjects P1 through P3 in the image data 200 such that the privacy masks M1 through M3 overlap the subjects P1 through P3. Since the privacy masks M1 through M3 mask the subjects P1 through P3, an unintended invasion of privacy can be avoided.

Subjects to be masked are not limited to people. The subjects to be masked may also include areas through which privacy can be violated, for example, objects such as windows of other people’s houses captured in the image data 200. When a subject is a person, a face region may be extracted using a face region extraction algorithm, and a privacy mask may be set only for the face region.

The face region extraction algorithm may use various known techniques including, but not limited to, a technique of detecting a face in a controlled background image using a 2D, monochromatic background image or a predefined fixed background image, a technique of detecting a face based on a face color and using a typical skin color, a technique of simply extracting a moving face region from image data, and a technique of accessing a neural network using statistical cluster information.

A privacy mask may color a region in an opaque color or a color that makes it impossible to identify a subject. However, the present invention is not limited thereto, and a mask region can also be generated by applying a mosaic effect to a subject.

The image editing unit 130 may employ an event-driven marking system. When an event occurs, the image editing unit 130 may generate or remove the mask regions M1 through M3 for the subjects P1 through P3.

For example, an event may occur when a photographing time of the image pickup unit 110 is night time. In this case, the image editing unit 130 may remove mask regions for subjects. For example, mask regions for all subjects in an image captured during crime-ridden hours from 10 p.m. to 4 a.m. may be removed regardless of area or position, or mask regions for subjects in some areas may be removed.

In another example, an event may occur when a subject is exposed to the image pickup unit 110 for more than a predetermined unit of time. In this case, the image editing unit 130 may remove a mask region for the subject. That is, when a subject is exposed for more than a predetermined time, it is perceived that the subject is highly likely to do a dangerous behavior. Therefore, a mask for the subject may be removed to obtain information about the subject.

In another example, privacy information may include information about a target surveillance zone and information about a privacy protection zone, and an event may occur when a subject enters the target surveillance zone. In this case, the image editing unit 130 may remove a mask region for the subject. That is, referring to FIG. 5 which will be described later, an image may be divided into a plurality of zones. When a subject enters a certain zone, a mask region for the subject may be set or removed.

When a subject that entered the target surveillance zone moves to the privacy protection zone, the image editing unit 130 may keep a unmask region for the subject in second image data while the second image data was processed to have privacy masks. That is, even after the subject explicitly leaves the target surveillance zone, the image editing unit 130 may keep the mask region for the subject removed. Thus, the subject can be continuously monitored.

A data buffer unit (not shown) may further be provided. The data buffer unit may temporarily store first image data corresponding to raw data without any privacy masks generated by the image pickup unit 110. The image editing unit 130 may replace the second image data of a subject before the subject enters the target surveillance zone with the first image data in the same time section as the second image data. That is, when an administrator monitors the second image data, the administrator should be able to track the entire path and behaviors of a subject already determined to be a dangerous person. To this end, raw data without privacy masks before the subject enters the target surveillance zone may be generated for the subject that entered the target zone.

The privacy information may further include information about an optional target surveillance zone. A mask region of the target surveillance zone and a mask region of the optional target surveillance zone may be set differently. For example, if an event occurs when the photographing time is night time as described above, a mask may be set for a subject in the target surveillance zone, and a mask for a subject in the optional target surveillance zone may be removed.

The image editing unit 130 may set the brightness of a mask region for a subject differently according to the type of an event that occurs. For example, in the case of an event that occurs when the photographing time is night time, since the luminance of captured image data is low, the image editing unit 130 may set the brightness of the mask region low. If the subject is exposed to the image pickup unit 110 for more than a predetermined time (e.g., more than 5 minutes), the image editing unit 130 may reduce the brightness of the mask region for the subject by a predetermined percentage, e.g., 30 %. The image editing unit 130 may also set the brightness of the mask region such that the brightness of the mask region is gradually reduced as the time during which the subject is exposed to the image pickup unit 110 increases.

Referring to FIG. 3, privacy information set by the information setting unit 120 may include coordinate data set as a range in the image data 200. The coordinate data may include 3D coordinate or distance information of subjects existing in the image data 200.

For example, if moving subjects existing in a first quadrant of X and Y axes are set as targets of privacy protection (i.e., as coordinate data in which privacy masks should be set) in the privacy information, the first through third subjects P1 through P3 existing in the first quadrant may be perceived as the targets of privacy protection. The image editing unit 130 may detect moving subjects by analyzing each frame of the image data 200 and set privacy masks at the positions of the subjects P1 through P3 existing in the first quadrant of a coordinate plane from among the moving subjects.

To this end, the image editing unit 130 may define an XY plane based on the ground on a Z axis (not shown). In addition, the image editing unit 130 may receive information about a photographing angle of the image pickup unit 110 and set the X axis and the Y axis in the image data 200 based on the received information.

Coordinates of each subject extracted based on the defined XY plane and the set X and Y axes may be determined based on a central point of the subject on the ground. However, the present invention is not limited thereto. When the bottom of a subject is higher than the ground, height information of the subject on the Z axis may also be taken into consideration. For example, since the second subject P2 is in contact with the ground, height information of the second subject P2 on the Z axis may be omitted. On the other hand, the first subject P1 located within a building is at a certain height from the ground. In this case, the privacy information may be set in view of the height of the first subject P1 on the Z axis.

In some embodiments, the position of the first subject P1 may be projected in the XY plane where Z=0. Thus, coordinate information in the XY plane only may be included in the privacy information without taking height into consideration.

The 3D coordinate information that forms the coordinate data included in the set privacy information is mapped to image data in the form of a 2D plane by the image editing unit 130, such that privacy masks can be set precisely.

When the photographing angle of the image pickup unit 110 changes, each axis of 3D coordinates which forms the coordinate data may rotate with respect to the angle at which the image data 200 was captured.

Therefore, since the axes that form the coordinate data rotate when the composition or focus of the image data 200 changes with respect to a reference line, privacy masks can be set precisely in various image data 200.

The system for processing an image to protect privacy according to the current embodiment will be described in more detail with reference to FIGS. 5 through 8B. FIG. 5 shows the image data of FIG. 2 divided into a plurality of zones. FIG. 6 shows picture-in-picture (PIP) data of a subject existing in a first zone of the image data of FIG. 5. FIG. 7 shows a privacy mask set for a subject existing in a third zone of the image data of FIG. 5. FIGS. 8A and 8B show a privacy mask set for a subject existing within a predetermined range in a second zone of the image data of FIG. 5.

The privacy information for the image data 200 may further include information about a plurality of zones into which a captured image in the image data 200 is divided.

For example, referring to FIG. 5, in the image data 200 obtained by the image pickup unit 110, a user’s residential area inside a boundary surface (fence) may be set as a first zone 200a, a public road area existing outside the residential space and used by pedestrians may be set as a second zone 200b, and other’s building area on the other side of the public road may be set as a third zone 200c. Then, a different privacy level may be set for each zone.

Referring to FIG. 6, the first zone 200a which is the user’s residential area is a target surveillance area to be actually monitored by the user. The first zone 200a is where no pedestrians are captured unless they trespass thereon. Therefore, PIP data 210 obtained by enlarging a sensed face region of a subject P4 in the first zone 200a may be added to the image data 200, so that the user can obtain information about the subject P4 in the first zone 200a. Depending on the user’s setting, the PIP data 210 may be displayed only when the subject P4 is sensed in the first zone 200a for a predetermined unit of time.

On the other hand, the second zone 200b which is the public road area used by general pedestrians is an optional target of surveillance. Therefore, depending on the user’s setting, privacy masks may or may not be applied to the subjects P2 and P3. The user can arbitrarily make the above settings using the information setting unit 120.

Referring to FIG. 7, the third zone 200c which is other’s building area is highly likely to have the problem of invasion of privacy since an image of the inside of the building can be captured. Therefore, the third zone 200c may be set as a target of privacy protection, and privacy masks may be applied to all subjects in the third zone 200c or to moving subjects only.

As described above, a sensed area may be divided into a plurality of zones in advance, and a different privacy level may be set for each zone. Also, each zone may be divided into a plurality of sections, and privacy masks may be set differently in each section.

For example, referring to FIGS. 8A and 8B, in the coordinate data mapped to the image data 200, the privacy mask M2 may be set only to a subject existing within a range of Y1 < Y < Y2 from among subjects in the second zone 200b. Therefore, no privacy mask may be set for a subject outside the Y-axis range even if the subject is located in the second zone 200b.

In addition, in the coordinate data mapped to the image data 200, the privacy mask M2 may be set only to a subject existing within a range of X1 < X < X2 and a range of Y1 < Y < Y2 from among the subjects in the second zone 200b. Therefore, no privacy mask may be set for a subject outside the X-axis range and the Y-axis range even if the subject is located in the second zone 200b.

In some embodiments, when a subject moves between zones having different privacy levels, the image editing unit 130 may sense this movement and set a privacy mask for the subject or remove a privacy mask set for the subject.

For example, referring to FIG. 7, the first subject P1 may leave the third zone 200c which is a privacy protection zone and then enter the second zone 200b or the first zone 200a. In this case, a privacy mask M1 that masks the first subject P1 may be removed, so that the first subject P1 can be identified.

Conversely, referring to FIG. 6, the fourth subject P4 may stay for a period of time in the first zone 200a which is under close surveillance and then enter the second zone 200b or the third zone 200c. In this case, the fourth subject P4 may be perceived as a dangerous person, and a privacy mask set for the fourth subject P4 may be removed regardless of the position of the fourth subject P4. Therefore, even if the fourth subject P4 enters the third zone 200c which is the privacy protection zone, the privacy mask for the fourth subject P4 may still remain removed.

Referring to FIG. 8A, the second subject P2 currently located in the range of Y1 < Y < Y2 in the second zone 200b may move to a position outside the range of Y1 < Y < Y2. In this case, the privacy mask M2 for the second subject P2 may be removed.

As described above, since a subject, in particular, a person, moves to a position in the image data 200 as time passes, a privacy mask for the subject may be set or removed in view of the position of the subject in each frame of the image data. In addition, various options may be set depending on circumstances. Therefore, a user can set, as desired, whether to set privacy masks in each zone and whether to set or remove a privacy mask for a subject when the subject moves between the zones.

A method of processing an image to protect privacy according to an embodiment of the present invention will now be described with reference to FIGS. 9 and 10. FIG. 9 is a flowchart illustrating a method of processing an image to protect privacy according to an embodiment of the present invention. FIG. 10 is a detailed flowchart illustrating a privacy information setting process of FIG. 9.

Referring to FIG. 9, the method of processing an image to protect privacy according to the current embodiment may include capturing image data (operation S120), loading preset privacy information (operation S130), determining whether a subject corresponding to the privacy information exists in the image data (operation S140), and generating a mask region for masking the subject (operation S150).

In some other embodiments, the privacy information may include coordinate data set as a range in the image data, and the coordinate data may include 3D coordinate information of subjects existing in the image data.

Referring to FIGS. 9 and 10, the privacy information is set according to a user’s circumstances (operation S110). In the drawings, the setting of the privacy information is performed before the capturing of the image data (operation S120). However, the present invention is not limited thereto, and the setting of the privacy information can also be performed after the capturing of the image data (operation S120). Also, privacy information can be newly set while a surveillance camera is being operated in a state where no privacy information is set, or the setting details of the existing privacy information can be modified.

As described above, image data captured by a surveillance camera may be divided into a plurality of zones in view of an installation position of the surveillance camera set by the user and surrounding circumstances. Even within one zone, a position corresponding to certain coordinates may be set, and a privacy mask may be generated only for a subject at the position. In addition, privacy masks or PIP data may be generated by applying a different privacy level to each of the zones.

The method of processing an image to protect privacy may further include setting the type of coordinates applied to the image data (operation S112). For example, in the coordinates applied to the image data, subjects existing in the image data may be treated as 3D coordinates. In this case, height information (Z-axis coordinates) from the ground may also be taken into consideration. That is, three axes may be set by analyzing the ground conditions, the disposition of buildings, etc. in the captured image data, and precise position information of subjects in the image data can be extracted based on the set three axes.

On the other hand, the subjects existing in the image data can also be treated as 2D coordinates. In this case, the subjects may be projected in a 2D XY plane (e.g., the ground) without regard to the difference in height between the subjects, and the position and distance information of each subject may be taken into consideration.

Coordinate data included in the privacy information may be set based on the set type of coordinates (operation S113). That is, the coordinate data may be generated by determining whether to mask or not mask a subject at which position in the image data. Later, whether to generate a privacy mask for a subject may be determined based on this coordinate data.

As described above, the coordinate data may be set as certain positions and may be set in each zone.

Various options may be set for zones and/or subjects included in the privacy information (operation S114). When a subject moves between the zones or when the subject moves within one zone, this movement may be sensed, and a privacy mask currently set for the subject may be generated or removed depending on the position of the subject. In addition, the type or range (e.g., the face or the whole body) of a privacy mask for a subject may be set. Also, a time period of stay may be set for each zone. Therefore, if a subject stays for more than the set time period in a zone, a privacy mask for the subject may be set or removed.

The privacy information generated as described above is stored (operation S115). Later, when image data is captured, the privacy information may be loaded to apply a privacy protection policy.

Next, image data is captured using an image pickup unit (operation S120). Once the image data is captured, the stored privacy information is read (operation S130). The privacy information can be altered, modified or deleted at any time by the user.

It is determined whether a subject corresponding to the privacy information exists by analyzing the image data (operation S140). Then, a privacy mask for the subject corresponding to the privacy information is generated to prevent the subject from being identified (operation S150).

As described above, first image data corresponding to raw data initially obtained by the image pickup unit may be stored and managed in a buffer unit.

Although not mentioned, other features of the method of processing an image to protect privacy according to the current embodiment may be substantially the same as the configuration and operation principle of the above-described system for processing an image to protect privacy, and thus a repetitive description thereof is omitted.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined by the following claims. The exemplary embodiments should be considered in a descriptive sense only and not for purposes of limitation.

Claims

A system for processing an image to protect privacy, the system comprising:

an image pickup unit generating first image data;

an information setting unit setting privacy information of the first image data; and

an image editing unit generating second image data by generating or removing a mask region which masks a subject by editing the first image data based on the privacy information,

wherein when an event occurs, the image editing unit generates or removes the mask region for the subject.
The system of claim 1, wherein the event occurs when a photographing time of the image pickup unit is night time, and the image editing unit removes the mask region for the subject.
The system of claim 1, wherein the event occurs when the subject is exposed to the image pickup unit for more than a predetermined unit of time, and the image editing unit removes the mask region for the subject.
The system of claim 1, wherein the privacy information comprises information about a first zone which is a target of surveillance and information about a second zone which is a target of privacy protection, wherein the event occurs when the subject enters the first zone, and the image editing unit removes the mask region for the subject.
The system of claim 4, wherein when the subject that entered the first zone moves to the second zone, the image editing unit keeps the mask region for the subject removed in the second image data.
The system of claim 4, further comprising a data buffer unit temporarily storing the first image data generated by the image pickup unit.
The system of claim 6, wherein the image editing unit replaces the second image data before the subject enters the first zone with the first image data in a same time section as the second image data.
The system of claim 4, wherein when the subject having a face region is sensed in the first zone for a predetermined unit of time, the image editing unit adds picture-in-picture (PIP) data obtained by enlarging the face region to the second image data.
The system of claim 1, wherein the privacy information comprises information about a first zone which is a target of surveillance, information about a second zone which is a target of privacy protection, and information about a third zone which is an optional target of surveillance, wherein the image editing unit sets the mask region for the subject that enters the second zone and the mask region for the subject that enters the third zone differently.
The system of claim 1, wherein the image editing unit sets brightness of the mask region for the subject differently according to a type of the event.
A method of processing an image to protect privacy, the method comprising:

generating first image data;

setting privacy information of the first image data; and

generating second image data by generating or removing a mask region which masks a subject by editing the first image data based on the privacy information,

wherein the generating of the second image data further comprises generating or removing the mask region for the subject when an event occurs.
The method of claim 11, wherein the event occurs when a photographing time of an image pickup unit is night time, and the mask region for the subject is removed in the generating of the second image data.
The method of claim 11, wherein the event occurs when the subject is exposed to the image pickup unit for more than a predetermined unit of time, and the mask region for the subject is removed in the generating of the second image data.
The method of claim 11, wherein the privacy information comprises information about a first zone which is a target of surveillance and information about a second zone which is a target of privacy protection, wherein the event occurs when the subject enters the first zone, and the mask region for the subject is removed in the generating of the second image data.
The method of claim 14, wherein in the generating of the second image data, when the subject that entered the first zone moves to the second zone, the mask region for the subject is kept removed in the second image data.
The method of claim 14, further comprising temporarily storing the first image data generated by the image pickup unit.
The method of claim 16, wherein in the generating of the second image data, the second image data before the subject enters the first zone is replaced with the first image data in a same time section as the second image data.
The method of claim 14, wherein in the generating of the second image data, when the subject having a face region is sensed in the first zone for a predetermined unit of time, PIP data obtained by enlarging the face region is added to the second image data.
The method of claim 11, wherein the privacy information comprises information about a first zone which is a target of surveillance, information about a second zone which is a target of privacy protection, and information about a third zone which is an optional target of surveillance, wherein in the generating of the second image data, the mask region for the subject that enters the second zone and the mask region for the subject that enters the third zone are set differently.
The method of claim 11, wherein in the generating of the second image data, brightness of the mask region for the subject is set differently according to a type of the event.