JP2005244550A - Method for generating monitored image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a device for generating a monitored picture signal that masks an image in which the protection of privacy is required with an image that does not interfere with the monitoring of a monitored image. <P>SOLUTION: A mask region including a photography-forbidden region is set onto a photography screen when a place, where photography is forbidden to protect the privacy, is included in the visual field of a camera for photographing an object to be monitored; a still image in the mask region to be photographed is stored into a memory 172 and an angle comparison means 161 detects whether the mask region is included in the visual field of a monitoring camera, when a substitution monitored image, where the mask region is substituted for an image in which the privacy is protected, is obtained while the mask region is included in the photography after the mask region is set; and the still image in which the image in the mask region is stored into the memory 172 is substituted by an image substitution means 142 when the inclusion of the mask region is detected, thus realizing the device for generating the monitored picture signal. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a method for generating a monitoring image having a function of previously setting a subject area for which imaging is prohibited for privacy protection and masking an image of the set portion.

  In recent years, a camera called a dome camera in which a camera and a camera turntable are housed in a dome-shaped housing has been used as a surveillance camera. This dome camera shoots a surveillance video while performing horizontal rotation (panning) and vertical rotation (tilting) by the movement of a turntable to which the camera is fixed. An endless rotation of 360 degrees is performed in the pan direction, and a vertical rotation of 0 to 90 degrees is performed in the tilt direction. The dome camera is installed on the ceiling of the building, and the user operates the controller connected thereto to monitor an image with a desired viewing angle.

FIG. 6 shows a configuration of a monitoring system using a conventional dome camera.
In the dome camera 4, pan and tilt shooting directions are changed by a controller 5. An image captured by the dome camera 4 is displayed on the monitor 6. The controller 5 has a numeric keypad 51 for instructing a photographing direction preset in the dome camera 4 and a joystick 52 for rotating by operation.

  When the dome camera 4 is installed, a number as a preset position is assigned by a numeric keypad 51 to a place where normal monitoring is performed, such as an entrance / exit of a store, a cash register, and a window. When the number assigned from the numeric keypad 51 of the controller 5 is input, the dome camera 4 faces the preset position of that number. The user monitors the inside of the store by displaying the photographed video on the monitor 6.

  When monitoring a place other than the preset position, the joystick 52 of the controller 5 is operated, and the camera can be directed in an arbitrary direction by panning and tilting the shooting direction of the camera. That is, when the joystick 52 is slightly tilted leftward or rightward, the panning is performed in that direction. Increasing the tilt angle increases the pan speed. Similarly, it is tilted back and forth and rotated in the tilt direction. The user operates the joystick while viewing the image on the monitor 6, and when the desired image is displayed, the joystick 52 is returned to the neutral position (vertical position) to stop the rotation of the camera.

  In many cases, incidents are resolved based on recorded images that are taken and recorded by the surveillance camera that can freely control the surveillance location. On the other hand, monitoring and recording of images taken by a surveillance camera also infringe on personal privacy. It is desirable that the surveillance camera is equipped with a means for preventing personal privacy from being violated.

  Patent Document 1 has the following description. In a surveillance camera device including a surveillance camera capable of 360 ° pan rotation and tilt rotation exceeding 90 ° and its control device, the surveillance camera holds mask data for masking a privacy zone shown in an image. The surveillance camera masks a part of the image taken according to the mask data. Since the masked image only masks a part of the captured image, privacy can be protected without impairing the monitoring function. In addition, the monitoring camera device has masking data in the monitoring camera itself so that a time delay related to the masking process of the captured image does not occur.

A dome camera is installed outdoors for monitoring public places. When installing outdoors, in order to protect privacy, a private mask is set so that nearby private houses are not displayed on the screen. For the private mask, a portion for which shooting is prohibited is set while viewing an image shot in advance, and the set mask data is stored in the memory of the dome camera. When shooting the area set in the private mask with the dome camera, the control unit of the dome camera compares the camera direction (pan, tilt) with the mask data, and only the area is white or gray on the screen. The display of the shooting screen is prohibited by painting with.
JP 2001-69494 A

However, in such a conventional privacy protection method, when the user looks at the monitor screen, the portion masked at a glance is known, and the mask portion is interested. Since the user gazes at the peripheral image of the mask that is different from the place to be monitored, the monitoring image is used for purposes other than monitoring.
In addition, since the mask screen is clearly different from the captured screen, there is a problem that monitoring cannot be sufficiently performed because the monitoring attention is distracted.

  Therefore, the present invention masks an image portion that needs privacy protection using an image that does not hinder the monitoring image. As a masking image, a still image of the same place taken in advance is used. It is intended to provide a monitoring image generation method for generating a monitoring image suitable for a monitoring purpose by using a still image that has been confirmed as having no privacy problem.

The present invention comprises the following means in order to solve the above problems.
That is,

When the field of view of the camera that shoots the monitoring target includes a place where shooting is prohibited for privacy protection, a mask area including the position where shooting is prohibited is set on the shooting screen, and the shooting is performed after setting the mask area. When a mask region is included, a monitoring image generation method for obtaining a replacement monitoring image in which the mask region is replaced with a privacy-protected image,
A first step (172) of storing in the memory a still image of the mask area obtained by photographing;
A second step (161) for detecting whether or not the mask region is included in the field of view of the monitoring camera;
If it is detected that the mask area is included in the second step, a third replacement monitoring image is obtained by replacing the image of the mask area with the still image stored in the memory in the first step. Step (142);
A method for generating a monitoring image, comprising:

According to the monitoring image generation method of the present invention, the following effects can be provided.
That is,

  Since the replacement monitoring image generated by the monitoring image generation method of the present invention is displayed by replacing the mask area including the area where shooting is prohibited for privacy protection with a still image shot in advance. Since the monitoring image can be watched without conspicuous that the prohibited portion is masked, it is possible to concentrate on the video watching of the portion to be monitored and to perform a smooth monitoring work.

  Hereinafter, preferred embodiments of the method for generating a monitoring image according to the present invention will be described.

FIG. 1 shows a schematic configuration of a still image capturing mask device equipped with the monitoring image generation method, and the operation will be described with reference to the drawings.
The still image capturing mask device shown in FIG. 1 includes an imaging unit 13, a mask processing unit 14, a pan head driving unit 15, a camera CPU 16, a built-in memory 17, a dome camera 21, a joystick 21, a numeric keypad 22, a TELE button 23, and a WIDE button. 24, a menu button 25, and a controller 2 including a set button 26. The dome camera 1 and the controller 2 are connected by a cable, a communication protocol such as RS-485 is used, and the operation of the dome camera 1 is controlled by the controller 2. The video output of the still image capturing mask device is input to the monitor 3.

  First, an image in the visual field is captured by the imaging unit 13, and an image signal obtained by capturing is supplied to the mask processing unit 14. The mask processing unit 14 supplies the input image to the monitor 3 as it is when the supplied image signal is an image in which an area that should not be photographed for private protection is not set in the visual field area. Display it there.

  Next, when the joystick 21 of the controller 2 is operated, a pan signal for rotating the viewing angle of the imaging unit 13 left and right generated according to the operation or a tilt signal for changing the viewing angle to the upper limit is a camera. It is supplied to the CPU 16. These pan and tilt signals are generated as drive signals for changing the direction of the pan head on which the imaging unit 13 is fixed in the camera CPU 16.

  The drive signal is supplied to the camera platform drive unit 15, and the camera platform drive unit 15 changes the direction from side to side and up and down. The pan / tilt head and the image pickup unit 13 together with the pan / tilt head drive unit 15 shoots an image of the visual field that is changed while changing the direction. The captured image is displayed on the monitor 3.

  Here, if an image taken with the direction changed is an area where a person lives, for example, the area should not be taken for private protection. In the private protection area, the direction of the area and the area size (viewing angle) in the direction are set by operating the button of the controller 2. Here, the set area is called a mask area.

  When the image of the mask area specified by the set direction and size is captured by the imaging unit 13, it is checked whether or not a person is captured in the captured image, and when private information is not included The one frame image (still image) in the mask area is stored in a predetermined storage area of the built-in memory 17. Direction information relating to panning and tilting of the mask area and information relating to the size are also stored in the built-in memory 17. The still image, the direction information, and the size information are assigned management numbers for a plurality of mask areas, and are stored in a predetermined area of the built-in memory 17.

  Thereafter, when the controller 2 is operated and an area including the mask area is photographed, the camera CPU 16 detects that the mask area portion is photographed. The mask processing unit 14 replaces the detected image of the mask area with the still image related to the mask area at the same position stored in the internal memory 17, and the replacement monitoring image obtained by the replacement is supplied to the monitor 3. Displayed.

  It is necessary to replace the still image with the photographed image without giving a sense of incongruity to the photographed image. Specifically, the edge portion smoothing process is performed to reduce the alignment error of the edge portion of the inserted image, thereby making the boundary portion inconspicuous. Further, the image quality is corrected so as to obtain an image having the same brightness, the same contrast, and the same color temperature with respect to the photographed image, and then superimposed (replaced).

  The camera CPU 16 detects a condition to be corrected by comparing the image captured by the imaging unit 13 and the still image stored in the built-in memory 17, and the still image corrected according to the detected condition is masked by the mask processing unit 14. To supply. Therefore, a replacement monitoring image is generated by replacing the image portion of the mask area with a still image.

The schematic configuration and operation of the still image capturing mask device have been described above. Next, a mask area setting method will be described.
FIG. 2 shows and explains the respective screens (a) to (f) sequentially displayed on the monitor 3 when the mask area is set.

  First, the user operates the joystick 21 of the controller 2 to take an image including an area to be designated as a mask area, and displays the taken image on the monitor 3. Next, the menu button 25 of the controller 2 is operated to display the “MENU” screen of the screen (a).

  A screen (a) displayed on the monitor 3 in FIG. 3 is a menu screen displayed when the menu button 25 of the controller 2 is operated. The menu mode includes a control related to the focus control of the imaging unit 13, an iris mode for brightness control of the captured image, a white balance related to the color temperature of the captured image, and a private mask mode related to setting of the mask area.

  The user operates the joystick 21, moves the cursor displayed by the mark “>” up and down, selects “PRIVATE MASK”, and presses the set button 26. An input request for ID (Identification) on the screen (b) is displayed. The user inputs the ID number registered in advance by operating the numeric keypad 22.

  When it is detected that the input ID matches that of the registered regular user, the “PRIVATE MASK” screen of screen (c) is displayed on the monitor 3. Use the joystick 21 to move the cursor, select “MASK NUMBER” from the “PRIVATE MASK” screen, and press the set button 26. The “MASK NUMBER” screen of screen (d) is displayed.

  In this embodiment, eight mask areas can be set for one dome camera. When there is a mask area that has already been set, a “*” mark is displayed next to the number on the “MASK NUMBER” screen. In this example, since the mask areas from 1 to 3 are already set, the mark “>” is moved and the number 4 of the new mask area is selected.

  Next, the set button 26 is pressed to display the “MASK NO4” screen on the screen (e). Then, “MASK EDIT” is selected on the “MASK NO4” screen and the set button 26 is pressed. The screen (f) is displayed by the operation. Screen (f) is a screen for setting a mask area.

  A translucent mask screen is displayed at the center of the screen (f). The joystick 21 is used to move the mask screen up / down / left / right. The position is set to a position to be set as a mask area on the screen imaged by the imaging unit 13. Further, the TELE button 23 is pressed to enlarge the mask, or the WIDE button 24 is pressed to reduce the mask. The mask area is set so that the masked area includes the private protection area.

  After setting the mask area, the set button 26 is pressed to determine the area. In accordance with the determination, the camera CPU 16 obtains the angle information related to the pan and the angle information related to the tilt from the angle sensor (not shown) built in the camera platform drive unit 15.

  On the other hand, the mask area may be set at a position moved from the center of the screen, so the horizontal movement amount is converted into a horizontal angle to correct the horizontal angle, and the vertical movement is performed in the same way. The vertical angle data is corrected based on the amount. Based on the corrected angle data, the mask area is obtained as angle data for displaying in the center of the monitor screen.

The corrected horizontal and vertical angle data and the size (viewing angle) data relating to the mask area are stored in the built-in memory 17 together with the mask number.
Next, the image of the mask area photographed by the photographing unit 13 is checked by the user to determine whether or not information relating to private protection is included. The stored image is stored in the internal memory 17 in an area for storing the fourth still image, for example.

Next, a method for generating and displaying a replacement monitoring image by replacing a previously stored still image with an image of a place where a mask area is set among images taken by the dome camera 2 will be described.
FIG. 3 is a block diagram showing the functions of the camera CPU 16 and the mask processing unit 14, and the operation will be described with reference to the drawings.

  In the figure, the mask processing unit 14 includes an outline processing unit 141 and an image replacement unit 142, the camera CPU 16 includes an angle comparison unit 161, an image comparison unit 162, an image correction unit 163, and a position correction unit 164, and a built-in memory. 17 includes functions of an angle storage unit 171 and a still image storage unit 172, respectively.

  First, when the controller 2 is operated and the imaging unit 13 is panned and tilted to be photographed, the angle information of the mask area stored in the angle storage unit 171 and the horizontal and vertical angle information detected by the camera platform driving unit 15 Are compared. When it is detected that an image including a mask area is shot on the monitor 3, a still image related to the mask area stored in the still image storage unit 172 is read out, and the read still image is stored in the mask area. The position correction unit 164 corrects the position to match the position.

  The image with the corrected display position and the image input from the photographing unit 13 are compared by the image comparison unit 162, and differences in brightness, contrast, and color temperature are detected. The still image whose display position has been corrected by the position correction unit 164 is corrected to an image similar to the image quality of the image captured by the imaging unit 13 by the image correction unit 163 and supplied to the image replacement unit 142.

  In the image replacement unit 142, the image of the mask area captured by the imaging unit 13 is replaced with a still image whose image quality has been corrected by the image correction unit 163. Here, if there is an error in the position of the image in the part to be replaced, the image in that part becomes a double image or an image lacking the boundary, which is unnatural.

  Therefore, the contour processing unit 141 obtains the display position of the mask region part based on the angle information supplied from the angle storage unit 171, and displays the image at the obtained boundary line from the outside of the part surrounded by the boundary line. Performs a softening process for the boundary line that gradually attenuates toward the inside. The image correction unit 163 performs a softening process based on the reverse characteristic of the boundary line.

FIG. 4 shows an example of a screen when the mask area of the captured image is replaced with a softened still image.
In the figure, a captured image 30 is displayed on the monitor 3. A mask area exists at the center of the screen. The area is replaced with the still image 31 and displayed. In order to make the boundary of the still image 31 inconspicuous, the still image 31 is displayed through the boundary 32 subjected to the software processing.

Next, the boundary softening process will be further described.
FIG. 5 shows a relationship between an image signal in one scanning line 33 indicated by a two-dot chain line and a level coefficient related to the attenuation amount constituting the screen 3 shown in FIG. The horizontal axis is the horizontal position of the screen, and h2 to h5 are mask areas. h <b> 1 to h <b> 3 and h <b> 4 to h <b> 6 are portions of the boundary 32.

  As for the image signal of the captured image 30, the level of the captured signal remains the same from the left end h0 to h1 of the screen 3. It is attenuated between h1 and h3. h3 to h4 are attenuated black levels, which gradually increase from h4 to h6 and reach a signal level without attenuation at h6. From h6 to h9 at the right end of the screen is the signal level of the imaged state.

  The level characteristic of the still image 31 shows a characteristic (attenuation characteristic) opposite to that of the captured image 30. When the image replacement unit 142 adds the captured image 30 and the still image 31 to which the level characteristics are given, the boundary region is softened (fade-in and fade-out), and the composite image in which the still image 31 is inserted into the captured image 30 As obtained. The composite image subjected to the boundary softening process reduces a sense of discomfort with respect to two images having different image quality taken at different times or two images including some positional errors.

  The above-mentioned boundary line softening process in the image correction unit 163 is a softening process that gradually increases from the outside to the inside of the portion surrounded by the boundary line of the still image. The image signal subjected to the software processing is obtained, and the addition processing is performed by the image replacement unit 142, whereby a replacement monitoring image subjected to a desired replacement processing is obtained.

The configuration and operation of the still image capturing mask device have been described in detail above.
Note that the still image storage unit 172 is described as storing captured images. The image signal is a digitally converted image signal, and may be a compression-coded image signal defined by JPEG (Joint Photographic Coding Experts Group) or the like.

  Further, in order to absorb the error of the angle detection element attached to the pan head drive unit 15, there is a method in which the position detection of the still image with respect to the captured image is performed using the motion vector method and the position of the still image is fine-tuned. Motion vector detection is widely used mainly in MPEG (moving picture experts group), and it is possible to improve performance with a small cost by using mass-produced elements.

  Further, the above-described mask area has been described as a rectangular area. The mask area may be a shape other than a rectangle, such as a parallelogram, trapezoid, polygon, circle, and ellipse, whose shape is uniquely determined by a predetermined parameter, and the angle storage unit stores parameters related to the shape and size. 171 may be stored. The shape data may be created by operating the joystick 21, stored in the built-in memory 17, and used.

  In addition, the configuration and the operation in the case where the imaging unit 13 is driven by the pan head driving unit 15 and rotated up and down and left and right have been described. Even when the imaging unit 13 rotates only to the left and right, the position detection and correction of the still image with respect to the above-described captured image can be performed. Even in the case where the camera platform 13 is not provided and the imaging unit 13 is fixed, an image currently being captured with respect to a still image captured in the past depends on a change in temperature of the optical system and a change over time. There may be a shift in the imaging position.

  Using the technology described in detail in this embodiment, a still image can be generated by correcting a minute change in the captured image position and a temporal change in image quality due to a difference in shooting time, and the mask area of the image where the still image is shot By using it as an image, it is possible to realize a still image capturing mask device that suitably performs a monitoring operation.

Further, the present invention includes a program loaded on the camera CPU 16 for controlling the still image capturing mask device. The program is as follows.
That is,

When the field of view of the camera that shoots the monitoring target includes a place where shooting is prohibited for privacy protection, a mask area including the position where shooting is prohibited is set on the shooting screen, and when shooting after setting the mask area, When a mask region is included, a monitoring image generation program for controlling a still image capturing mask device configured to obtain a replacement monitoring image in which the mask region is replaced with a privacy-protected image,
A first step of storing a still image of the mask area obtained by photographing in a memory;
A second step of detecting whether the mask region is included in the field of view of the surveillance camera;
If it is detected in the second step that the mask area is included, the replacement monitoring image obtained by replacing the image of the mask area with the still image stored in the memory in the first step is obtained. Steps,
And a monitoring image generation program for controlling the apparatus.

  The present invention can be used to realize a surveillance camera device that can perform private protection processing by replacing an image of a predetermined portion of an image to be captured with an image of the same viewing angle taken at a different time.

It is the figure which showed schematic structure of the still image imaging mask device based on the Example of this invention. FIG. 6 is a diagram illustrating a display screen for setting a mask area according to an embodiment of the present invention. It is the figure which showed the component part of the still image imaging mask device based on the Example of this invention in detail. It is a figure for demonstrating the display operation | movement of the still image imaging mask device based on the Example of this invention. It is the figure which showed the level characteristic of the image signal which concerns on the soft boundary process based on the Example of this invention. It is a figure which shows the structure of the monitoring system using the conventional dome camera.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Dome camera 2 Controller 3 Monitor 13 Imaging part 14 Mask process part 15 Pan head drive part 16 Camera CPU
17 built-in memory 21 numeric keypad 22 joystick 23 TELE button 24 WIDE button 25 menu button 26 set button 30 captured image 31 still image 32 boundary 33 scanning line 141 contour processing unit 142 image replacement unit 161 angle comparison unit 162 image comparison unit 163 image correction unit 171 Angle storage unit 172 Still image storage unit

Claims (1)

When the field of view of the camera that shoots the monitoring target includes a place where shooting is prohibited for privacy protection, a mask area including the position where shooting is prohibited is set on the shooting screen, and the shooting is performed after setting the mask area. When a mask region is included, a monitoring image generation method for obtaining a replacement monitoring image in which the mask region is replaced with a privacy-protected image,
A first step of storing a still image of the mask area obtained by photographing in a memory;
A second step of detecting whether the mask area is included in the field of view of the monitoring camera;
If it is detected in the second step that the mask area is included, the replacement monitoring image obtained by replacing the image of the mask area with the still image stored in the memory in the first step is obtained. Steps,
A method for generating a monitoring image, comprising:

Images