JP2005294912A - Method of generating monitoring image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a method of generating a monitoring image which can synthesize a private region image photographed in the past with an image other than a currently photographed private region, and naturally displaying the composite image. <P>SOLUTION: A region deciding means 121 sets a boundary frame having a predetermined boundary width in the photographing region of a monitoring camera, and an image extracting means 124 extracts the private region image designated by the boundary frame and a region excluding image surrounded by the inner side of the boundary frame from a photographic image of the monitoring camera. A motion determining means 129 compares the private region image with the past private region past image, and the private region image photographed in the past is acquired as a private protection image if the position of an object has not changed, and the private region past image is acquired as the private protection image if it has changed. An adder 337 sequentially switches the private protection image and the private region excluding image to generate a replaced monitoring image. Thus, the monitoring image signal generating device is realized. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method for generating a monitoring image having a function of previously setting a subject area for which imaging is prohibited for privacy protection and masking an image of the set portion.

  In recent years, a camera called a dome camera in which a camera and a camera turntable are housed in a dome-shaped housing has been used as a surveillance camera. This dome camera shoots a surveillance video while performing horizontal rotation (panning) and vertical rotation (tilting) by the movement of a turntable to which the camera is fixed. An endless rotation of 360 degrees is performed in the pan direction, and a vertical rotation of 0 to 90 degrees is performed in the tilt direction. The dome camera is installed outdoors or on the ceiling of a building, and the user monitors an image with a desired viewing angle by operating a connected controller.

FIG. 6 illustrates a configuration diagram of a monitoring system using a network camera.
An image captured by the network camera 31 is compressed and encoded by a method such as MPEG-4 and supplied to the PC 33 via the network 32. Therefore, an image signal obtained by decoding is displayed on the monitor 34. On the monitor screen 34 of the PC 33, in addition to the image taken by the network camera 31, a numeric keypad 34a, up / down / left / right buttons 34b, a zoom button 34c, and a menu button 34d are displayed at the bottom of the screen as an operation unit for instructing the shooting direction. ing.

  The network camera 31 is assigned a number for a preset position and registers the direction for shooting in a place where normal monitoring is performed, such as a store entrance / exit, a cashier, and a window. When the number of the numeric keypad 34a on the monitor screen 34 is clicked with the mouse, the network camera 31 takes a picture by facing the preset position of that number.

  When monitoring a position other than the preset position, the up / down / left / right button 34b of the monitor screen 34 is operated to point the camera in an arbitrary direction. When the up / down / left / right button 34b is clicked, the network camera 31 rotates in the tilt direction by an angle corresponding to the number of times the button is clicked. Pan left and right by an angle corresponding to the number of clicks on the button. While viewing the image displayed on the monitor, the operator stops clicking the up / down / left / right button 35 and stops the rotation of the camera when the desired image is displayed on the monitor. When the user wants to enlarge or reduce the screen, the zoom lens of the network camera 31 is driven when the zoom button 36 is clicked.

  The network camera 31 may be installed outdoors to introduce sightseeing spots. The network camera is often accessible by anyone over the Internet. In order to protect privacy, it is necessary to set a private mask in order to prevent neighboring private houses from being displayed on the screen.

  Patent Document 1 has the following description. In a surveillance camera device comprising a surveillance camera capable of 360-degree pan rotation and tilt-rotation exceeding 90 degrees and its control device, the surveillance camera holds mask data for masking a privacy zone reflected in an image. The surveillance camera masks a part of the image taken according to the mask data. Since the masked image only masks a part of the captured image, privacy can be protected without impairing the monitoring function. In this surveillance camera device, the surveillance camera itself is provided with masking data so as not to cause a time delay related to the masking processing of the photographed image.

The private mask sets a portion for which shooting is prohibited while viewing an image shot in advance, and stores the set mask data in the memory of the dome camera. When shooting the area set in the private mask with the dome camera, the control unit of the dome camera compares the camera direction (pan, tilt) with the mask data, and only the area is white or gray on the screen. The shooting screen is not displayed by painting with the mask data.
JP 2001-69494 A

  However, since the conventional monitoring system does not consider the difference in importance even for the masked image portion, the simple background portion is recorded with the same amount of information as the important portion. There is a problem that it is difficult to store an image for monitoring for a long time until the incident is solved due to waste of recording capacity.

  Also, in a privacy protection method using a surveillance camera placed in a public place and using a private mask, when the user looks at the monitor screen, the part masked at a glance is known. The user is interested in the mask portion. Since the user gazes at the peripheral image of the mask that is different from the place to be monitored, the monitoring image is used for purposes other than monitoring. There has been a problem that sufficient monitoring cannot be performed because the monitoring attention on the monitoring image is distracted.

  Therefore, the present invention masks an image portion that needs privacy protection using an image that does not hinder the monitoring image. The masking image seeks to provide a method for generating a monitoring image that is generated as a monitoring image suitable for a monitoring purpose when the captured image can be displayed as an image with little data amount and no privacy problem. Is.

The present invention comprises the following means in order to solve the above problems.
That is,

When the captured image captured in the private area set in the capturing area captured by the surveillance camera moves within a predetermined time, it is replaced with the image captured when there is no movement instead of the captured image. A method of generating a monitoring image generated as an image,
A first step (121, 122, 123) of setting the private area within a border frame having a predetermined border width within the photographing area of the surveillance camera;
The private area image specified by the boundary frame set in the first step is acquired from the captured image of the monitoring camera, and specified by the boundary frame set in the first step from the captured image of the monitoring camera. A second step (124, 125) of acquiring a private area removal image that is an area that is not a private area
When the private area image acquired in the second step is compared with the private area past image acquired in the past, and it is detected that the position of the subject of the private area image has not changed between the past and the present The private area image is acquired as an image taken when there is no movement, or the subject position does not change when it is detected that the subject position of the private area image has changed between the past and the present A third step (13, 129, 126) of acquiring the private area past image acquired as a captured image when there is no movement;
The image captured when there is no movement acquired in the third step and the private area removed image acquired in the second step are gradually switched within the boundary area frame set in the first step. And a fourth step (335, 336, 337) for generating the private protection monitoring image.

According to the monitoring image generation method of the present invention, the following effects can be provided.
That is,

  The replacement monitoring image generated by this generation method compares the image captured in the private area with the private area past image captured in the past, and the private area image or the private area past image that does not move in the imaging target and the non-private area The replacement monitoring image is generated in such a manner that the private region removed image is switched with a predetermined region width. Therefore, when the past private image captured in the past and the private region removed image currently captured are combined, it is difficult to be affected by the viewing angle fluctuation or viewing angle error of the surveillance camera caused by the time difference, and the private region A method of generating a monitoring image that can be displayed as a natural image can be realized.

Hereinafter, preferred embodiments of the method for generating a monitoring image according to the present invention will be described.
FIG. 1 shows a schematic configuration of a network camera system equipped with the monitoring image generation method, and the operation thereof will be described with reference to the drawings.

  The network camera system shown in FIG. 1 includes a dome camera 1 including an area specifying unit 11, a camera CPU 12, a built-in memory 13, a pan head driving unit 14, a network camera 15, an encoding unit 16, and an interface 17, a PC 33, and a numeric keypad 35a. The controller 35 includes a set button 35b, a joystick 35c, a WIDE button 35D, a TELE button 35E, and a menu button 35f.

  The dome camera 1 and the controller 35 are connected via a network 32 and operate using a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol) or UDP (User Datagram Protocol). The operation of the dome camera 1 is controlled by the controller 35. Video output from the network camera system is input to the monitor 34 and displayed.

  First, an image in the field of view is photographed by the network camera 15, and an image signal obtained by photographing is coded by a coding unit 16 by a compression coding method defined in, for example, MPEG (moving picture experts group) -4. It becomes. The bit stream obtained by encoding is transmitted to the PC 33 via the interface 17 and the network 32.

  The bit stream input to the PC 33 is decoded, and the video signal obtained by the decoding is input to the monitor 34 and displayed. The display screen includes a monitoring area to be monitored and a private area that should not be monitored. The joystick 35c of the controller 35 is operated by the user.

  A control signal generated by the controller 35 by the operation is transmitted to the camera platform drive unit 14 via the PC 33, the network 32, the interface 17, and the camera CPU 12. The transmitted control signal rotates the pan / tilt head drive unit 14 in the vertical and horizontal directions. The shooting direction of the network camera 31 is adjusted so that the image shot at the rotated position becomes the center position of the private area.

  Next, the WIDE button 35D and the TELE button 35E are operated to specify the size of the private area on the displayed image. The private area is determined by pressing the set button 35b. The determined position information of the private area is obtained by the PC 33 based on the angle information output from the rotational position detection unit (not shown) detected by the camera platform drive unit 14 and the position information displayed on the screen. The private area position information is supplied to the area designating unit 11 and stored therein.

  The monitoring image input from the network camera 15 to the encoding unit 16 is divided into a private region image and a monitoring region image that is another region based on the private region position information stored in the region specifying unit 11. Each region image is designated as a video object (VO; Video Object) having an arbitrary shape defined by MPEG-4.

  The video object is a unit of an image having a boundary which is independent for each object and can be optimally encoded. The image object can be operated independently for each object, and the image definition for each object can be set, the image position can be moved, enlarged, and deleted.

  Here, the image object in the monitoring area is encoded with high quality as low-compression image data. The image object in the private area is encoded as highly compressed image data. The encoded image data is time-division multiplexed and transmitted to the PC 33 via the interface 17 and the network 32.

  The transmitted signal is received by the PC 33. In the PC 33, the received signal is decoded as each arbitrarily shaped video object based on the MPEG-4 standard. Decoded arbitrary-shaped video objects are arranged and synthesized in a predetermined arrangement by a method described later, and output as one monitoring image.

  Compared to encoding the image portion to be monitored to have a sufficient amount of information, the private area image, for example, lowers the resolution (number of pixels) of the image, reduces the frame rate, and It is generated as a highly compressed signal by increasing the compression rate. The private area image is an image that prevents the movement of the photographed person from being discriminated. For example, an image of one frame or less may be transmitted per minute. The highly compressed signal reduces the transmission band of the network 32 and reduces the capacity of the storage medium when the transmitted monitoring image data is stored.

The general configuration and operation of the network camera system have been described above. Next, a mask area setting method will be described.
FIG. 2 shows and explains the respective screens (a) to (f) sequentially displayed on the monitor 34 when the mask area is set.

  First, the user operates the joystick 35c of the controller 35 to take an image including an area to be designated as a mask area, and displays the taken image on the monitor 34. Next, the menu button 35f of the controller 35 is operated to display the “MENU” screen of the screen (a).

  A screen (a) displayed on the monitor 34 in FIG. 5 is a menu screen displayed when the menu button 35 f of the controller 35 is operated. The menu mode includes a focus control of the network camera 15, an iris mode related to brightness control of the captured image, a white balance related to the color temperature of the captured image, and a private mask mode related to the mask area setting.

  The user operates the joystick 35c, moves the cursor displayed by the “>” mark up and down, selects “PRIVATE MASK”, and presses the set button 35b. An input request for ID (Identification) on the screen (b) is displayed. The user inputs a previously registered ID number by operating the numeric keypad 35a.

  If it is determined that the input ID matches that of the registered regular user, the “PRIVATE MASK” screen of screen (c) is displayed on the monitor 34. Move the cursor using the joystick 35c, select "MASK NUMBER" from the "PRIVATE MASK" screen, and press the set button 35b. The “MASK NUMBER” screen of screen (d) is displayed.

  In the embodiment shown here, eight mask areas can be set for one network camera. When there is a mask area that has already been set, a “*” mark is displayed next to the number on the “MASK NUMBER” screen. In this example, since the mask areas from 1 to 3 are already set, the mark “>” is moved and the number 4 of the new mask area is selected.

  Next, the set button 35b is pressed to display the “MASK NO4” screen on the screen (e). Select "MASK EDIT" on the "MASK NO4" screen and press the set button 35b. Screen (f) is displayed. Screen (f) is a screen for setting a mask area.

  A translucent mask screen is displayed at the center of the screen (f). The joystick 35c is operated to move the mask screen up / down / left / right. The screen shot by the network camera 15 is adjusted to the center position to be set as the mask area. Further, the TELE button 35E is pressed to enlarge the mask, or the WIDE button 35D is pressed to reduce the mask. The mask area is set so that the masked area includes the private protection area.

  After setting the mask area, the set button 35b is pressed to determine the mask area. In accordance with the determination, the camera CPU 12 obtains angle information data related to panning and tilting from an angle sensor (not shown) built in the pan head driving unit 14.

  On the other hand, since the mask area may be set to a position moved from the center of the screen, the horizontal movement amount is converted into a horizontal angle to correct the horizontal angle, and the vertical movement is similarly performed. The vertical angle data is corrected based on the amount. From these corrected angle data, the angle information of the mask area is obtained as angle data for display at the center of the monitor screen.

  The corrected horizontal and vertical angle data and the size (viewing angle) data related to the mask area are stored in the built-in memory 13 together with the mask number as area data of the second arbitrary shape video object.

  An image captured by the network camera 15 after the storage is divided into a private area image and a monitoring area image not including the private area based on the area data stored in the built-in memory 13. Each divided image is individually encoded and transmitted to the PC 33. Therefore, the compressed image is decoded and the image obtained by decoding is synthesized.

  Here, the private image is a highly compressed image and is transmitted with a reduced frame rate. When a private area image encoded with a reduced frame rate and a monitoring area image encoded at the same frame rate as the network camera 15 are combined, images captured at different times are combined. In the meantime, when the shooting direction of the network camera 15 changes due to, for example, external vibration, the image may be doubled or missing at the boundary of the composite image.

Therefore, a boundary region having a predetermined width is arranged between the monitoring region image and the private region image, and the disturbance of the image due to the positional deviation of both images is absorbed by the boundary region.
FIG. 3 shows each area of the monitoring image.

In the figure, the monitor screen 341 includes a monitoring area image 341a, a private area image 341b, and a boundary area 341c.
Here, the boundary region is surrounded by a horizontal line and a vertical line. This is because the boundary area is defined by the area of the macroblock used at the time of encoding.

  That is, a macro block such as MPEG in which image data is encoded by discrete cosine transform and divided vertically and horizontally into 16 pixels is used. Therefore, when the width of the macro block is set as the width of the boundary area and the monitoring area image and the private area image are overlapped so as to fade in / out in the area of the width, the overlapping process is easy. The signal level at the scanning line 341b constituting the monitor screen 341 will be further described.

FIG. 4 shows the level characteristic of the image signal with respect to the horizontal position of the scanning line.
In the figure, monitoring area images are displayed on h0 to h1 and h4 to h9, and private area images are displayed on h2 to h3. h1 to h2 and h3 to h4 are boundary areas, and the levels of the monitoring area image and the private area image are changed in the boundary area.

  Here, the interval between h1 to h2 and h3 to h4 is 16 pixels, which is the same as the size of the macroblock. A plurality of macro blocks exist between h0 to h1 of the monitoring image. That is, the number of pixels in between is a positive multiple of 16. Similarly, the interval between h2 and h3 is an integral multiple of 16 pixels.

  When creating a composite image by adding a monitoring area image to a private area image, a composite image can be obtained by adding an image weighted at the level shown in the figure. Also, in order to obtain a composite image by superimposing a semi-transparent private area image on a monitoring image, h0 to h1 are made transparent in the private area image, the transparency is gradually lowered from h1 to h2, and the level of the private area image is increased. , H2 to h3 include a method of superimposing the monitoring area image as an opaque private area image.

  In the latter method, a composite image can be obtained by superimposing private area images having the above characteristics without processing a monitoring image transmitted every frame. The photographed private area image can be translucently processed in software without using dedicated hardware.

  FIG. 5 is a block diagram showing functions for executing coding by dividing an image and synthesis of an image obtained by decoding, and will be described with reference to FIG. The same functional parts as those of the elements shown in FIG.

  In the figure, the camera CPU 12a includes an area determination unit 121, an outer region frame unit 122, an inner region frame unit 123, an inner image extraction unit 124, an outer image extraction unit 125, an inner image encoding unit 126, an outer image encoding unit 127, A multiplex unit (MUX) 128 and a motion determination unit 129 are included. The PC 33a includes a demultiplexing unit (DEMUX) 331, a region frame unit 332, an inner image decoding unit 333, an outer image decoding unit 334, an inner region filter unit 335, an outer region filter unit 336, and an image composition unit (MIX) 337. Is done.

  First, when the controller 35 is operated to change the viewing angle of the network camera 15, information related to the direction of the camera is input from the camera platform drive unit 14 to the region determination unit 121. The area determination unit 121 detects whether there is an area that needs private protection in the image captured by the camera CPU 12 a based on information stored in the area designation unit 11.

  When it is detected that a private area image is captured, a private area frame is determined. The private area frame determines the position information of both the outer frame (the frame including the points h1 and h4) and the inner frame (the frame including the points h2 and h3) of the boundary region 341c.

  Note that when the width of the frame is determined to be one macroblock, either the inside or the outside, for example, the outside frame may be determined. When the frame for the screen is defined as the scanning line direction and the direction perpendicular to the scanning line, for example, the diagonal position of the outer frame can be designated by the position of the effective pixel. A method of specifying by the position of the macroblock may be used.

  The outer region frame unit 122 creates a position signal of the inner region frame with respect to the position of the screen imaged by the network camera 31 based on the inner frame information determined by the region determination unit 121. Similarly, the inner area frame unit 123 creates a position signal of the outer area frame.

  The inner image extraction unit 124 extracts an inner image surrounded by the outer region frame from the images captured by the network camera 31 based on the created position signal of the inner region frame. The outer image extraction unit 125 extracts an outer image from images taken by the network camera 31 excluding the inner image surrounded by the inner area frame.

  The encoding unit 126 encodes the inner image so as to obtain a low-rate encoded signal. The low-rate encoding is, for example, encoding for pixels thinned out by rough quantization or encoding as a low frame rate signal. The encoded image is input to the built-in memory 13 and temporarily stored therein.

  Then, the motion determination unit 129 analyzes, for example, whether there is an image difference between the image encoded one second ago and the currently encoded image. As a result, when it is detected that there is no movement amount caused by the movement of a person, it is determined that the encoded image has no problem in private protection. The image is supplied to the MUX 128.

  When a motion amount of a predetermined level or more is detected, there is a possibility that a person has been photographed in the image. Therefore, the internal image data encoded in the past instead of the internal image encoded this time is below the predetermined level. An image detected as the amount of motion is obtained from the built-in memory 13 and supplied to the MUX 128.

  Next, the outer image is encoded by the encoding unit 127 using standard compression encoding parameters. The private area frame information obtained from the area determination unit 121, the image of the private area encoded by the inner image encoding unit 126, and the monitoring image encoded by the outer image encoding unit 127 are multiplex units. 128 is time-division multiplexed. The data is transmitted to the PC 33a via the interface 17 and the network 32.

  The demultiplex unit 331 separates the private area frame information, the private area image, and the monitoring image from the input signal. In the area frame portion 332, an inner area frame signal and an outer area frame signal are generated based on the private area frame information. The inner image decoding unit 333 decodes the private area image, and the outer image decoding unit 334 decodes the monitoring image.

  The inner area filter 335 gives the level characteristics of the private area image shown in FIG. 4 to the decoded private area image. In the outer region filter unit 336, the level characteristic of the monitoring image shown in FIG. 4 is given to the decoded monitoring image. In the image composition unit 337, the filtered private area image and the monitoring image are added to generate an image for display on the monitor 34.

  The configuration and operation of the network camera system have been described in detail above. Here, the method of gradually switching images in the boundary region using the inner region filter unit 335 and the outer region filter unit 336 has been described.

  Similar image encoding and decoding are performed by the MPEG-4 system, and the private image is handled as the first video object and the monitoring image is handled as the second video object, thereby encoding, transmitting and decoding the same image. And generation of a composite image.

  In other words, the image signal having an arbitrary shape is composed of three pixel value signals, shape signals, and transparency signals. The private area image and the monitoring area image are expressed by the pixel value signal and the shape signal, and the transparency signal is used as signal processing of the guard area for absorbing the position error generated when the area image and the monitoring area image are superimposed.

  In the above description, the inner region filter unit 335 and the outer region filter unit 336 are described as being arranged after the inner image decoding unit 333 and the outer image decoding unit 334. However, after the inner image extraction unit 124 and the outer image extraction unit 125, It may be arranged.

The private area image may be an image that has been captured in the past and in which no motion is detected unless the shooting direction of the network camera 31 is changed. The inner image decoding unit 333 has an image memory (not shown) that stores the decoded image. The private area image may be transmitted as necessary, and the private area image in that case is output as a still image.
Further, when the code amount is further reduced, the private area image may not be transmitted, and a solid image may be generated and combined on the receiving side.

  The present invention can be used to realize a surveillance camera device that can perform private protection processing by replacing an image of a predetermined portion of an image to be captured with an image of the same viewing angle taken at a different time.

It is the figure which showed schematic structure of the network camera system based on the Example of this invention. FIG. 6 is a diagram illustrating a display screen for setting a mask area according to an embodiment of the present invention. It is the figure shown about the area | region of the monitoring image based on the Example of this invention. It is the figure which showed the level characteristic of the image signal with respect to the horizontal position of a scanning line based on the Example of this invention. It is the figure which showed the structure which concerns on the area division | segmentation and composition of an image based on the Example of this invention. It is a figure which shows the structure of the monitoring system using the conventional dome camera.

Explanation of symbols

1 Dome Camera 11 Area Designation Unit 12, 12a Camera CPU
13 Built-in memory 14 Head drive unit 15 Network camera 16 Encoding unit 17 Interface 31 Network camera 32 Network 33, 33a PC
34 monitor 34a numeric keypad 34b up / down / left / right button 34c zoom button 34d menu button 35 controller 35a numeric keypad 35b set button 35c joystick 35D WIDE button 35E TELE button 35f menu button 121 area determination section 122 inner area frame section 123 outer area frame section 124 inner image extraction Unit 125 outer image extraction unit 126 inner image encoding unit 127 outer image encoding unit 128 multiplex unit 129 motion determination unit 331 demultiplex unit 332 region frame unit 333 inner image decoding unit 334 outer image decoding unit 335 inner region filter unit 336 Outer region filter unit 337 Image composition unit

Claims (1)

When the captured image captured in the private area set in the capturing area captured by the surveillance camera moves within a predetermined time, it is replaced with the image captured when there is no movement instead of the captured image. A method of generating a monitoring image generated as an image,
A first step of setting the private area with a boundary frame having a predetermined boundary width within the imaging area of the surveillance camera;
The private area image specified by the boundary frame set in the first step is acquired from the captured image of the monitoring camera, and specified by the boundary frame set in the first step from the captured image of the monitoring camera. A second step of obtaining a private area removal image that is a non-private area
When the private area image acquired in the second step is compared with the private area past image acquired in the past, and it is detected that the position of the subject of the private area image has not changed between the past and the present The private area image is acquired as an image taken when there is no movement, or the subject position does not change when it is detected that the subject position of the private area image has changed between the past and the present A third step of acquiring the private region past image acquired as a captured image when there is no movement;
The image captured when there is no movement acquired in the third step and the private area removed image acquired in the second step are gradually switched within the boundary area frame set in the first step. And a fourth step of generating the private protection monitoring image.

Images