JP2008301162A - Photography device, and photographing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photography device which photographs a person who moves in a distance direction while stably tracking him/her. <P>SOLUTION: When a face is extracted, processing by a face extraction circuit 75 stops. A zoom lens 41 is controlled on the basis of size of the extracted face and the face is kept to fixed size in an image. In addition, the attitude of a monitoring camera 21 is controlled on the basis of the motion vector of the face, the face is tracked in the image and kept to a fixed position. Photography while tracking the face is sequentially performed on the basis of motion vectors obtained by each frame. When the motion vector of the face becomes no longer detected, the processing by the face extraction circuit 75 is started again. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a photographing apparatus and a photographing method for photographing while tracking a moving person.

  In recent years, surveillance cameras are often installed in stores, other facilities, and public places. The surveillance camera is an imaging device that captures a moving image of the surroundings, and can record an image captured by the moving image or monitor it with a monitor.

  As such an imaging apparatus, a fixed type that captures a certain direction (see, for example, Patent Document 1), and an omnidirectional type that rotates in the up, down, left, and right directions to capture a variable direction (for example, see Patent Document 2) There is a variable focal length type (for example, refer to Patent Document 3) that shoots by changing the focal length periodically.

In addition, recently, there is a photographing apparatus that enables photographing while tracking a moving person (see, for example, Patent Documents 4 and 5). For tracking a person, a motion vector detected from a moving image is used. The motion vector is detected after comparing the images between frames and identifying corresponding pixels as the same subject.
JP 2005-239313 A JP 2001-094838 A JP 2006-262225 A JP 2001-268425 A Japanese Patent Laid-Open No. 06-339056

  However, in the photographing apparatuses described in Patent Documents 4 and 5, there is a possibility that a motion vector cannot be detected and tracked with respect to a person who moves in the perspective direction and scales in the image.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging apparatus that captures an image while stably tracking a person who moves in the perspective direction.

  In order to achieve the above object, an imaging apparatus of the present invention is an imaging apparatus that captures images while tracking a moving person, and captures a moving image to acquire an image, and extracts a person's face from the image. Face extraction means for intermittently repeating the process, motion detection means for detecting a motion vector indicating the amount and direction of movement of the face extracted by the face extraction means, and posture control for controlling the posture of the photographing means based on the motion vector Means.

  It is preferable to include a focal length adjusting unit that changes the focal length and a drive control unit that controls the driving of the focal length adjusting unit based on the size of the face extracted by the face extracting unit.

  Further, when no motion vector is detected, it is preferable that the face extraction unit performs face extraction again.

  The photographing method of the present invention is a photographing method of photographing while tracking a moving person, and intermittently repeats extracting a person's face from an image acquired by moving image photographing and moves the extracted face. A motion vector indicating the amount and direction is detected, and the shooting direction is controlled based on the motion vector.

  Note that it is preferable to control the focal length based on the size of the extracted face.

  It is also preferable to perform face extraction again when no motion vector is detected.

  According to the photographing apparatus and the photographing method of the present invention, the extraction of a person's face from an image acquired by moving image photographing is repeated intermittently, the motion vector of the extracted face is detected, and the motion vector is extracted. Since the shooting direction is controlled based on this, it is possible to take a picture while stably tracking the person even when the person moves in the perspective direction.

  In addition, since the focal length is controlled based on the size of the extracted subject, it is possible to track a person moving in the perspective direction while maintaining a constant size in the image.

  Hereinafter, the surveillance camera (imaging device) 21 of the present invention will be described with reference to the drawings. As shown in FIG. 1, a transparent protective plate 23 that protects the photographing lens 51 (see FIG. 2) is provided on the front surface of the housing 22 of the monitoring camera 21. An attitude control mechanism 25 provided at the tip of the mounting arm 24 is connected to the lower surface of the housing 22. The attitude control mechanism 25 controls the attitude of the monitoring camera 21 by rotating the housing 22 in the vertical direction (U, D) and the horizontal direction (L, R). Note that such a posture control mechanism 25 is a well-known technique and will not be described in detail.

  As shown in FIG. 2, the monitoring camera 21 is provided with a zoom lens 41, a diaphragm 42, a focus lens 43, and a CCD image sensor 44. The zoom lens (focal length adjusting means) 41 changes magnification by driving a magnification changing lens motor 45. The aperture 42 changes its aperture by driving the iris motor 46. The focus lens 43 adjusts the focus by driving the focusing lens motor 47. Driving of these motors 45 to 47 is controlled via a motor driver 49 connected to the CPU 48. In addition to the motors 45 to 47, the motor driver 49 is connected to a posture control motor 50 that drives the posture control mechanism 25. The attitude control mechanism 25 controls the attitude of the monitoring camera 21 by driving the attitude control motor 50. The zoom lens 41, the aperture 42, and the focus lens 43 constitute a photographing lens 51.

  A timing generator (TG) 53 controlled by the CPU 48 is connected to the CCD image sensor 44. The CCD image sensor (photographing means) 44 is driven by a timing signal (clock pulse) input from the timing generator 53 to form an image of subject light and image it.

  An imaging signal obtained from the CCD image sensor 44 is input to a CDS (correlated double sampling circuit) 54 and an AMP (amplifier) 55. The CDS 54 outputs R, G, B image data that accurately corresponds to the amount of charge stored in each cell of the CCD image sensor 44, and the AMP 55 amplifies the image data. The amplified image data is converted into a digital signal by the A / D converter 56.

  The image input controller 57 is connected to the CPU 48 via the data bus 58, and controls the CCD image sensor 44, the CDS 54, the AMP 55, and the A / D converter 56 in accordance with a control command from the CPU 48. The image data output from the A / D converter 56 is output at predetermined intervals and temporarily stored in the SDRAM 59. The SDRAM 59 is a working memory.

The image signal processing circuit 60 reads the image data from the SDRAM 59, performs various image processing such as gradation conversion, white balance (WB) correction, and γ correction, and stores the image data in the SDRAM 59 again. The YC conversion processing circuit 61 reads the image data subjected to various processes by the image signal processing circuit 60 from the SDRAM 59 and converts it into a luminance signal Y and color difference signals C _r and C _b .

  The VRAM 62 is a memory for outputting a through image to a monitor (not shown) of an external device connected via the communication I / F 63, and image data that has passed through the image signal processing circuit 60 and the YC conversion processing circuit 61 is stored in the VRAM 62. Stored. The VRAM 62 has a memory for two frames so that image data can be written and read in parallel. The image data stored in the VRAM 62 is converted into an analog composite signal by a monitor driver (not shown) and displayed as a through image on the monitor.

  A power supply control circuit 64 is connected to the communication I / F 63. The power supply control circuit 64 includes, for example, a filter for removing power supply noise and a limiter for preventing overcurrent. The power supply control circuit 64 converts the bus power supplied from the external device via the communication I / F 63 to DC / DC. This is supplied to each part of the monitoring camera 21 through the converter 65. In addition, after various image processing is performed, the image data stored in the VRAM 62 is output to the communication I / F 63 and transmitted to an external device.

  The compression / decompression processing circuit 66 performs image compression on the image data YC converted by the YC conversion processing circuit 61 in a predetermined compression format (for example, MPEG format). The compressed image data is recorded on a recording medium (not shown) of an external device connected via the communication I / F 63. Note that the compression / decompression processing circuit 66 includes a motion detector 67 that detects a motion vector and a motion compensator 68 that performs motion compensation processing using the motion vector.

  The motion detector (motion detection means) 67 detects the motion vector by comparing the image of the Nth frame with the image of the immediately preceding (N−1) frame by pattern matching. The motion vector represents the spatial distance and direction of a macroblock (for example, 8 × 8 pixels) having the same pattern between frames.

For example, regarding the (N-1) th frame image shown in FIG. 3A and the Nth frame image shown in FIG. 3B, the nth frame from the upper end of the image frame and the mth frame from the left end. If the pixel is _{a n-1, m-1} , a 0,0 ~a 0,7 in the n _{frame, a 1,0 ~a 1,7, a 2,0} ~a 2,7, a 3, _{0 to} a _3,7 , a _{4,0 to} a _4,7 , a _{5,0 to} a _5,7 , a _{6,0 to} a _6,7 , a _{7,0 to} a _7,7 The macroblocks 101 are a _{3,4 to} a _3,11 , a _{4,4 to} a _4,11 , a _{5,4 to} a _5,11 , a _6,4 to, in the (N-1) th frame. _{a 6,11, a 7,4 ~a 7,11,} a 8,4 ~a 8,11, a 9,4 ~a 9,11, composed of pixels of _{a 10, 4 ~a 10, 11} Black blocks 102 have the same pattern as the corresponding as the same subject. That is, the motion vector 103 of the macroblock 101 in the Nth frame represents the distance and direction for four pixels in the left direction and three pixels in the upward direction.

  The motion compensator 68 generates an image of the (N−1) th frame (see FIG. 3A) and a motion vector 103 of the Nth frame detected by the motion detector 67 (see FIG. 3B). In response, a subject moving spatially between frames is followed, and an image of the Nth frame is predicted to obtain a difference image between frames.

  Here, a case where a subject moving in a direction that does not cause a change in the shooting distance is shot will be described as an example. It is assumed that the image is divided into 4 × 4 macroblocks BLK0 to BLK15. Further, it is assumed that the monitoring camera 21 is shooting in a certain direction. The human face 104 shown in the macroblock BLK5 of the (N-1) th frame shown in FIG. 4A is followed by the motion vector 105 (see FIG. 4B) of the macroblock BLK5, and FIG. The image of the Nth frame shown in Fig. 6 is predicted, and a difference image between frames is obtained. Thus, high compression efficiency can be obtained by subtracting data that is common between frames and obtaining the difference image.

  On the other hand, an example will be described in which a subject moving in a direction that causes a change in the shooting distance is shot. It is assumed that the image is divided into 4 × 4 macroblocks BLK0 to BLK15. Further, it is assumed that the monitoring camera 21 is shooting in a certain direction. The human face 104 shown in the macroblock BLK5 of the (N-1) th frame shown in FIG. 5A is enlarged and shown in the macroblocks BLK6 and BLK10 in the Nth frame shown in FIG. 5B. In other words, the motion detector 67 cannot recognize the corresponding macroblock as the human face 104 due to the scaling of the subject and cannot detect the motion vector in the comparison of the images by pattern matching. Since the motion vector cannot be detected, the motion compensator 68 cannot predict the image of the Nth frame and cannot obtain a difference image between the frames. Note that motion vectors that cannot be detected are interpolated by processing by a face extraction circuit 75 described later in detail.

  In addition to the motor driver 49 and the like, an EEPROM 69 is connected to the CPU 48. In the EEPROM 69, various control programs and setting information are recorded. The CPU 48 reads these pieces of information from the EEPROM 69 to the SDRAM 59 and executes various processes.

  The data bus 58 includes an AF detection circuit 72 that detects whether or not the focus position of the zoom lens 41 is appropriate, and whether or not the exposure conditions such as the imaging sensitivity of the CCD image sensor 44 and the aperture value of the aperture 42 are appropriate for imaging. Are connected to an AE detection circuit 73 for detecting whether or not white balance correction is appropriate for photographing.

  Each of the detection circuits 72 to 74 sequentially transmits detection results to the CPU 48 via the data bus 58. The CPU 48 individually controls operations of the zoom lens 41, the diaphragm 42, the focus lens 43, the CCD image sensor 44, and the like based on the detection results transmitted from the detection circuits 72 to 74.

  The AF detection circuit 72 performs focus control by a contrast detection method. First, a shooting distance to a subject such as a person's face is calculated based on one or a plurality of focus evaluation areas (AF areas) set in advance to evaluate the in-focus state of the image. Determine the focal position.

  The AE / AWB detection circuits 73 and 74 calculate appropriate exposure values and white balance correction amounts based on the luminance information of the image data stored in the VRAM 62 at predetermined intervals, and send these pieces of information to the CPU 48. The CPU 48 continuously controls the aperture 42 and image processing based on the obtained information.

  The data bus 58 is connected to a face extraction circuit (face extraction means) 75 that extracts a human face from an image. The face extraction circuit 75 reads out image data displayed on a monitor (not shown) of an external device as a through image from the VRAM 62, and extracts a person's face by extracting both eyes of the person.

  Specifically, when extracting both eyes of a person, image data is divided into a grid (for example, 16 × 16), and the skin level is determined from the signal levels of the R, G, and B color signals included in each divided region. A divided region including many estimated skin color pixels is extracted. Then, an area having white pixels of the eye part and black pixels estimated to be pupils is selected from the extracted area, and the positions of both eyes of the person on the image are obtained from the areas, and the positions of both eyes are determined. The point is defined as the face position with the representative point. The face extraction circuit 75 extracts an area including a lot of surrounding skin color pixels determined as the face position as a face area.

  As shown in FIGS. 5A and 5B, when a motion vector cannot be detected for the human face 104, the face extraction circuit 75 extracts the human face 104 so that the motion vector can be detected. Can be interpolated. By limiting the shooting target to a person's face, it is possible to follow even if the person's face moves in a direction in which the shooting distance varies. Further, the face extraction circuit 75 can be driven at predetermined frame intervals to extract a person's face, thereby stably following the face.

  The motor driver 49 receives a command from the CPU 48 based on the size of the person's face extracted by the face extraction circuit 75 and drives the zoom lens motor 45. The zoom lens 41 is zoomed by driving the zoom lens motor 45, and the face of the person who moves in the perspective direction can be kept constant in the image. Further, the motor driver 49 receives a command from the CPU 48 based on the motion vector of the person's face extracted by the face extraction circuit 75 and drives the posture control motor 50. By controlling the posture of the monitoring camera 21 so that the shooting range moves by the motion vector, the face of the person can be kept at a certain position in the image. In this way, the monitoring camera 21 can take an image while tracking the face of a person even when moving in the perspective direction.

  Hereinafter, the monitoring camera 21 (see FIG. 1) of the above embodiment will be described with reference to FIG. When shooting is started, the face extraction circuit 75 is driven to start extracting a person's face. When a person's face is extracted, the processing by the face extraction circuit 75 stops. Based on the size of the extracted face, zooming of the zoom lens 41 is controlled, and the face of the person is kept constant in the image. Further, the posture of the monitoring camera 21 is controlled based on the motion vector of the person's face, and the person's face is tracked in the image and kept at a certain position (for example, the center of the image). Shooting while tracking a person's face is sequentially performed based on a motion vector obtained for each frame. Note that when the motion vector of the person's face is no longer detected, the processing by the face extraction circuit 75 is started again.

  When processing is performed for a predetermined number of frames (for example, 5 frames), processing by the face extraction circuit 75 is started along with shooting while tracking the face of a person. When the person's face is extracted again, zooming of the zoom lens 41 is controlled based on the newly extracted face size, and the person's face is maintained at a constant size in the image. Then, the posture of the monitoring camera 21 is controlled based on the newly extracted face motion vector, and the face of the person is kept at a fixed position in the image. The above process is repeated until shooting is completed. In this way, the surveillance camera 21 can shoot while tracking the face of the person even when the face of the person moves in a direction that causes a change in the shooting distance.

  In the above-described embodiment, the case where images between frames are compared by pattern matching has been described as an example. However, it is only necessary that a motion vector can be detected, and another method may be used.

  Further, in the above embodiment, the face extraction by the face extraction circuit 75 has been described by taking the case where the face area is extracted by extracting the skin color. However, it is only necessary that the face area can be extracted, for example, pattern recognition using a template. The face area may be extracted by

  In the above embodiment, the case where a motion vector in units of macroblocks is detected has been described as an example. However, the present invention is not limited to this, and a motion vector in units of pixels may be detected.

It is an external appearance perspective view of a surveillance camera. It is a block diagram which shows the electrical structure of a surveillance camera. It is a figure for demonstrating the detection method of a motion vector, (A) shows the image of the (N-1) th frame, (B) shows the image of the Nth frame, respectively. It is a figure which shows the example in which a motion vector is detected, (A) shows the image of the (N-1) th frame, (B) shows the image of the Nth frame, respectively. It is a figure which shows the example in which a motion vector is not detected, (A) shows the image of the (N-1) th frame, (B) shows the image of the Nth frame, respectively. It is a flowchart which shows the imaging | photography process sequence of a surveillance camera.

Explanation of symbols

21 Surveillance camera (photographing device)
25 Attitude control mechanism (Attitude control means)
41 Zoom lens (focal length adjusting means)
44 CCD image sensor (photographing means)
45 Zoom lens motor (drive control means)
67 Motion detector (motion detection means)
75 Face extraction circuit (face extraction means)

Claims (6)

In an imaging device that shoots while tracking a moving person,
Shooting means for capturing videos and capturing images;
Face extraction means for intermittently repeating the process of extracting a human face from the image;
Motion detection means for detecting a motion vector indicating the amount and direction of movement of the face extracted by the face extraction means;
An imaging apparatus comprising: an attitude control unit that controls an attitude of the imaging unit based on the motion vector. A focal length adjusting means for changing the focal length;
The photographing apparatus according to claim 1, further comprising: a drive control unit that controls driving of the focal length adjustment unit based on a face size extracted by the face extraction unit.   3. The photographing apparatus according to claim 1, wherein when the motion vector is no longer detected, the face extracting unit performs face extraction again. In the shooting method of shooting while tracking a moving person,
Extracting a person's face from an image obtained by moving image shooting is repeated intermittently, and a motion vector indicating the amount and direction of movement of the extracted face is detected, and the shooting direction is controlled based on the motion vector. An imaging method characterized by the above.   5. The photographing method according to claim 4, wherein the focal length is controlled based on the size of the extracted face.   6. The photographing method according to claim 4, wherein the face is extracted again when the motion vector is no longer detected.

Images