JP4281338B2 - Image detection apparatus and image detection method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image detection apparatus and an image detection method for detecting a predetermined object in an image, for example, an image detection apparatus and an image detection method suitable for detecting a human face from a video image.
[0002]
[Prior art]
Conventionally, a technique for detecting and recognizing a predetermined object such as a human face from an image such as a video image has been proposed (see, for example, Patent Document 1), and application to a monitoring system, a robot apparatus, etc. is considered. It has been. As a detection / recognition method in this field, a method using a template matching method such as a support vector machine (SVM) is known. Prior art document information related to the present invention includes the following.
[0003]
[Patent Document 1]
JP-A-2001-216515 [0004]
[Problems to be solved by the invention]
By the way, in the apparatus for performing face detection / recognition as described above, the amount of calculation required for the pattern recognition algorithm becomes enormous. Therefore, it is important and demanded to realize practically sufficient detection accuracy while reducing arithmetic processing. Conventionally, a window image of a predetermined size consisting of a reduced-scale image is created from a video image, it is roughly determined whether or not it is a face image, and the window image that is clearly not a face image is removed, thereby reducing the total amount of computation. Techniques for reducing the number have been devised. However, with this method alone, when detecting faces from continuous moving images, the redundancy in the time direction is not used, and the amount of computation cannot be reduced sufficiently.
[0005]
The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image detection apparatus and an image detection method that can promote reduction in the amount of calculation in detecting a predetermined object in an image.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problem, according to a first aspect of the present invention, there is provided an image detection device for detecting a position of a predetermined object in an image, wherein the predetermined object is detected in a plurality of consecutive frames. A non-search frame that does not search the target object at all, and a full search frame that searches the predetermined target object over all regions in the frame, and sets the full search frame between the non-search frames at a predetermined cycle. There is provided an image detecting apparatus comprising the detecting means provided in (1).
[0007]
In the present invention, a non-search frame that does not search for an object at all is set instead of searching for an object for all frames. Since the object is not searched at all in the non-search frame, the amount of calculation can be greatly reduced. By providing all search frames with a predetermined period between non-search frames, it is possible to reduce the amount of calculation while detecting the position of a predetermined object.
[0008]
Here, it is preferable that the detection means further includes a neighborhood search frame for searching around the vicinity of the position of the predetermined object after the frame in which the position of the predetermined object is detected.
[0009]
Since it is usually difficult for an object to change its position extremely instantaneously, after searching for an object, it is only necessary to search around the position of the object, thereby reducing the number of images to be searched. it can.
[0010]
At that time, the search range in the neighborhood search frame is determined and adjusted in accordance with the size of the predetermined object detected, the zoom amount and the moving angle of the image capturing means capturing the image, the motion vector, and the like. It is preferable to use these pieces of information, and the number of images to be searched can be reduced, and highly accurate detection is possible. Here, the movement angle corresponds to, for example, pan / tilt according to the present embodiment described later.
[0011]
The detection means may be configured to exclude an object that does not move over a predetermined frame from the detection target as a stationary object. As a result, erroneous detection can be eliminated and the amount of calculation can be reduced.
[0012]
Further, according to a second aspect of the present invention, there is provided an image detection method for detecting a predetermined object in an image, and a non-search frame that does not search for the predetermined object in a continuous frame. An image detection method comprising: setting a full search frame for searching the predetermined object over all regions in a frame, and providing the full search frame at a predetermined cycle between the non-search frames. Provided.
[0013]
In the present invention, a non-search frame that does not search for an object at all is set instead of searching for an object for all frames. Since the object is not searched at all in the non-search frame, the amount of calculation can be greatly reduced. By providing all search frames with a predetermined period between non-search frames, it is possible to reduce the amount of calculation while detecting the position of a predetermined object.
[0014]
Here, it is preferable that a neighborhood search frame for searching around the position of the predetermined object is provided after the frame in which the position of the predetermined object is detected.
[0015]
Since it is usually difficult for an object to change its position extremely instantaneously, after searching for an object, it is only necessary to search around the position of the object, thereby reducing the number of images to be searched. it can.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image detection apparatus and an image detection method according to preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0017]
First, the configuration of the image detection apparatus according to the present embodiment will be described with reference to FIG. Here, a case where the object to be detected is a human face will be described as an example. As shown in FIG. 1, the image detection apparatus according to the present embodiment mainly includes a CCD (Charge Coupled Device) camera 10 as an image input means, and a face detection unit 20 that detects a face by performing image processing. Have as part.
[0018]
The apparatus further includes an image compression / decompression unit 30, a TV (television) monitor 40, a microphone 50, an audio direction detection unit 60, an audio compression / decompression unit 70, a speaker 80, a multiplex / network interface for compressing and transmitting an image. It is also possible to configure each of the 90 combinations as necessary.
[0019]
The CCD camera 10 is an image input unit that captures a moving image from a video input device, and sends the captured moving image to the face detection unit 20. It is preferable that the direction of the CCD camera 10 can be freely changed by an electric PTZ device (pan / tilt / zoom device) or the like. In this case, for example, an area recognized as a face is positioned at the center of the image. It becomes easy to control.
[0020]
The face detection unit 20 stores the image signal captured by the CCD camera 10 in an internal memory (not shown) in units of frames, and detects a human face image from the captured video. In some cases, it may be configured to have a function of processing identification of a person in addition to that.
[0021]
The microphone 50 is preferably composed of a microphone array in which a plurality of microphones are arranged. In this case, the voice direction can be detected as will be described later, and the search range can be reduced. The multiplex / network interface 90 is connected to a network such as a telephone line.
[0022]
As an overall information flow in the configuration shown in FIG. 1, an image captured by the CCD camera 10 is input to the face detection unit 20 and face detection is performed. When the CCD camera 10 is equipped with a PTZ device, an instruction for PTZ control is issued from the face detection unit 20 to the CCD camera 10 according to the processing result of the face detection unit 20. The video data is subjected to image processing by the face detection unit 20, then sent to the image compression / expansion unit 30, compressed / expanded as necessary, and then sent to the TV monitor 40 and the multiplexing / network interface 90. Depending on the case, motion vector information is sent from the image compression / decompression unit 30 to the face detection unit 20.
[0023]
On the other hand, the direction of the voice collected by the microphone 50 is detected by the voice direction detection unit 60. The detected voice direction data is sent to the face detection unit 20. The voice is sent from the voice direction detection unit 60 to the voice compression / expansion unit 70, compressed / expanded as necessary, and then sent to the speaker 80 and the multiplexing / network interface 90.
[0024]
FIG. 2 is a functional block diagram for explaining the processing content of the face detection unit 20. As shown in FIG. 2, it can be divided into an input image scale conversion unit 202, a window cutout unit 204, a template matching unit 206, a preprocessing unit 208, a pattern identification unit 210, and an overlap determination unit 212. Hereinafter, the function of each part will be schematically described.
[0025]
The input image scale conversion unit 202 reads a frame image based on an image signal from the CCD camera 10 (FIG. 1) from an internal memory (not shown), and converts the frame image into a plurality of scale images having different reduction rates. For example, a frame image composed of 25344 (= 176 × 144) pixels is sequentially reduced by 0.8 times to obtain 5 stages (1 times, 0.8 times, 0.64 times, 0.51 times, It is conceivable to convert it into a scale image of 0.41 times.
[0026]
The window cutout unit 204 sequentially cuts out a rectangular area having a predetermined pixel amount from the first scale image among the plurality of scale images. Hereinafter, this cut-out area is referred to as a window image.
[0027]
Then, the window cutout unit 204 sends the top window image among the plurality of window images cut out from the first scale image to the template matching unit 206 at the subsequent stage.
[0028]
The template matching unit 206 determines whether or not the window image is a face image with respect to the top window image obtained from the window cutout unit 204. Here, in the template matching unit 206, for example, an average face image of about 100 persons can be used as a template to roughly match the window image.
[0029]
The window image determined to be a face image by the template matching unit 206 is sent to the subsequent preprocessing unit 208 as a score image, and the window image determined not to be a face image is sent to the subsequent overlap determination unit 212 as it is. The
[0030]
The pre-processing unit 208 removes an area corresponding to the background portion that is irrelevant to the human face image from the score image, and corrects shading, contrast, and the like due to illumination during shooting. Further, the preprocessing unit 208 vector-converts the score image and sends it to the pattern identification unit 210.
[0031]
Here, the pattern identification unit 210 determines whether or not face data exists for a score image obtained as a vector using a support vector machine. If face data exists, the image position, size, reduction ratio, etc. are listed and stored in the internal memory as list data.
[0032]
In addition, the pattern identification unit 210 notifies the window cutout unit 204 that face detection has been completed for the top window image. In response to this notification, the window cutout unit 204 sends it to the next window image template matching unit 206. The pattern identification unit 210 notifies the input image scale conversion unit 202 that face detection has been completed for the first scale image. In response to this notification, the input image scale conversion unit 202 sends the second scale image to the window cutout unit 204.
[0033]
The overlap determination unit 212 reads a plurality of list data stored in the internal memory, compares score images included in the list data, determines whether or not an overlapping portion is included, and based on the determination result. In each scale image, a single image area is finally collected from the score images without overlapping, and the image area is newly stored in the internal memory as face determination data. Store.
[0034]
If the template matching unit 206 determines that the image is not a face image, the overlap determination unit 212 does nothing and does not store the internal memory.
[0035]
In this way, a face image can be detected from the original frame image. The operation as described above requires a large amount of calculation. Therefore, the face detection unit 20 according to the present embodiment has various functions described below in addition to the above functions in order to reduce the calculation amount.
[0036]
(1) Frame skip search function When a human face included in a continuous moving image is recognized, it takes a very long calculation time to pattern-match all regions of the image for each frame. Therefore, a frame that searches for a face by pattern matching over the entire area in the frame (hereinafter referred to as a full search frame) and a frame that does not perform any face search (hereinafter referred to as a non-search frame) And are set.
[0037]
In addition, when the recognition target is a human face, the range in which the position of the human face moves between frames is limited by judging from the normal human moving speed etc. Range. Therefore, it is almost impossible for the human position to suddenly jump from one end of the screen to the other, and there are many cases where the face moves within a limited range on the screen.
[0038]
Therefore, in addition to the above-mentioned two types of frames, a frame that searches for a peripheral region with reference to a frame whose face location is specified in the previous image, that is, a frame that searches for a neighborhood (hereinafter, this frame is called a neighborhood search frame) ) Is set. In this way, the search is performed by providing three types of frames, the full search frame, the non-search frame, and the neighborhood search frame.
[0039]
As a search method, first, all search frames are provided at a predetermined cycle between a plurality of non-search frames among consecutive frames. Once the presence of a face is detected, only the neighborhood is defined as a search range, and a neighborhood search frame is provided.
[0040]
FIG. 3 shows a conceptual diagram when the above three types of frames are provided. In FIG. 3, for the sake of distinction, the entire search frame A is indicated by black, the non-search frame B is indicated by white, and the neighborhood search frame C is indicated by hatching. The horizontal direction indicates time, and each frame is shown in time series as shown in FIG.
[0041]
In other words, the full search frame A is provided in a large number of non-search frames B at a constant cycle, and when the presence of a face is detected in the head full search frame A, the neighborhood search frame C is provided thereafter.
[0042]
For example, when a moving image of 30 frames / second is input as in the NTSC system, only one frame is searched for every 30 frames, and the rest are non-search frames. In this case, the full search only needs to be performed once per second, and the amount of calculation can be reduced to 1/30 compared with the case of full search for each frame. Until a face is detected for the first time, processing is performed by providing all search frames at a constant period in this way. In order to reduce the amount of calculation processing associated with face detection and face recognition, it is preferable to provide a large number of non-search frames in a continuous moving image.
[0043]
FIG. 4 shows an example of the search range of the neighborhood search frame. In this example, it is assumed that the camera is stationary. FIG. 4 shows a range f1 in which the face is detected in the previous search frame, a range f2 in which the search is performed in the next neighborhood search frame, and a range f3 in which the search is not performed in the next neighborhood search frame. A range f3 indicates a range obtained by removing the range f2 from the entire image (a range indicated by the outermost frame in FIG. 4). A range f1 indicates a human face portion, and a range f2 is a range including the vicinity of the range f1.
[0044]
By providing the neighborhood search frame as described above, it is possible to reduce the number of the above-described image scaling processes and the pattern matching process. For example, conventionally, the above-described input image scale conversion unit 202 searches for a face over the entire region in a frame and creates a five-stage scaled image of 0.8 times. On the other hand, in this embodiment, it is possible to reduce the scaled image detected in the previous search frame in the neighborhood search frame to a total of three levels of scaled images, one step before and after that.
[0045]
In addition, when cutting out the window image used for template matching, it is possible to greatly reduce the amount of calculation by limiting the entire range of the normal scaled video to only the range near the previous detected coordinate. .
[0046]
By inserting, for example, one such neighborhood search frame in 5 frames, it is possible to smoothly follow human movements and reduce the frequency of all search frames to reduce the amount of calculation. become.
[0047]
Further, in the scaling process of the neighborhood search, by cutting out the scaled image so that the face is at the center of the scaled image, it is possible to reduce the probability that the face is placed at the boundary of the scaled image.
[0048]
(2) Limiting the search range according to the size of the target In the vicinity search, the search range is limited depending on the size of the target to be detected and recognized. There is a characteristic that the range of movement displayed on the screen differs between when the human face image is large and small when the human moves the face. Think about using it.
[0049]
For example, if the face image is large, the face position on the screen may change greatly between adjacent search frames even if the amount of movement of the face or camera is small. It is necessary to take it widely. On the other hand, when the face image is small, the position of the face on the screen does not change so much between adjacent search frames, so the search range may be relatively narrow. By combining this characteristic with the scaling algorithm, it is possible to reduce the scaled image to be searched.
[0050]
(3) Search range adjustment function in conjunction with the camera When the camera itself is panned to the left or right, the face image in the screen is expected to move according to the movement of the camera. can do. In determining the search area in the vicinity search, it is possible to further narrow the search range or improve the search accuracy by linking with the motion information of the CCD camera 10 (FIG. 1) that captures the image. .
[0051]
For example, taking a video conference system using a swing camera having an electric PTZ mechanism as an example, when the camera is panned to the right, the face image included in the video is expected to move to the left. In addition, the amount of motion at that time can be predicted from the size of the face image and the zoom amount (view angle), and the accuracy can be improved by using the motion prediction. The same applies not only to panning but also to tilting the camera.
[0052]
(4) Reduction function of search range by combination with voice direction detection A voice direction detection technique that uses two or three microphone arrays and detects the direction of the sound source from the time difference between the voices reaching the microphone array. Are known. By using such a known technique, the microphone 50 (FIG. 1) is constituted by a microphone array, and the voice direction detection unit 60 (FIG. 1) is provided with a voice direction detection circuit and used in combination. Direction can be detected.
[0053]
For example, in an application in which a camera is pointed at a speaker in a TV conference, the direction of sound is roughly detected by the voice direction detection circuit, and the detection result is transmitted to the face detection circuit, so that only the vicinity of the direction considered to be the sound source direction Can be searched for pattern matching. This reduces the pattern matching process.
[0054]
(5) Limiting function of search range by using motion vector A motion vector representing the direction and distance of the face as a target moves between the previous frame and the current frame by processing in the image compression / decompression unit 30. can get. By returning the motion vector, the search range can be limited without using information such as the zoom amount and movement angle of the camera, and the face can be detected.
[0055]
(6) Exclusion function of stationary object When a poster including a person's photograph is pasted on the wall and the person's photograph is reflected in the screen, including the poster, the normal pattern The face detection method based on matching recognizes the person as a human being, which may hinder the application. In addition, there is a case where a face detection algorithm erroneously detects when there is a pattern having a feature similar to a human face and it is captured in an image.
[0056]
The original detection object is a real human face, and the above is different from the detection object. In order to prevent such a false detection, a characteristic that “living humans are not usually still” is used. If a face image of the same size is detected at the exact same location on the screen every time the camera orientation and magnification are fixed, it is determined that it is a stationary object and excluded from the detection target. Such an erroneous detection can be eliminated by adding an algorithm. For example, if a face image is detected at the same pixel position with the same scaling magnification every time in all 10 consecutive search frames, it is determined that this is a stationary object and is not detected as a face.
[0057]
As described above, according to the present embodiment, it is possible to greatly reduce the amount of calculation processing when detecting a human face from an image. As a result, it is possible to construct a system using an inexpensive device and to reduce power consumption due to a reduction in the load on a CPU (Central Processing Unit). In addition, the detection error can be reduced and the detection accuracy can be improved while the calculation amount is low.
[0058]
It goes without saying that the image detection apparatus and the image detection method of the present embodiment can be applied to a robot, a monitoring system, and the like in addition to the TV conference system, and are not limited to the detection apparatus, and can also be applied to a recognition apparatus and the like. Yes. Further, in the above description, the case where the object to be detected is a human face has been described as an example. However, the present invention is not necessarily limited to this, and the same application can be applied to a search system in which other objects are detected and recognized. Is possible. For example, the present invention may be applied to a parking lot management system using an object to be detected and recognized as a vehicle.
[0059]
In the above description, an example in which a frame unit is set such as a non-search frame, a full search frame, and a neighborhood search frame has been described. However, a non-search field, a full search field, and a neighborhood search are performed by replacing a frame with a field. Of course, it is conceivable to set the field unit as in the field.
[0060]
As mentioned above, although preferred embodiment concerning this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this example. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea described in the claims. It is understood that it belongs to.
[0061]
【The invention's effect】
As described above in detail, according to the present invention, it is possible to provide an image detection device and an image that can promote reduction in the amount of calculation in detecting a predetermined object in the image.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an image detection apparatus according to an embodiment of the present invention.
FIG. 2 is a functional block diagram for explaining processing contents of a face detection unit;
FIG. 3 is a conceptual diagram when various frames are set.
FIG. 4 is a diagram illustrating an example of a search range of a neighborhood search frame.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 CCD camera 20 Face detection part 30 Image compression / decompression part 50 Microphone 60 Audio | voice direction detection part 202 Input image scale conversion part 204 Window extraction part 206 Template matching part 208 Pre-processing part 210 Pattern identification part 212 Overlapping judgment part A Full search Frame B Non-search frame C Neighborhood search frame

Claims (12)

  Among consecutive frame images, a non-search frame that does not search for a predetermined object in the frame image at all and a non-search frame are provided at a predetermined cycle, and all the predetermined objects are included in the frame image. And set all search frames to search over the region
  Detecting the predetermined object in the full search frame;
  After the frame in which the predetermined object is detected in the entire search frame, the frame image is converted into a plurality of scale images of a plurality of stages having different reduction ratios, and the predetermined object in the plurality of scale images is converted. A neighborhood search frame for searching around the vicinity of the position of the object is set, and the predetermined object in the scale image is detected in the neighborhood search frame;
  When the predetermined object is detected in the neighborhood search frame, the scale object is converted into a plurality of scale images having a smaller number of stages than in the previous conversion, including the stage of the scale image in which the predetermined object is detected. An image detecting apparatus comprising detecting means for detecting the predetermined object in the scale image in the next neighborhood search frame. It said detecting means, the search range in the vicinity of the search frame, that determine in accordance with the magnitude of the detected predetermined object, an image detecting apparatus according to claim 1. Said detecting means, said the search range in the vicinity search frame, Ru make adjustments based on the zoom amount and the moving angle of the photographing means which photographs the image, the image sensing apparatus according to claim 1. It said detecting means, the search range in the vicinity of the search frame, that determine the use of the motion vector, the image detection device according to claim 1. It said detecting means, to exclude entirely immobile object over a predetermined frame from the detection target image detecting apparatus according to claim 1.   Voice direction detecting means for detecting the direction of the sound source;
  The image detection apparatus according to claim 1, wherein the detection unit determines only the vicinity in the direction of the detected sound source as a search range in the vicinity search frame.   Among consecutive frame images, a non-search frame that does not search for a predetermined object in the frame image at all and a non-search frame are provided at a predetermined cycle, and all the predetermined objects are included in the frame image. And set all search frames to search over the region
  Detecting the predetermined object in the full search frame;
  After the frame in which the predetermined object is detected in the entire search frame, the frame image is converted into a plurality of scale images of a plurality of stages having different reduction ratios, and the predetermined object in the plurality of scale images is converted. A neighborhood search frame for searching around the vicinity of the position of the object is set, and the predetermined object in the scale image is detected in the neighborhood search frame;
  When the predetermined object is detected in the neighborhood search frame, the scale object is converted into a plurality of scale images having a smaller number of stages than in the previous conversion, including the stage of the scale image in which the predetermined object is detected. An image detection method for detecting the predetermined object in the scale image in a next neighborhood search frame. Wherein the search range in the vicinity search frame, that determine in accordance with the magnitude of the detected predetermined object, an image detecting method according to claim 7. Wherein the search range in the vicinity search frame, the image Ru make adjustments based on the zoom amount and the moving angle of the imaging means are taken, the image detecting method according to claim 7. The search range in the vicinity of the search frame, that determine the use of the motion vector, the image detecting method according to claim 7. To exclude entirely immobile object over a predetermined frame from the detection target image detecting method according to claim 7.   Detect the direction of the sound source,
  The image detection apparatus according to claim 7, wherein only the vicinity of the detected direction of the sound source is determined as a search range in the vicinity search frame.

Images