JP2005149143A - Object detecting device and method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform detection without deteriorating a processing speed or detecting precision even when any inconvenience is caused in a photographic environment in detecting an object. <P>SOLUTION: A human body detecting device 1 is provided with an image inputting part 101 which inputs images FG obtained after an object is photographed in every predetermined time, plane generating parts 301 to 305 which detect features of a person projected on the image FG by using methods different from each other, a featured value arithmetic control part 106 which controls the plane generating parts 301 to 305 so that the features of the person can be detected by the different plane generating parts 301 to 305 for the images FG adjacent to each other on time series, and a detection processing part 105 which detects the person projected on the image FG on the basis of the features detected from the image FG by any of those plane generating parts 301 to 305 and features detected from the images FG other than the image FG by the other plane generating parts 301 to 305. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an object detection apparatus or an object detection method for detecting a target object from an image.

  Conventionally, human body detection methods for the purpose of detecting pedestrians and intruders, personal authentication, or framing have been proposed.

  As a human body detection method, there is a known method for detecting a pedestrian by installing a camera in a place where a pedestrian passes and analyzing an image captured by the camera to detect an image area of the pedestrian. It has been. As a method for detecting an image area of a pedestrian, for example, a method using a background difference, a method using a motion difference (time difference), a method for obtaining an optical flow, a method for detecting a skin color area, and detecting an elliptical shape of a head A method or a method for detecting a body part such as an eye or a nose has been proposed.

  In the above detection method, there are cases where a pedestrian cannot be detected well when there is the following inconvenience.

  For example, if the background image is close to a color such as skin or if an error occurs in the background reference image due to a change in lighting conditions, the background difference cannot be obtained well, so the method using the background difference can detect pedestrians well. There are things that cannot be done. When the moving speed of the pedestrian is slow or when the pedestrian is stationary, the difference between the two images is unlikely to appear. Therefore, the method using the motion difference (time difference) and the method for obtaining the optical flow are good for the pedestrian. Detection may not be possible.

  If the background area contains a lot of skin color, it will be easy to misrecognize the background area as a human body area.If the pedestrian is looking back, the skin color area will be smaller. May not be able to detect a person well. If a pedestrian wears a hat, glasses, sunglasses, or mask, these parts will be hidden, so a method to detect the elliptical shape of the head and body parts such as eyes and nose are detected. This method may not be able to detect pedestrians well.

  As a method for solving these problems, a method as described in Patent Document 1 has been proposed. According to this method, a plurality of subjects having different characteristics are extracted, and the subjects extracted by each subject extraction are combined. If the synthesis result does not satisfy a predetermined determination criterion, auxiliary subjects with different characteristics are further extracted, and a main subject is extracted using the synthesis result and the extraction result of the auxiliary subject extraction.

However, in such a method, it must always be determined whether or not the synthesis result satisfies a predetermined criterion. Further, if the setting of the first plurality of subject extraction is not appropriate, auxiliary subject extraction that is an additional process must always be performed. Therefore, the number of processes executed simultaneously (that is, for one frame) increases, and the processing speed becomes very slow.
JP 11-316845 A

  In view of the above-described problems, the present invention enables detection to be performed while suppressing a decrease in processing speed and detection accuracy even when there is a problem in a shooting environment or the like when detecting a target. The purpose is to do.

  An object detection apparatus according to the present invention is an object detection apparatus that detects a target object from an image, and is different from an image input unit that inputs captured images obtained at predetermined time intervals. And a plurality of feature detection means for detecting features of the photographed image and the feature detection means so that the feature detection means is detected by the feature detection means different for the photographed images adjacent to each other in time series. A control means for controlling the object, and the object shown in the photographed image other than the photographed image by the feature detection means other than the feature detection means and the feature detected by the feature detection means. And an object detection means for detecting based on the feature detected from the photographed image.

  Alternatively, a plurality of probability calculation means for obtaining a probability of the object appearing in the photographed image for each region dividing the pixel surface of the photographed image using different methods, and adjacent to each other in time series Control means for controlling each of the certainty calculation means so that the certainty is calculated by the different certainty calculation means for each of the matching photographed images, and the object appearing in the photographed image is any one of the certainty Object detection means for detecting based on the likelihood obtained from the photographed image other than the photographed image by the probability computed means other than the photographed image other than the photographed image by the likelihood computed from the photographed image by the likelihood calculating means; It is characterized by having.

  The probability is represented by a numerical value, and the detection means adds the numerical value representing the certainty for each region of the photographed image between the regions at the same position, and examines the peak of the added value. The object may be detected by.

  When the object is a person, as the probability calculation means, means for determining the likelihood of each region by performing template matching of a semi-elliptical template to the photographed image, Means for determining the likelihood of the region by detecting the likelihood of skin color in the region, means for determining the likelihood of the region by detecting the likelihood of hair color in the region of the captured image, or It is desirable to provide means for obtaining the certainty of each region by performing template matching of a shoulder-shaped template on a photographed image.

  The certainty may be expressed by the proximity of the object to a predetermined position. In this case, the numerical value indicating the probability increases as the region considered to be closer to the predetermined position of the object among the regions.

  According to the present invention, even if an inconvenience or problem occurs in the shooting environment when detecting a target, the detection can be performed without significantly reducing the processing speed and detection accuracy.

  FIG. 1 is a diagram illustrating an example of the overall configuration of the monitoring system 100, FIG. 2 is a diagram illustrating an example of the position and orientation of the video camera 2, and an imaging situation, and FIG. 3 is an example of a hardware configuration of the human body detection device 1. FIG. 4 is a diagram illustrating an example of a functional configuration of the human body detection device 1.

  As shown in FIG. 1, the monitoring system 100 includes a human body detection device 1, a video camera 2, a communication line 3 and the like according to the present invention. The human body detection device 1 and the video camera 2 are connected to each other via a communication line 3. As the communication line 3, a LAN, a public line, a dedicated line, the Internet, or the like is used.

  The video camera 2 includes an image sensor such as a CCD, an optical system, an interface for transmitting / receiving data to / from an external device, a control circuit, and the like, and detects a human body using an image obtained by photographing as image data 70. Transmit to device 1.

  As shown in FIG. 2, the video camera 2 is installed on the ceiling of a place where people pass, such as a passage or entrance of a facility such as a store, an underground mall, a building, or an event venue. Hereinafter, a case where the video camera 2 is installed in a passage of a facility and used for monitoring the state of the passage will be described as an example. The video camera 2 has a horizontal angle of view of about 60 degrees, a distance from a detection target (subject), that is, a shooting distance of about 3 to 5 m, and an output image resolution of 640 × 480 pixels (so-called VGA). It shall be. The width of the passage shall be about 1 to 1.5 m.

  As shown in FIG. 3, the human body detection device 1 includes a CPU 1a, a RAM 1b, a ROM 1c, a magnetic storage device (hard disk) 1d, a communication interface 1e, a display device 1f, and an input device 1g such as a mouse or a keyboard.

  The magnetic storage device 1d includes an image input unit 101, a preprocessing unit 102, a feature amount calculation unit 103, a head centrality plane generation unit 104, a detection processing unit 105, a feature amount calculation control unit 106, as shown in FIG. Programs and data for realizing the functions of the head image display unit 107, the head image storage unit 108, the feature plane storage unit 1M1, the previous frame storage unit 1M2, the template storage unit 1M3, the result plane storage unit 1M4, etc. Is installed. These programs and data are loaded into the RAM 1b as necessary, and the programs are executed by the CPU 1a.

  The human body detection device 1 is installed in a management room of a facility, and is used for monitoring the state of a passage while a guard is in the management room. Further, it is possible to detect a passerby's head in an image taken by the video camera 2 and to enlarge the head or store an image (video) of the head. As the human body detection device 1, a workstation or a personal computer is used.

  Hereinafter, processing contents of each part of the human body detection device 1 shown in FIG. 4 when detecting the center position of the pedestrian's head (for example, the tip of the nose) from the image taken by the video camera 2 will be described. .

[Input and preprocessing of detection target image]
FIG. 5 is a diagram illustrating an example of an image FG photographed by the video camera 2, FIG. 6 is a flowchart illustrating an example of the flow of color space conversion processing, and FIG. 7 is a diagram illustrating an example of a method for generating a cutout image GC. FIG. 8 is a diagram for explaining an example of a method for generating a cut-out reduced image GS, FIG. 9 is a flowchart for explaining an example of the flow of image reduction processing, and FIG. 10 is a diagram for showing an example of a method for generating a time difference plane ST. FIG. 11 is a flowchart illustrating an example of the flow of the time difference detection process, FIG. 12 is a flowchart illustrating an example of a method of generating the spatial difference plane SS, and FIG. 13 is a flowchart illustrating an example of the flow of the spatial difference detection process. FIG. 15 is a diagram illustrating an example of a method for generating a logical product plane AN, FIG. 15 is a flowchart illustrating an example of a flow of logical product image generation processing, and FIG. 16 is a diagram illustrating an example of a logical product plane AN.

  The image input unit 101 in FIG. 4 performs reception processing of the image data 70 transmitted from the video camera 2. Thereby, an image FG (video) as shown in FIG. 5 having the number of frames (for example, 30 frames per second) according to the shooting speed of the video camera 2 is obtained.

  The preprocessing unit 102 includes a color space conversion unit 201, an image reduction processing unit 202, a time difference calculation unit 203, a space difference calculation unit 204, a logical product image generation unit 205, and the like, and a pedestrian reflected in the image FG Processing for preparing an image necessary for processing for obtaining the center position of the head of the HMN is performed.

  When the image FG input by the image input unit 101 is an RGB color space image, the color space conversion unit 201 performs a process of converting the image data into YUV color space data. Such processing is performed according to the procedure shown in FIGS.

  An image area in which the head of the pedestrian HMN is likely to be captured is set as shown in FIG. 7 (# 101 in FIG. 6). Hereinafter, such an image region is referred to as “attention image region RC”. The attention image area RC is set based on the position and orientation of the video camera 2 and the size of the human head. Or you may set with reference to the image image | photographed with the video camera 2 previously. In the human body detection device 1 according to the present embodiment, the size and shape of the attention image region RC is determined to be a rectangle of 640 × 100 pixels. Accordingly, it is only necessary to set the start address (the coordinates of the pixel in the upper left corner) of the target image area RC. For example, as shown in FIG. 7, only by giving (0, 20) as the coordinates (Xs, Ys) of the start address, the end address, that is, the coordinates (Xe, Ye) of the pixel in the lower right corner is determined automatically, and the target image area RC is set.

  For the image FG, the RGB value of the pixel is converted into a YUV value so that the pixel is sequentially scanned from the pixel at the coordinates (Xs, Ys) in the X-axis direction (the horizontal direction of the image) (No in # 102). , # 103, # 104). Such conversion can be performed, for example, by substituting the RGB value of the pixel into the following equation (1).

  When the conversion is completed for pixels on the line whose Y coordinate is Ys (Yes in # 102), pay attention to the line immediately below that, that is, the line of “Ys + 1” (# 105), and for the pixels on the line, Similarly, the RGB values of the pixels are converted into YUV values in order of each pixel (No in # 102, # 103, # 104). Thereafter, the above processing is repeated until the conversion for the coordinates (Xe, Ye) is completed.

  Then, after the conversion for all the pixels is completed (Yes in # 106), the cut-out image GC is generated by cutting out the image of the target image area RC converted into the YUV value (# 107). Note that the attention image area RC may be cut out from the image FG first, and the conversion process (# 103) may be performed on all the pixels in the attention image area RC.

  If the image FG input from the video camera 2 is an image in the YUV color space from the beginning, the conversion processing in steps # 102 to # 106 is not executed, and the setting of the attention image area RC in step # 101 and # 107 are performed. The cut image GC is acquired by performing only the cutout process.

  Returning to FIG. 4, as shown in FIG. 8, the image reduction processing unit 202 reduces the cut-out image GC to a predetermined magnification (in this embodiment, 1/4), and generates a cut-out reduced image GS. Process. Such processing is performed in the procedure as shown in FIG.

  The cut image GC is divided into blocks BK each having a size of 4 × 4 pixels, and the coordinates (X, Y) of the start address are set to (0, 0) (# 111 in FIG. 9). . Of the pixels of the cut-out image GC, the pixel values (YUV values) of the pixels belonging to the block BK whose coordinates at both ends of the diagonal line are (4X, 4Y) and (4X + 3, 4Y + 3) are substituted into the following equation (2). .

However, m = 4X and n = 4Y. Q (X, Y) is the YUV value of the pixel at the coordinate (X, Y) of the cut-out reduced image GS, and P (m, n) is the YUV of the pixel at the coordinate (m, n) of the cut-out image GC. Value.

  Thereby, the average (simple average) of the YUV values of the block BK at the upper left corner of the cut-out image GC is calculated (# 113). The calculated value is the YUV value of the pixel at the upper left corner of the cut-out reduced image GS.

  The average of the YUV values is similarly calculated for 159 blocks BK arranged to the right of the block BK (# 113, # 114), and the remaining 159 blocks of the first line in the horizontal direction of the cut-out reduced image GS are calculated. Get the YUV value of the pixel.

  Similarly, the YUV values of the 2nd to 100th lines of the cut-out reduced image GS are obtained by calculating the average of the YUV values of the pixels of the blocks BK in the 2nd to 25th stages of the cut-out image GC (# 112 to ## 116). Then, when the YUV value of the pixel at the lower right corner of the cut-out reduced image GS is obtained (Yes in # 116), the process ends.

  In this way, the cut-out image GC is reduced, and the cut-out reduced image GS is generated. The generated cut-out reduced image GS is used for a skin chromaticity plane generation process, a hair chromaticity plane generation process, a time difference calculation process, and a space difference calculation process, which will be described later. Further, the generated cut-out reduced image GS is stored in the previous frame storage unit 1M2 as necessary, as will be described later.

  Returning to FIG. 4, the time difference calculation unit 203, as shown in FIG. 10, is a cutout reduced image GS generated by the image reduction processing unit 202 (in the description of FIG. 10, “current cutout reduced image GSc”). And the brightness of the cut-out reduced image GS (referred to as “pre-cut-out reduced image GSp” in the description of FIG. 10) at a time before that (for example, two frames before). A difference (frame difference) is calculated, and a time difference plane ST is generated. It can be said that the generated time difference plane ST is a time difference image of the brightness of the current cut-out reduced image GSC and the previous cut-out reduced image GSp. In the present embodiment, the value of each pixel of the time difference plane ST is represented by a binary value (binary). Therefore, the time difference plane ST can be represented as a black and white image. The generation of the time difference plane ST is performed according to the procedure shown in FIG.

First, the coordinates (X, Y) of the start address are set to (0, 0) (# 121). Based on the following equation (3), the brightness of each (0, 0) pixel of the current cut reduced image Gsc and the previous cut reduced image GSp, that is, the difference between the Y components of the YUV values (lightness difference) is obtained. .
Buffer = abs | Yc (i, j) −Yp (i, j) | (3)

However, Yc (i, j) and Yp (i, j) are Y components of the YUV value of the pixel at the coordinates (i, j) of the current cut reduced image GSp and the previous cut reduced image GSp, respectively. abs | A | is the absolute value of A.

  The obtained buffer is binarized. For example, when Buffer exceeds the threshold value THst, the brightness difference between the two images is set to “1”, and when it is equal to or less than the threshold value THst, the brightness difference is set to “0” (# 123). When the YUV value is 256 gradations, for example, “10” is set as the threshold value THst.

  Similarly, the brightness difference between the corresponding pixels is obtained while shifting one pixel at a time in the X-axis direction (the horizontal direction of the image) (# 123, # 124). When the brightness difference of the rightmost pixel is obtained (Yes in # 122), the pixel is shifted by one pixel in the Y-axis direction (vertical direction of the image) (# 125), and similarly, the brightness difference is obtained from the left edge toward the right edge ( # 123, # 124). When the brightness difference of the pixel in the lower right corner is obtained (Yes in # 126), the process is terminated. In this way, the time difference plane ST is generated.

  Returning to FIG. 4, the spatial difference calculation unit 204 calculates the spatial difference of the cut-out reduced image GS generated by the image reduction processing unit 202 according to the procedure shown in FIGS. Difference image).

  Spatial difference processing is performed on each pixel of the cut-out reduced image GS (# 133 in FIG. 13). For example, as in the case of the time difference detection process described with reference to FIG. 11, the spatial difference process is performed from the upper left corner pixel to the lower right corner pixel in order.

  In the spatial difference processing, as shown in FIG. 12, first, the pixel values of the pixel to be processed and the surrounding eight pixels are substituted into the following equations (4) and (5). That is, the SOBEL filter is applied.

However, P (i, j) is the value of the brightness (Y component) of the pixel (i, j) of the cut-out reduced image GS, and Q1 (i, j) and Q2 (i, j) are cut out respectively. It is the output result by the horizontal edge detection vertical SOBEL filter and the vertical edge detection horizontal SOBEL filter for the pixel (i, j) of the reduced image GS. K1 (m, n) and K2 (m, n) are a horizontal edge detecting vertical SOBEL filter and a vertical edge detecting horizontal SOBEL filter having values as shown below.

  The Sobel plane SB is obtained by substituting the calculated values of Q1 (i, j) and Q2 (i, j) into the following equation (6) (# 141 in FIG. 12).

  The Sobel plane SB is smoothed by applying a smoothing filter shown in the following equation (7) (# 142).

However, QS (i, j) is the value of the brightness (Y component) of the pixel at the coordinates (i, j) of the smoothed Sobel plane SB, and KS (m, n) is a value as shown below. This is a smoothing filter.

  Then, binarization processing is performed based on the following equation (8) (# 143).

However, Buffer = QS (i, j) −Sbl (i, j). Thss is a threshold value. For example, “6” is set as THss.

  By performing such an operation, a spatial difference plane SS is generated. The spatial difference plane SS is represented as a black and white image.

  Returning to FIG. 4, the logical product image generation unit 205 is calculated by the time difference plane ST and the space difference calculation unit 204 calculated (generated) by the time difference calculation unit 203 as shown in FIGS. 14 and 15. A logical product plane AN is generated by calculating a logical product of pixel values (binary values) of pixels corresponding to each other in the spatial difference plane SS.

  In the logical product plane AN, as shown in FIG. 16, only the edge (contour) of the moving object appears. That is, it can be said that the logical product plane AN is an edge image (contour image) of a moving object. The procedure for generating the logical product plane AN is as shown in the flowchart of FIG. In the flowchart of FIG. 15, the AND operation process (# 153) is described so as to sequentially scan from the upper left corner pixel to the lower right corner pixel. Since the processing in this order is the same as in the case of FIGS. 11 and 13, detailed description thereof is omitted. The same applies to FIG. 19, FIG. 20, FIG. 25, FIG.

[Generation of feature plane]
17 is a diagram showing examples of four types of feature planes 8, FIG. 18 is a diagram showing the relationship between the skin chromaticity and the values of the U and V components of the pixel values in the YUV space, and FIG. 19 is the skin chromaticity plane. FIG. 20 is a flowchart illustrating an example of the flow of hair chromaticity plane generation processing, FIG. 21 is a diagram illustrating offset correction, and FIG. 22 is a pixel of hair chromaticity and YUV space. FIG. 23 is a diagram illustrating an example of templates TP1 and TP2, FIG. 24 is a diagram illustrating an example of a method for creating templates TP1 and TP2, and FIG. 26 is a flowchart illustrating an example of the flow of centrality plane calculation processing, FIG. 26 is a flowchart illustrating an example of the flow of voting processing, and FIG. 27 is a template matching method using the template TP1. Diagram illustrating the FIG. 28 FIG, 29 illustrates an example of aging characteristics of plane storage unit 1M1 is a diagram showing an example of a control method of the feature calculation unit 103 by the feature amount calculation control unit 106.

  Returning to FIG. 4, the feature amount calculation unit 103 includes a skin chromaticity plane generation unit 301, a semi-elliptical centrality plane generation unit 302, a hair chromaticity plane generation unit 303, a shoulder centrality plane generation unit 304, and the like. An arithmetic process for generating four types of feature degree planes 8 (skin chromaticity plane 8FC, semi-elliptical centrality plane 8SE, hair chromaticity plane 8HC, shoulder centrality plane 8SH) is performed. As shown in FIG. 17, these feature level planes 8 are represented as a grayscale image (brightness image) in which the magnitude of the value of each pixel is expressed by the density. The same applies to the head centrality plane 84 (see FIG. 30) described later.

  The skin chromaticity plane generation unit 301 generates the skin chromaticity plane 8FC by detecting the skin chromaticity of each pixel of the cut-out reduced image GS generated by the image reduction processing unit 202. “Skin chromaticity” means the skin color. That is, the skin chromaticity increases as the skin color is closer or similar. In this embodiment, when the values of the U component and the V component of the pixel value (here, the YUV value) are FCu and FCv, respectively, the skin color is set to be maximum. For example, when the beige color is used as the skin color, “107” and “157” are set as FCu and FCv, respectively. The skin chromaticity is calculated by substituting the pixel value into the following equation (9).

Where abs | A | is the absolute value of A.

  Note that the relationship between the U component value and the V component value of the pixel and the skin chromaticity shown in the equation (9) is expressed as shown in FIG.

  The procedure for detecting the skin chromaticity of each pixel of the cut-out reduced image GS is as shown in steps # 161 to # 166 of FIG. That is, the skin chromaticity detection plane is generated by sequentially obtaining the skin chromaticity of each pixel of the cut-out reduced image GS (# 163).

  By the way, if the face (head) of the pedestrian HMN is shown in the image FG (see FIG. 5) photographed by the video camera 2, the skin chromaticity detection plane obtained by the processing of steps # 161 to # 166 is included. The image area with high skin chromaticity and the area of the head of the pedestrian HMN should almost coincide. Therefore, it is considered that the center of the head of the pedestrian HMN, which is the search target, is located in the pixel or image region where the skin chromaticity peak is in the skin chromaticity detection plane.

  However, not all of the face is skin-colored, but includes parts with colors other than skin-colored, such as eyebrows, eyes, nostrils, and lips. Further, the pedestrian HMN may be wearing glasses, or the original image FG itself may contain noise. Therefore, in the present embodiment, for example, the following correction process is performed in order to increase the accuracy of detecting the center of the head.

  That is, first, a 3 × 3 luminance (brightness) maximum value filter is applied to the skin chromaticity detection plane obtained in steps # 161 to # 166 in order to remove portions other than the skin color and noise. Dilation processing is performed (# 167). In order to increase the sharpness, Erosion processing is performed by applying a 3 × 3 luminance (lightness) minimum value filter to the skin chromaticity detection plane that has been subjected to Dilation processing, and the skin color before the processing of Erosion processing is performed. An average process of the degree detection plane and the processed skin color degree detection plane is performed (# 168). The process of step # 168 is repeated “(HW / 2) +1” times. “HW” is the number of pixels indicating the width of the head, which is a detection target assumed to appear in the cut-out reduced image GS. For example, when it is assumed that the number of pixels indicating the width of the head is “7”, the process is repeated four or five times.

  As described above, the skin chromaticity of the cut-out reduced image GS is detected (# 161 to # 166), and the correction processing (# 167, # 168) for improving the detection accuracy is executed, thereby FIG. A skin chromaticity plane 8FC as shown in FIG.

  Returning to FIG. 4, the hair chromaticity plane generating unit 303 detects the hair chromaticity of each pixel of the cut-out reduced image GS generated by the image reduction processing unit 202, as shown in FIG. 17C. A simple hair chromaticity plane 8HC is generated. “Hair chromaticity” means the color of hair. That is, the closer to or similar to the hair color, the greater the hair chromaticity. In the present embodiment, the hair chromaticity is set to the maximum when the U component and V component values of the pixel value are HCu and HCv, respectively. For example, when black is used as the hair color, “112” and “142” are set as FCu and FCv, respectively.

  The procedure for generating the hair chromaticity plane 8HC is basically the same as the procedure for generating the skin chromaticity plane 8FC. That is, as shown in FIG. 20, the hair chromaticity is calculated for each pixel of the cut-out reduced image GS (# 171 to # 176), and processing for increasing the accuracy of detecting the center of the head is performed (# 177, # 178). However, the function of the following equation (10) is used as a hair chromaticity detection function instead of the function of equation (9).

  As shown in FIG. 21, the region where high hair chromaticity is detected is a region RYk at the top of the head where the hair grows. However, as described above, the purpose of this embodiment is to detect the center of the head (point Ptc). Therefore, in the equation (10), the Y-axis direction (vertical direction) is used so that the center of the region where the high hair chromaticity is detected (point Ptk) matches the center of the head (point Ptc) as much as possible. ) Is adjusted for offset (offset correction). In the equation (10), “offset” is an offset value, and is set to, for example, offset = HS / 2. “HW” is the number of pixels indicating the length from the top of the head to the chin, which is a detection target assumed to be captured in the cut-out reduced image GS. For example, when the number of pixels indicating the length is assumed to be “9”, offset = 4.5.

  The relationship between the U component value and the V component value of the pixel and the hair chromaticity shown in equation (10) is expressed as shown in FIG.

  Returning to FIG. 4, the semi-elliptical centrality plane generating unit 302 detects the semi-elliptical centrality of each pixel of the logical product plane AN (edge image) generated by the logical product image generating unit 205, thereby performing FIG. A semi-elliptical centrality plane 8SE as shown in (b) is generated. “Semi-elliptical centrality” means that when the template matching using the semi-elliptical template TP1 of FIG. 23A is performed on the logical product plane AN, the pixel of the logical product plane AN is the center of the template. It means how close to the position (reference point CT1), that is, the centrality. As a template matching method, for example, a method based on the Hough transform method described later is used.

  The template TP1 and the template TP2 used in the shoulder centrality plane generation unit 304 described later are created as follows, for example.

  First, a person who becomes a model stands at the reference position L1 (see FIG. 2) of the passage and the video camera 2 takes a picture. The model person should have a standard figure. As shown in FIG. 24 (a), attention is paid to the contour portion of the model in the image obtained by photographing.

  As shown in FIG. 24B, one open curve indicating the upper half of the head and two open curves indicating both shoulders are extracted as edges EG1 and EG2, respectively, from the contour of the model. At this time, predetermined positions apart from the edges EG1 and EG2 are determined as reference points CT1 and CT2, respectively. The reference points CT1 and CT2 indicate the reference positions (center positions) of the templates TP1 and TP2, respectively. The reference points CT1 and CT2 may be set at predetermined positions on the edges EG1 and EG2, respectively. The reference points CT1 and CT2 are also extracted together with the edges EG1 and EG2.

  Then, a template image as shown in FIGS. 24B and 24C is obtained. Then, the edges EG1 and EG2 are rotated halfway (rotated 180 degrees) around the reference points CT1 and CT2, respectively. In this way, templates TP1 and TP2 are created. The templates TP1 and TP2 are stored (stored) in the template storage unit 1M3 of FIG.

  Alternatively, if the size of the head shown in the logical product plane AN is assumed not to be very large (for example, assumed to be about 10 pixels in length and width), a semi-ellipse having the assumed size is represented by CG software or the like. May be used as the template TP1. The same applies to the template TP2.

  Α1 and α2 in FIG. 24A are offset values used for adjustment of position shift (offset correction) in template matching. As described above, the purpose of this embodiment is to detect the center of the head. Therefore, it is desired that the peak position of the semi-elliptical centrality matches the center position of the head as much as possible.

  The procedure for generating the semi-elliptical centrality plane 8SE is as shown in FIG. That is, first, one counter is prepared for each pixel of the AND plane AN, these counters are reset to “0” (# 180), and the coordinates of the start address are set to (0, 0). (# 181). The voting process is performed with the pixel of (0, 0) in the logical product plane AN as the target pixel (# 183). The voting process is performed according to the procedure shown in FIG.

  First, it is determined whether or not the pixel of interest is on an edge (outline) reflected in the logical product plane AN (# 191). If it is not on the edge (No in # 191), the voting process for that pixel of interest ends, and the process proceeds to step # 184 in FIG.

  As shown in FIG. 27B, when the target pixel is on the edge RN of the logical product plane AN (Yes in # 191), the logical value is set so that the target pixel matches the reference point CT1 of the template TP1. The template TP1 is overlaid on the product plane AN (# 192).

  Paying attention to the txsize × size area (dotted line area in FIG. 27B) of the logical product plane AN overlapping the template TP1, the processes of steps # 193 to # 198 are performed. That is, a pixel that overlaps the edge EG1 is found in the dotted line area. Then, as shown in FIG. 27D, “1” is added to the counter of the pixel shifted (offset correction) by the offset value α1 (see FIG. 24A) from the found pixel to obtain one vote. cast.

  In FIG. 27C, a thick square indicates a pixel on the edge RN of the logical product plane AN, a black square indicates a target pixel, and a hatched square overlaps the edge EG1 of the template TP1. The pixel is shown.

  Returning to FIG. 25, similarly, the other pixels of the AND plane AN are similarly subjected to voting processing (# 182 to # 186). As a result of voting as described above, the distribution of the number of votes counted by the counter of each pixel becomes a semi-elliptical centrality plane 8SE shown in FIG.

  Returning to FIG. 4, the shoulder center degree plane generation unit 304 detects the shoulder center degree of each pixel of the logical product plane AN (edge image) generated by the logical product image generation unit 205, thereby FIG. ) Is generated as shown in FIG. The “shoulder center degree” means that when the template matching is performed with the template TP2 having the shoulder shape of FIG. 23B on the logical product plane AN, the pixel of the logical product plane AN is the center position of the template ( It means how close to the reference point CT2), that is, the centrality.

  The procedure for generating the shoulder centrality plane 8SH is the same as the procedure for generating the semi-elliptical centrality plane 8SE shown in FIGS. However, template TP2 (see FIG. 23B) for template matching is used, and α2 (see FIG. 24A) is used as an offset value for offset correction.

  As shown in FIG. 28, the feature plane storage unit 1M1 stores a skin chromaticity plane storage area MFC, a semi-elliptical centrality plane storage area MSE, a hair chromaticity plane storage area MHC, a shoulder centrality plane storage area MSH, and the like. Has a region. In these storage areas, the latest skin chromaticity planes generated by the skin chromaticity plane generation unit 301, the semi-elliptical centrality plane generation unit 302, the hair chromaticity plane generation unit 303, and the shoulder centrality plane generation unit 304, respectively. 8FC, semi-elliptical centrality plane 8SE, hair chromaticity plane 8HC, and shoulder centrality plane 8SH are stored (stored).

  The feature amount calculation control unit 106 controls each of the plane generation units 301 to 304 included in the feature amount calculation unit 103 at the timing shown in FIG.

  First, when the image FG of the “4n-3” frame (1, 5, 9,...) Is input by the image input unit 101, the feature amount calculation control unit 106 uses the image FG to reduce the image reduction processing unit. A command is given to the skin chromaticity plane generating unit 301 so as to generate the skin chromaticity plane 8FC based on the cut-out reduced image GS generated by 202 (FIG. 29A). In FIG. 29, “n” is a positive integer. In the skin chromaticity plane storage area MFC, the previously stored skin chromaticity plane 8FC is deleted, and the new skin chromaticity plane 8FC generated this time is stored. That is, the skin chromaticity plane 8FC is updated (replaced).

  When the next image FG is input, the semi-elliptical centrality plane generation is performed so as to generate the semi-elliptical centrality plane 8SE based on the logical product plane AN generated by the logical product image generation unit 205 from the image FG. A command is given to the unit 302 (FIG. 29B). In the semi-elliptical centrality plane storage area MSE, the semi-elliptical centrality plane 8SE is updated to a new one. Note that the cut-out reduced image GS generated in the process of generating the logical product plane AN is stored in the previous frame storage unit 1M2 (see FIG. 4) to be used when generating the logical product plane AN after two frames. Remembered.

  Further, when the next image FG is input, a command is given to the hair chromaticity plane generation unit 303 to generate the hair chromaticity plane 8HC based on the cut-out reduced image GS generated from the image FG. (FIG. 29 (c)). In the hair chromaticity plane storage area MHC, the hair chromaticity plane 8HC is updated to a new one.

  Further, when the next image FG is input, a command is given to the shoulder center degree plane generation unit 304 to generate the shoulder center degree plane 8SH based on the logical product plane AN generated from the image FG. (FIG. 29 (d)). In the shoulder centrality plane storage area MSH, the shoulder centrality plane 8SH is updated to a new one. Note that the cut-out reduced image GS generated in the process of generating the logical product plane AN is stored in the previous frame storage unit 1M2 (see FIG. 4).

  When the next image FG is further input, the process returns to FIG. 29A and gives the above-described command to the skin chromaticity plane generation unit 301. Hereinafter, similarly, the commands in FIG.

[Detection of the center of the head]
30 is a diagram illustrating an example of the head center degree plane 84, FIG. 31 is a flowchart illustrating an example of the flow of the head center detection process, FIG. 32 is a diagram illustrating an example of extraction of the rectangular region KR, and FIG. Is a flowchart illustrating an example of the flow of the head extraction process, FIG. 34 is a diagram illustrating an example of a method of generating the head detection result plane TK, and FIG. 35 is a diagram illustrating an example of the search region RT and the square sum calculation range NR. FIG. 36 is a diagram for explaining an example of the shape and size of the regions TR1 and TR2 to be extracted or cleared from the rectangular region KR1, and FIG. 37 uses the head detection result plane TK of the previous frame. It is a figure which shows the example of the method of detecting a head center.

  Returning to FIG. 4, the head centrality plane generating unit 104 performs processing for generating the head centrality plane 84 as shown in FIG. 30. “Head centrality” means the degree of centrality of the head, that is, the degree of proximity to the center of the head. The head centrality plane 84 is based on the four types of characteristic degree planes 8 shown in FIG. 17, that is, the skin chromaticity plane 8FC, the semi-elliptical centrality plane 8SE, the hair chromaticity plane 8HC, and the shoulder centrality plane 8SH as follows. Is generated.

For example, assume that an image FG of the 21st frame is input. At this timing, as can be seen from FIG. 29, only the skin color plane 8FC of the four types of feature planes 8 is generated. The head centrality plane generation unit 104 acquires the newly generated skin chromaticity plane 8FC, and at the same time, generates the other three types of characteristic planes 8 (the semi-elliptical centrality plane 8SE of the 18th frame). , 19th frame hair chromaticity plane 8HC and 20th frame shoulder centrality plane 8SH) are obtained from the feature plane storage unit 1M1. Then, the plane values (skin chromaticity, semi-elliptical centrality, hair chromaticity, and shoulder centrality) of pixels corresponding to each other of these four feature planes 8 are substituted into the following equation (11).
TB (i, j) = {Min (fc (i, j), hc (i, j))
+ Min (fc (i, j), se (i, j))
+ Min (hc (i, j), se (i, j))
+ Min (fc (i, j), sh (i, j))
+ Min (hc (i, j), sh (i, j))
+ Se (i, j) + sh (i, j)} / 7 (11)
However, Min (A, B) is a function that selects and outputs the smaller value of A and B. TB (i, j) means the pixel value of the pixel at the coordinates (i, j) of the head centrality plane 84. fc (i, j) means the pixel value (skin chromaticity) of the pixel at the coordinates (i, j) of the skin chromaticity plane 8FC. se (i, j) means the pixel value (half-elliptic centrality) of the pixel at the coordinates (i, j) of the semi-elliptical centrality plane 8SE. hc (i, j) means the pixel value (hair chromaticity) of the pixel at the coordinates (i, j) of the hair chromaticity plane 8HC. sh (i, j) means the pixel value (shoulder center degree) of the pixel at the coordinates (i, j) of the shoulder center degree plane 8SH.

  In this way, by calculating TB for 160 × 25 pixels, the head centrality plane 84 shown in FIG. 30 is generated.

  The detection processing unit 105 includes a head centrality peak detection unit 501, a detection result plane generation unit 502, a count processing unit 503, and the like, and the head centrality plane 84 generated by the head centrality plane generation unit 104. Based on the above, a process is performed to detect the center of the head of the pedestrian HMN shown in the image FG (see FIG. 5).

  The head centrality peak detection unit 501 detects one or a plurality of positions predicted to have the center of the head of the pedestrian HMN in the head centrality plane 84 related to the detection target image FG. The detection result plane generation unit 502 finally determines the position of the center of the head of the pedestrian HMN based on the position detected by the head centrality peak detection unit 501, and the head detection result indicating the result A plane TK is generated. These processes are performed according to the procedure shown in FIG.

  First, an oblique histogram as shown in FIG. 32 is obtained according to the density (head center degree) value of each pixel of the head center degree plane 84 (# 201 in FIG. 31). Paying attention to the horizontal shading histogram, the value of each pixel is examined in order from the coordinate (0, 0) in the horizontal direction, and a range in which the frequency equal to or higher than the threshold value HIST_MIN (for example, “10”) is detected is detected. (# 202, # 203). If nothing is detected (No in # 204), it is assumed that no pedestrian is shown in the detection target image FG, and this detection process is terminated.

  If it is detected (Yes in # 204), the vertical shading histogram of the area belonging to the detected horizontal range is noticed, and a range in which the frequency equal to or higher than the threshold HIST_MIN is detected (# 206). ).

  If no continuous range is found (No in # 207), the process returns to step # 203 to check whether there is any other horizontal range in which the frequency equal to or higher than the threshold value HIST_MIN is continuous.

  If found (Yes in # 207), for the rectangular area KR in the range found in steps # 203 and # 206 (for example, the rectangular area KR1 in xa ≦ x ≦ xb, ya ≦ y ≦ yb in FIG. 32) Then, the head extraction process described below is performed (# 208).

  As shown in FIG. 33, the peak of the pixel value (head centrality) is detected from the rectangular area KR (for example, the rectangular area KR1) (# 221). Then, as shown in FIG. 34A, three peak pixels (hereinafter referred to as “peak pixel Pk”) are detected.

The center of the head is detected by determining whether or not the detected peak pixel Pk satisfies the following two requirements.
(A) The pixel value (head centrality) of the peak pixel Pk exceeds the threshold value TH_MAX.
(B) The sum of squares (maximum sum of squares) of the sum of squares calculation range around the peak pixel Pk after fine adjustment is equal to or greater than the threshold value TH_SUM2.

  There is a possibility that the center of the head exists in or around the peak pixel Pk that satisfies the above requirement (A), and there is no possibility that the center of the head exists in and around the peak pixel Pk that does not satisfy the requirement (A). (# 222). As a result of the determination, it is assumed that the peak pixel Pk1 does not satisfy the requirement (A) (No in # 222), and the peak pixels Pk2 and Pk3 satisfy the requirement (A) (Yes in # 222). In this case, attention is paid only to the peak pixels Pk2 and Pk3.

  Fine adjustment (fine correction) of the positions of the current peak pixels Pk2, Pk3 is performed as follows (# 223) so that the peak pixels Pk2, Pk3 and the center of the head of the detection target coincide with each other more accurately. .

  First, as shown in FIG. 35A, a region of horizontal ((w / 2) +1) pixels and vertical ((h / 2) +1) pixels centered on the position of the current peak pixel Pk2. (Hereinafter referred to as “search area RT”). For each pixel in the search region RT, the pixel is set to the center of the square sum calculation range NR shown in FIG. 35B, and the square sum of the pixel values in the square sum calculation range NR is calculated. The pixel at the center of the square sum calculation range NR when the calculated square sum is maximized is set as a new peak pixel Pk2. Thereby, fine adjustment of the peak pixel Pk is completed. Similarly, fine adjustment is performed for the peak pixel Pk3.

  Returning to FIG. 33 and FIG. 34, it is determined whether or not the sum of squares of the finely adjusted peak pixels Pk2 and Pk3 is equal to or greater than the threshold value TH_SUM2 (# 224 in FIG. 33). That is, it is determined whether or not the requirement (B) is satisfied. As a result of such determination, it is assumed that the peak pixel Pk2 satisfies the requirement (B) (Yes in # 224), but the peak pixel Pk3 does not satisfy the requirement (B) (No in # 224).

  In this case, the peak pixel Pk2 is regarded as the center of the head, and the region TR1 including the peak pixel Pk2 and surrounding pixels is extracted from the rectangular region KR1 as shown in FIG. 34B (# 225). Then, the extracted rectangular region KR1 is stored in the result plane storage unit 1M4 as the head detection result plane TK, or is output to the display device 1f (see FIG. 3) or the like. This head detection result plane TK is the final detection result of the center position of the head. In the rectangular region KR1, all the pixel values of the portion from which the region TR1 is extracted are erased (cleared) so as not to affect other detections (# 226).

  On the other hand, the peak pixel Pk3 and its surrounding pixels are considered not to be the head, and the region TR2 including the peak pixel Pk2 and its surrounding pixels is set as shown in FIG. ) Is erased (cleared) as shown in FIG.

  Note that the sizes of the regions TR1 and TR2 in FIG. 34 and the size of the head of the detection target are determined by a template as shown in FIG.

  Then, returning to FIG. 31, the detection process of steps # 203 and # 206 is repeated, and it is checked whether there is any other rectangular area KR as shown in FIG. If there is, the process shown in FIGS. 33 and 34 described above is performed on the rectangular area KR (# 208).

  When generating the head detection result plane TK, the head detection result plane TKp that is the detection result in the immediately preceding frame may be used to improve the detection accuracy. This is because the position of the center of the head indicated by the head detection result plane TKp of the immediately preceding frame and the position of the center of the head to be detected this time are almost the same, although there is a slight deviation depending on the movement of the pedestrian. Because it should be. For example, the head detection result plane TKp is used in the procedure as shown in FIG.

  The head centrality plane 84 ′ is generated by calculating the average of the pixel values of the corresponding pixels of the head centrality plane 84 related to the current frame and the head detection result plane TKp related to the previous frame. Then, based on the head centrality plane 84 ′, the detection processing unit 105 performs the processes shown in FIGS. 31 and 33 described above to generate a head detection result plane TK. The generated head detection result plane TK is stored in the result plane storage unit 1M4 in order to generate the head detection result plane TK of the next frame.

  Returning to FIG. 4, the count processing unit 503 counts the number of head detection result planes TK generated by the detection result plane generation unit 502, that is, the number of pedestrians HMN detected from the image FG.

  The head image display unit 107 extracts and enlarges the area of the head of the pedestrian HMN from the image FG based on the center position of the head shown in the head detection result plane TK, and displays this as an enlarged image. Displayed on the device 1f (see FIG. 3). Thereby, the monitoring person can identify pedestrian HMN easily. The head image storage unit 108 stores (records) an enlarged image of the head of the pedestrian HMN in the magnetic storage device 1d or an external recording medium (DVD-ROM, MO, CD-R, etc.).

  FIG. 38 and FIG. 39 are flowcharts for explaining an example of the overall processing flow of the human body detection device 1. Next, a processing flow of the human body detection device 1 when detecting a pedestrian from an image photographed by the video camera 2 will be described with reference to a flowchart.

  When the human body detecting device 1 receives an image FG (frame image) taken by the video camera 2 at a certain time (# 1 in FIG. 38), the human body detecting device 1 performs color space conversion as necessary (see FIG. 6), A cut-out reduced image GS is generated by cutting out and reducing the image area where the pedestrian's head is assumed (# 2). The procedure for generating the cut-out reduced image GS is as described above with reference to FIG.

  When the input image FG is an image of the 1st, 5th, 9th,... Frame, that is, an image of the “4n-3” frame (Yes in # 3, Yes in # 4), as shown in FIG. To generate a smooth skin chromaticity plane 8FC (# 5). In FIG. 38, “n” is a positive integer. 3, 7, 11,..., In the case of an image of the “4n−1” frame (Yes in # 3, No in # 4), the hair chromaticity plane 8HC as shown in FIG. Is generated (# 6). The procedure for generating the skin chromaticity plane 8FC and the procedure for generating the hair chromaticity plane 8HC are as described above with reference to FIGS. 19 and 20, respectively.

  If the input image FG is of an even frame (No in # 3), a logical product plane AN is generated based on the cut-out reduced image GS and the cut-out reduced image GS two frames before (# 7). ). When the input image FG is an image of the second, sixth, tenth,... Frame, that is, an image of the “4n-2” frame (Yes in # 8), a semi-elliptical shape as shown in FIG. A process of generating the centrality plane 8SE is performed (# 9). 4, 8, 12,..., In the case of the 4n frame image (No in # 8), a process for generating the shoulder center degree plane 8 SH as shown in FIG. ). The procedure for generating the AND plane AN is as described above with reference to FIG. The procedure for generating the semi-elliptical centrality plane 8SE and the shoulder centrality plane 8SH is as described above with reference to FIG.

  The latest plane generated in step # 5, # 6, # 9, or # 10 is stored in a predetermined area of the feature plane storage unit 1M1 as shown in FIG. 28 (# 11).

  Based on the latest plane generated in any of steps # 5, # 6, # 9, or # 10 and the plane generated in the immediately preceding three frames, that is, the latest skin chromaticity plane 8FC, half Based on the ellipse centrality plane 8SE, the hair chromaticity plane 8HC, and the shoulder centrality plane 8SH, a head centrality plane 84 as shown in FIG. 30 is generated (# 12 in FIG. 39). At this time, as described with reference to FIG. 37, the head detection result plane TK generated in the process for the previous frame may be used.

  Based on the generated head center degree plane 84, the position and center of the detection target pedestrian's head is detected (# 13), and the head showing the final detection result as shown in FIG. The unit detection result plane TK is generated and stored in the result plane storage unit 1M4 (# 14). The process of step # 13 is as described above with reference to FIG.

  The processing from step # 1 in FIG. 38 to step # 14 in FIG. 39 is repeatedly performed while the video camera 2 is shooting (No in # 15).

  According to the present embodiment, it is possible to detect a pedestrian without slowing down the processing speed even if inconvenience occurs in the shooting environment of the video camera 2.

  For example, when the pedestrian is wearing a hat, the outline of the pedestrian's head does not appear in the photographed image (frame), so the semi-elliptical centrality plane 8SE (see FIG. 17) cannot be obtained well. By supplementing this with the other three feature planes 8 obtained from the three frames, a pedestrian can be detected well. If the pedestrian is wearing a mask or sunglasses, the skin color area of the pedestrian will be very narrow, and the skin chromaticity plane 8FC will not be obtained well, but this will be supplemented by the other three feature planes 8 as well. Therefore, it is possible to detect a pedestrian well.

  In addition, pedestrians are decolorizing their hair, overlapping with other pedestrians walking in front, causing occlusion in the shoulder area, etc., or depending on the lighting and background (floor and wall) conditions Even if some feature planes 8 cannot be obtained successfully due to the fact that the hair color cannot be detected well, pedestrians can be detected well using other feature planes 8.

  FIG. 40 is a diagram for explaining a modification of the timing control method for generating the feature plane 8.

  In the present embodiment, any one type of feature plane 8 shown in FIG. 17 is generated for each frame of captured images (image FG in FIG. 5), but a plurality of feature planes 8 are generated depending on the generation processing time of each feature plane 8. It is also possible to generate the feature plane 8 of the type.

  For example, it is assumed that a captured image (frame) is transmitted from the video camera 2 every Tf seconds. In addition, the head is detected based on the degree of the four types of features A to D. FIG. 40A shows a processing time for generating a feature plane (hereinafter referred to as “feature A plane”, “feature B plane”,...) Indicating the degree of features AD. In this way, it is assumed that the times are Ta to Td, respectively. However, the times Ta to Td are 0.5Tf <Ta <Tf, 0 <Tb <0.5Tf, 0 <Tc <0.5Tf, 0.5Tf <Td <Tf, Ta + Tb> Tf, Ta + Tc> Tf, Tb + Td> It is assumed that the relationship of Tf, Tc + Td> Tf is satisfied.

  Assume that processing for generating a feature B plane is performed immediately after a certain frame is acquired. Then, although the time of “Tf−Tb” seconds remains until the next frame is acquired, it can be seen from the above relational expression that the time is 0.5 seconds or more. Therefore, immediately after the process of generating the feature B plane is completed, the process of generating the feature C plane can be performed. Therefore, as shown in FIG. 40B, processing for generating both the feature B plane and the feature C plane may be performed in Tf seconds.

  On the other hand, when the process of generating the feature A plane or the feature D plane is performed, any other feature degree is calculated from the above relational expression for the remaining “Tf-Ta” or “Tf-Td” seconds. It can be seen that a plane cannot be generated. Therefore, after the process of generating the feature A plane or the feature D plane is executed, generation of the feature plane is paused until the next frame is acquired.

  In this embodiment, the processing for each pixel constituting each image or each plane is performed in the order in which the horizontal direction is the main scanning direction, but may be performed in the order in which the vertical direction is the main scanning direction. Good. Alternatively, the processing order can be changed as appropriate, for example, processing is performed in order from the pixel in the lower right corner.

  In the present embodiment, four types of feature degrees, skin chromaticity, hair chromaticity, semi-elliptical center degree, and shoulder center degree, are obtained as feature degrees, and the center of the human head is detected based on these feature degrees. However, the detection may be performed by combining other types of features. For example, the human head based on the optical flow, the degree of matching with a template having a shape of a part such as an eye or mouth, the detection result using a detection method using a background difference, or the detection result using a detection method using a texture. The center of the part may be detected.

  The same type of feature degree may be calculated by a plurality of different methods and used for detecting the center of the head. For example, two different parameters A and B may be set, the skin chromaticity may be calculated using the parameter A in the 4nth frame, and the skin chromaticity may be calculated using the parameter B in the 4n + 2th frame. .

  The coefficients, constants, threshold values, functions, etc. used in the expressions (1) to (10) depend on the purpose of using the monitoring system 100, the environment or specifications of the installation location of the video camera 2, or other various conditions. It can be changed as appropriate. Further, the combination or generation order of the feature planes 8 can be changed as appropriate according to these conditions.

  In the present embodiment, when the center of the head of the pedestrian HMN is detected from the image FG in FIG. 5, the image FG is reduced and the reduced image (the cut reduced image GS in FIG. 8) is used. The image FG may be used as it is without being reduced. Although the skin chromaticity and the hair chromaticity are obtained based on the YUV space values, they may be obtained based on values of other color spaces such as the RGB space.

  It is also possible to detect an object other than a person using the human body detection device 1 according to the present invention. For example, the present invention can be applied to detection of an animal body, detection of a rectangular parallelepiped, or detection of a license plate such as an automobile or a motorcycle.

  In addition, the configuration of the whole or each part of the monitoring system 100, the human body detection device 1 and the video camera 2, the generation method of the feature plane 8, the calculation method of the centrality, the processing content, the processing order, and the like are in accordance with the spirit of the present invention. It can be changed as appropriate.

  According to the present invention, even if various inconveniences or problems occur, a target can be detected without reducing the processing speed. Therefore, the present invention is particularly preferably used for detecting a pedestrian or an intruder in a facility where the photographing environment is likely to change.

It is a figure which shows the example of the whole structure of a monitoring system. It is a figure which shows the example of the position and orientation of a video camera, and an imaging condition. It is a figure which shows the example of the hardware constitutions of a human body detection apparatus. It is a figure which shows the example of a functional structure of a human body detection apparatus. It is a figure which shows the example of the image image | photographed with the video camera. It is a flowchart explaining the example of the flow of a color space conversion process. It is a figure explaining the example of the production | generation method of a cutout image. It is a figure explaining the example of the production | generation method of a cut-out reduced image. It is a flowchart explaining the example of the flow of an image reduction process. It is a figure which shows the example of the production | generation method of a time difference plane. It is a flowchart explaining the example of the flow of a time difference detection process. It is a figure which shows the example of the production | generation method of a space difference plane. It is a flowchart explaining the example of the flow of a spatial difference detection process. It is a figure which shows the example of the production | generation method of a logical product plane. It is a flowchart explaining the example of the flow of a logical product image generation process. It is a figure which shows the example of a logical product plane. It is a figure which shows the example of four types of feature-value planes. It is a figure which shows the relationship between skin chromaticity and the value of U component of the pixel value of YUV space, and the value of V component. It is a flowchart explaining the example of the flow of a skin chromaticity plane production | generation process. It is a flowchart explaining the example of the flow of a hair chromaticity plane production | generation process. It is a figure explaining offset correction. It is a figure which shows the relationship between the hair chromaticity, the value of U component of the pixel value of YUV space, and the value of V component. It is a figure which shows the example of a template. It is a figure which shows the example of the preparation method of a template. It is a flowchart explaining the example of the flow of centrality plane calculation processing. It is a flowchart explaining the example of the flow of a voting process. It is a figure explaining the example of the method of template matching by a template. It is a figure which shows the example of the elderly of a feature-value plane memory | storage part. It is a figure which shows the example of the control method of the feature-value calculation part by the feature-value calculation control part. It is a figure which shows the example of a head centrality plane. It is a flowchart explaining the example of the flow of a process of a head center detection process. It is a figure which shows the example of extraction of a rectangular area. It is a flowchart explaining the example of the flow of a head extraction process. It is a figure explaining the example of the production | generation method of a head detection result plane. It is a figure which shows the example of a search area | region and a square sum calculation range. It is a figure explaining the example of the shape and size of the area | region used as the object extracted or cleared from a rectangular area. It is a figure which shows the example of the method of detecting the head center using the head detection result plane of the front frame. It is a flowchart explaining the example of the flow of the whole process of a human body detection apparatus. It is a flowchart explaining the example of the flow of the whole process of a human body detection apparatus. It is a figure explaining the modification of the control method of the timing which produces | generates a feature-value plane.

Explanation of symbols

1 Human body detection device (object detection device)
101 Image input unit (image input means)
105 Detection processing unit (object detection means)
106 Feature value calculation control unit (control means)
301 skin chromaticity plane generation unit (feature detection means, proximity calculation means)
302 Semi-elliptic centrality plane generation unit (feature detection means, proximity calculation means)
303 Hair chromaticity plane generation unit (feature detection means, proximity calculation means)
304 Shoulder centrality plane generation unit (feature detection means, proximity calculation means)
FG image (photographed image)
HMN pedestrian (object)

Claims (6)

An object detection device for detecting a target object from an image,
Image input means for inputting captured images obtained by shooting every predetermined time;
A plurality of feature detection means for detecting features of the captured image using different methods;
Control means for controlling the feature detection means so that the feature detection means is detected by the different feature detection means for the captured images adjacent to each other in time series;
The object in the photographed image is detected from the photographed image other than the photographed image by the feature detection means other than the feature detected by the feature detection means and the feature detection means. Object detection means for detecting based on the characteristics;
An object detection apparatus comprising: An object detection device for detecting a target object from an image,
Image input means for inputting captured images obtained by shooting every predetermined time;
A plurality of probability calculating means for determining the probability of the object appearing in the photographed image for each region partitioning the pixel plane of the photographed image using different methods;

Control means for controlling each of the likelihood calculation means so that the certainty is calculated by the different probability calculation means for each of the captured images adjacent to each other in time series;
The object shown in the photographed image is a photographed image other than the photographed image by the certainty calculation means other than the certainty calculated from the photographed image by the certainty calculating means and the certainty calculating means. Object detection means for detecting based on the probability obtained from
An object detection apparatus comprising: The certainty is represented by a numerical value,
The detection means adds the numerical value representing the probability for each region of the captured image between the regions at the same position, and detects the object by examining the peak of the added value.
The object detection apparatus according to claim 2. The object is a person;
As the probability calculation means, means for determining the likelihood of each area by determining the degree of matching with a semi-elliptical template, and detecting the skin color likelihood in the area of the photographed image The means for determining the likelihood of the above, the means for determining the likelihood of the area by detecting the color likelihood of the hair in the area of the photographed image, or the degree of matching with a shoulder-shaped template Means for determining the certainty of the area,
The object detection apparatus of Claim 2 or Claim 3. An object detection method for detecting a target object from an image,
Enter the captured images that were taken every predetermined time,
For the captured images that are adjacent to each other in time series, the features of the captured images are detected by different feature detection methods,
Features detected from the photographed image other than the photographed image by the feature detection method other than the feature detected from the photographed image by any one of the feature detection methods and the feature detection method. Detect based on the
An object detection method characterized by comprising: A computer program used in a computer for detecting a target object from an image,
A process of inputting a photographed image obtained by photographing every predetermined time;
A process for detecting features of the object with different feature detection methods for the captured images adjacent to each other in time series,
Based on the feature detected from the photographed image by any one of the feature detection methods and the feature of the photographed image other than the photographed image by a feature detection method other than the feature detection method, the object reflected in the photographed image Processing to detect,
A computer program for causing a computer to execute.
An object detection device for detecting a target object from an image,
Image input means for inputting captured images obtained by shooting every predetermined time;
A plurality of proximity calculation means for determining the proximity of the object in the captured image for each region that divides the pixel plane of the captured image using different methods;
Control means for controlling the proximity calculation means so that the proximity is calculated by the different proximity calculation means for the captured images adjacent to each other in time series;
The object shown in the photographed image is a photographed image other than the photographed image by the proximity calculating means other than the proximity calculated from the photographed image by any of the proximity calculating means. Object detection means for detecting based on the proximity obtained from:
An object detection apparatus comprising:

Images