JP4613558B2 - Human body detection device using images - Google Patents

Description

  The present invention relates to a human body detection apparatus using an image for detecting the presence or absence of a person in a detection area using a plurality of images arranged in order of time obtained by imaging a desired detection area by an imaging means. .

  2. Description of the Related Art Conventionally, there has been proposed a human body detection device that detects an existence of a person in a detection area using an image obtained by imaging a desired detection area by an imaging unit such as a TV camera. In this type of human body detection device, it is necessary to separate the moving object from the background within the detection region, and to separate the moving object from a person and a person other than the person. As a technique for separating a moving object from a background, a reference image in which a background image is registered in advance for a detection area is compared with a current image captured by an imaging unit, or a past image and a current image captured by an imaging unit are compared. Comparison is considered (see, for example, Patent Document 1). In either case, a difference image is generated in which the difference between the luminance values of each pixel of the two images to be compared is a pixel value, and the difference image is an area composed of pixels having a difference value larger than a predetermined threshold value. Generate a change region. In the change region, one person is divided into a plurality of regions, or noise other than a person is generated as the change region. Therefore, in the technique described in Patent Document 1, the area of the change region is a minute area with a predetermined threshold value or less. This area is regarded as a change area corresponding to one person by removing adjacent areas as noise and integrating adjacent change areas. For the integrated change area, a circumscribed rectangle (a circumscribed rectangle composed of sides in the vertical direction and the horizontal direction) that includes the change area is generated, thereby making the inside of the circumscribed rectangle an integrated area. The integrated area obtained in this way is an area where a person may exist, and it is evaluated whether the integrated area is an area corresponding to a person. Correlation is used for evaluation of the integrated region. When the correlation of the integrated region is large (similarity is high) between two images to be compared, it is regarded as a disturbance, and when the correlation is small, it is detected as a moving object, that is, a person. Yes.

  In addition, since the amount of calculation increases when correlation is used in evaluating the similarity of the integrated region, a simple similarity evaluation technique is to use the vertical and horizontal sides of the circumscribed rectangle that is the integrated region in the image. If the similarity is high compared to the aspect ratio corresponding to a preset person (if the difference in aspect ratio is less than the specified value), the integrated area It is considered that it is determined that the change area in the map corresponds to a person. The aspect ratio can be used to evaluate the change area in the integrated area because the integrated area generated corresponding to an upright person becomes vertically long (vertical dimension> horizontal dimension). The corresponding integrated region can be distinguished from an object that is horizontally long (vertical dimension <horizontal dimension).

  Similarly, the area of the circumscribed rectangle that is the integrated region in the image is obtained, and if the degree of similarity is high compared to the area corresponding to a preset person (if the area difference is less than the specified value), It is considered to determine that the change area in the integrated area corresponds to a person. The reason why the area is used for evaluating the change area in the integrated area is to distinguish a large object such as an automobile from a small object such as a mouse.

When evaluating the similarity of the integrated region, both the technology using the aspect ratio of the integrated region and the technology using the area of the integrated region can easily represent the feature value of the integrated region, so the feature value is calculated. The amount of calculation is small, and the degree of similarity can be evaluated in real time without requiring a special high-speed processing.
Japanese Patent Laid-Open No. 11-41589

  By the way, the technique using the correlation for evaluating whether the integrated region is a person or a person has a large amount of calculation, and if a similarity is to be evaluated in real time, an arithmetic unit with high processing capability is required. On the other hand, if the evaluation is performed based on the aspect ratio and the area, the amount of processing is reduced, so that the degree of similarity can be evaluated in real time without requiring an arithmetic device with high processing capability.

  However, in the technique using the aspect ratio, for example, when a small animal such as a dog is moving in the depth direction of the image, that is, when only the perspective changes within the field of view of the imaging means, The integrated area may be vertically long and cannot be distinguished from a person. Further, when a plurality of people overlap in the image, the integrated region may be horizontally long. In this case, there is a possibility that a person is mistaken for a small animal.

  On the other hand, in the technique using an area, for example, a car and a person can be easily distinguished if their sizes are clearly different. Depending on the area occupied in the image, the area occupied by the dog in the vicinity of the image sensor and the person distant from the image sensor are approximately equal in the image, and the person and the dog cannot be distinguished. There is.

  The present invention has been made in view of the above-mentioned reasons, and the object thereof is a human body using an image that can accurately distinguish whether or not an area extracted from an image corresponds to a person with a small amount of processing. It is to provide a detection device.

According to the first aspect of the present invention, there are provided an imaging unit for imaging a predetermined field of view, and three or more time-series images using edge images which are binary images obtained by extracting edges from images captured at different times by the imaging unit. A moving area extracting means for extracting an area corresponding to the moved object in the edge image at the time of interest by performing a logical operation to remove the background by combining the edge images, and the area extracted by the moving area extracting means on the edge Extraction of moving area by calculating frequency distribution of direction code of pixel and using similarity of calculated frequency distribution and reference data which is frequency distribution of direction code of pixel on edge previously obtained for human edge image A region analyzing means for evaluating whether or not the region extracted by the means is a region corresponding to a person. .

According to this configuration, the edge image is used when the region corresponding to the person is extracted from the image captured by the imaging unit. Since the edge image is a binary image, it is easy to use three or more edge images. A region corresponding to the moved object can be extracted simply by performing a simple logical operation. Further, a frequency distribution of direction codes is created for pixels on the edge of the extracted region, and it is determined whether or not the person is a person based on the similarity between the frequency distribution as reference data and the frequency distribution obtained from the image captured by the imaging means. Therefore, it is possible to reduce the processing amount by reducing the amount of information used for determining the degree of similarity, and it is also possible to accurately discriminate whether or not a person is using the frequency distribution of direction codes.

  In the invention of claim 2, in the invention of claim 1, the frequency distribution is normalized by the total number of pixels on each target edge, and the similarity evaluation value includes a frequency difference for each direction code. Using the sum of squares, the area analyzing means determines that the area extracted by the moving area extracting means is an area corresponding to a person when the evaluation value is equal to or less than a prescribed threshold value.

  According to this configuration, since the frequency distribution is normalized, it is possible to easily evaluate the degree of similarity between the frequency distribution obtained from the image captured by the imaging unit and the frequency distribution created in advance as reference data. It is possible to determine the degree of similarity with a simple calculation to the extent that the sum of squares of the frequency difference for each direction code is used as the evaluation value.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the region analysis means sets an upper limit value and a lower limit for the frequency of each direction code of the frequency distribution obtained for the region extracted by the moving region extraction means. A normal range based on values is set, and a region where a frequency distribution including a direction code whose frequency deviates from the normal range is obtained is regarded as a disturbance other than a person.

  According to this configuration, since the disturbance is determined by setting the upper limit value and the lower limit value for the frequency of the direction code, the disturbance can be easily removed, and the similarity degree is determined only for an area that is highly likely to correspond to a person. Since the evaluation is performed, the time required for calculating the similarity can be shortened.

  According to a fourth aspect of the present invention, in the first to third aspects of the present invention, the region analysis unit is configured to determine a pixel direction on an edge for each region extracted by the moving region extraction unit for a plurality of time-series edge images. Find the frequency distribution normalized with the total number of pixels on each edge for the code, and then evaluate the similarity of the frequency distribution between each pair of adjacent edge images in time series, and the same object between different edge images The areas corresponding to are associated with each other, and the associated areas are evaluated using the similarity to the reference data.

  According to this configuration, since the similarity of the frequency distribution is evaluated between adjacent edge images in time series and the region corresponding to the same object is associated, the correct correspondence can be obtained even when a plurality of objects are adjacent in the edge image. By comparing the reference data only with respect to the area where the association is possible, it is possible to correctly and efficiently determine whether or not the area indicates a person.

  According to a fifth aspect of the present invention, in the first to fourth aspects of the present invention, the field of view of the imaging unit is set at a ratio corresponding to the size of the image of the person existing in the field of view of the imaging unit in each part of the imaging surface. A monitoring area setting means has been added that has a function to divide the area into a plurality of areas, specify the distinction between the effective area and the invalid area for each area, and use the effective area as a monitoring area for detecting the presence or absence of a person. It is characterized by.

  According to this configuration, it is possible to divide the field of view of the imaging unit into a plurality of areas, and to designate each area as an effective area that monitors a person and an invalid area that does not monitor a person. By designating an area where disturbance is known to occur as an invalid area, the influence of the disturbance can be reduced, and the accuracy of detecting a person is increased.

  According to a sixth aspect of the present invention, in the first to fourth aspects of the invention, it is possible to select a region setting mode for setting a monitoring region for detecting the presence or absence of a person, and in the region setting mode, the imaging means is within the field of view. Only the light of a specific wavelength from the light source moved along the boundary line of the monitoring area is received, and the position where the density value is maximized in a plurality of images obtained in time series from the imaging means in the area setting mode is in time order. A monitoring area setting means is added, in which one of the inside and outside of the closed area obtained by connecting is an effective area, the other is an invalid area, and the effective area is a monitoring area for detecting the presence or absence of a person. To do.

  According to this configuration, a part that does not need to detect the presence / absence of a person in the field of view of the imaging unit or a part that is likely to generate a disturbance can be excluded from the monitoring target and designated as an invalid area. By designating an area with known as an invalid area, the influence of disturbance can be reduced and the accuracy of detecting a person is increased.

  According to a seventh aspect of the invention, in the first to sixth aspects of the invention, an area corresponding to a person is extracted by the image memory for temporarily storing an image for each image picked up by the image pickup device and the area analyzing means. Then, a storage memory for cutting out and saving an image stored in the image memory for the area is provided.

  According to this configuration, when an area corresponding to a person is extracted, the area is cut out from the image and stored in the storage memory, so that a useless image in which no area corresponding to the person exists is not stored. Moreover, since only the area corresponding to the person is cut out from the necessary images, the amount of data to be saved can be greatly reduced, and in addition, detailed images can be saved for the necessary images. Is possible.

  According to an eighth aspect of the present invention, in the first to sixth aspects of the invention, an area corresponding to a person is extracted by the image memory for temporarily storing an image for each image picked up by the image pickup device and the area analyzing means. Then, an image transmission unit that cuts out an image stored in the image memory for the region and transfers the image to another apparatus is provided.

  According to this configuration, when an area corresponding to a person is extracted, the area is cut out from the image and transferred to another apparatus via the image transmission unit. Traffic on the transmission line can be reduced without being transferred to another device. Moreover, since only the area corresponding to the person is cut out from the necessary images, the amount of data to be transferred can be greatly reduced, and in addition, detailed images can be transferred for necessary images. become.

  According to a ninth aspect of the present invention, in the first to eighth aspects of the invention, a detection signal output means for outputting a detection signal when there is an area that is evaluated as an area corresponding to a person in the area analysis means, and a detection When the detection signal is output from the signal output means, the area analysis means starts tracking the area evaluated as the area corresponding to the person, and a partially enlarged image obtained by enlarging the area from other areas is displayed on the screen of the image display means. An image output means for displaying is added.

  According to this configuration, when an area corresponding to a person is detected in the field of view of the imaging unit, it is determined that there is an intruder, and the area is tracked and enlarged on the screen of the image display unit than other areas. Therefore, when used for monitoring the field of view of the imaging means like a monitoring camera, the characteristics of the intruder can be easily grasped on the screen, and the burden on the monitoring person monitoring the screen of the image display means is small. Become.

  In the invention of claim 10, in the invention of claim 9, the image output means displays the partial enlarged image on the screen of the image display means at an enlargement ratio that matches the size of the screen of the image display means, An entire image that is the entire field of view of the imaging unit is displayed on a part of the screen of the image display unit.

  According to this configuration, the feature of the intruder can be easily captured on the screen by the partially enlarged image, and the intruder can be seen within the field of view of the imaging means by displaying the entire image on a part of the screen. You can know where it is at the same time.

  According to an eleventh aspect of the present invention, in the ninth aspect, the image output means is configured to display a whole image that is the entire field of view of the imaging means on the screen of the image display means, and then the screen size of the image display means. Until the state in which the partially magnified image is displayed at a magnification corresponding to the height, the region analysis means gradually increases the magnification rate of the partially magnified image over time starting from the region corresponding to the person and the region initially evaluated. It is characterized by being enlarged.

  According to this configuration, the image to be displayed on the screen of the image display means is not suddenly switched from the entire image to the partially enlarged image having the constant enlargement ratio, but the partially enlarged image is started from the place where the person is first detected on the screen. Since the enlargement ratio of the image is increased with time, the partially enlarged image is zoomed up with the passage of time from the place where the person was first detected on the screen, and the partial enlarged image with a constant enlargement ratio is obtained. It is easier to grasp the position where the intruder is present than when switching suddenly.

  According to a twelfth aspect of the present invention, in the invention according to the ninth aspect, when there are a plurality of regions that are evaluated as regions corresponding to a person by the region analyzing unit, the image output unit is configured to display the portion corresponding to each region. The enlarged image is displayed on the screen of the image display means by switching at regular intervals.

  According to this configuration, when there are a plurality of regions corresponding to a person, a partial enlarged image of each region is displayed by switching at regular intervals, so that it is easy to check the characteristics of each of a plurality of intruders. Become.

  In the invention of claim 13, in the invention of claim 9, when there are a plurality of areas evaluated as areas corresponding to people by the area analysis means, the image output means displays the screen of the image display means. The image is divided into sections corresponding to the number of areas, and the partial enlarged image corresponding to each area is displayed in each section.

  According to this configuration, since the actions of a plurality of intruders can be listed in one screen, the characteristics and actions of a plurality of persons can be monitored at a time, and the presence or absence of an intruder whose behavior is suspicious can be grasped. It becomes easy.

  According to the configuration of the present invention, the edge image is used when extracting the region corresponding to the person from the image picked up by the image pickup means, and the edge image is a binary image. There is an advantage that a region corresponding to can be extracted. Further, a frequency distribution of direction codes is created for pixels on the edge of the extracted region, and it is determined whether or not the person is a person based on the similarity between the frequency distribution as reference data and the frequency distribution obtained from the image captured by the imaging means. Therefore, there is an advantage that the amount of information used for determining the degree of similarity can be reduced and the amount of processing can be reduced, and the presence or absence of a person can be accurately distinguished by using the frequency distribution of direction codes.

(Embodiment 1)
In the present embodiment, as shown in FIG. 1, an imaging unit 1 that captures a desired field of view and a plurality of images captured at different times by the imaging unit 1 are used to extract a region corresponding to an object that has moved. The moving area extracting means 2 and the area analyzing means 3 for evaluating whether or not the area extracted by the moving area extracting means 2 is an area corresponding to a person. The moving area extracting means 2 and the area analyzing means 3 are realized by causing a computer to execute an appropriate program.

  The imaging unit 1 includes a camera 11 that outputs images captured at predetermined time intervals, and an A / D converter 12 that converts analog information of the images captured by the camera 11 into digital information. As the camera 11, a solid-state image sensor such as a CCD image sensor or a CMOS image sensor is used. When a CMOS image sensor having a function of outputting a digital signal is used for the camera 11, the A / D converter 12 is not necessary. A color image can be used as an image captured by the camera 11, but a monochrome grayscale image is employed in the present embodiment. The time interval captured by the imaging means 1 may be set as appropriate within a range where the presence or absence of a moving object can be determined from time-series images obtained at the time interval, and is not intended to obtain a smooth moving image. Therefore, it is not necessary to output an image of 30 frames per second.

  The moving area extracting unit 2 includes an image memory 21 that temporarily stores a grayscale image output from the imaging unit 1 and an image obtained by performing processing described later on the grayscale image. In the present embodiment, the differential processing unit 22 obtains a differential value and a direction code for the grayscale image, and the differential image in which the pixel value of each pixel is a differential value, and the pixel value of each pixel is the direction code. The direction code image is stored in the image memory 21 together with the grayscale image.

  Various methods for obtaining the differential value have been proposed. Basically, for the neighboring pixels of the pixel of interest (eight neighborhoods are widely adopted), the density difference in the vertical direction of the image is expressed as the density difference in the horizontal direction. The divided value is used as a differential value. However, the differential image is generated from the grayscale image by utilizing the fact that the differential value becomes large near the boundary between the object and the background due to the difference in the density value between the object and the background in the image. In this embodiment, weighted differentiation using a Sobel filter is performed to enhance the contour line.

  The direction code is a value in which the differential value is associated with the change direction of the density value, and an integer value code is associated in 8 directions in units of 45 degrees (in this case, from the neighboring pixels of 8). The direction code is associated with the obtained normal differential value). The direction code of each pixel is set so as to represent a direction orthogonal to the direction in which the density value change is maximized in the image. Therefore, the direction indicated by the direction code in each pixel substantially coincides with the extending direction of the contour line (the three pixels adjacent to each other within the range of ± 45 degrees with respect to the direction indicated by the direction code of each pixel are pixels on the contour line of the object. Is likely to be).

  In the differential image obtained by the differential processing unit 22 as described above, a portion having a large contrast is emphasized. Therefore, by binarizing the differential image with an appropriate threshold, candidates for the contour line of the object included in the differential image Can be extracted. The differentiation processing unit 22 thins the extracted region that is a candidate for a contour line to a width of one pixel, and extracts a candidate for an edge that is a candidate for a contour line. Since the edge candidates may be interrupted, the pixels are tracked using the direction code for the edge candidates, and an edge image composed of the edges that are concatenated with the edge candidates that can be regarded as the contour lines of the object is generated. Store in the memory 21. The image memory 21 is also used as a work area when obtaining an edge image.

  In this embodiment, the logic synthesis unit 23 extracts edges corresponding to a moving object using three or five edge images. Here, a technique for extracting an edge corresponding to a moving object using three edge images will be described with reference to FIG. Now, as shown in FIGS. 2A to 2C, three edge images E (T−ΔT), E (T), E (T + ΔT) imaged at times T−ΔT, T, and T + ΔT are logical. It is assumed that it is given to the synthesis unit 23. In the illustrated example, each edge image P includes a moving object Ob.

  First, the logic synthesis unit 23 obtains a difference between each pair of edge images adjacent in time series (that is, E (T−ΔT) and E (T), E (T) and E (T + ΔT)) (this image). Is an edge image difference, and is hereinafter referred to as a “difference edge image”). However, since the edge image is a binary image having different pixel values in the edge portion and the non-edge portion, the logic synthesis unit 23 performs exclusive logic for each pair of pixels at the same position for each pair of edge images. When the logical operation for obtaining the sum is performed, the difference between the pair of edge images of interest is obtained. In the two differential edge images obtained from the edge image in the illustrated example, the moving object Ob appears twice in each differential edge image.

  In the logic synthesis unit 23, in order to extract the moving object Ob included in the edge image E (T) at time T, a logical operation is performed to obtain a logical product for each pair of pixels at the same position for the two differential edge images. The resulting image is output as a candidate image as shown in FIG. That is, since the background is almost removed from the two differential edge images, the background is removed from the edge image E (T) at the time T, which is a common part, when the logical product operation is performed on the two differential edge images. A candidate image is obtained, and it is considered that this candidate image only includes noise in addition to the moving object Ob.

  Here, in the present embodiment, an example using three edge images E (T−ΔT), E (T), and E (T + ΔT) is shown, but candidate images are generated using four or more edge images. It is also possible to do. For example, when five edge images E (T−2ΔT), E (T−ΔT), E (T), E (T + ΔT), and E (T + 2ΔT) are used, two edge images (E For (T−2ΔT) and E (T + 2ΔT), E (T−ΔT) and E (T + ΔT)), a background edge image is generated by removing the moving object Ob by a logical operation for obtaining a logical product. When the logical operation for obtaining the logical product of the two edge images thus obtained and inverting the edge image E (T) is performed, a moving object is obtained in the edge images E (T−2ΔT) and E (T + 2ΔT). Edge image including moving object Ob in background and edge image E (T) hidden by Ob, and background image and edge image E () hidden by moving object Ob in edge images E (T−ΔT) and E (T + ΔT) An edge image including the moving object Ob in T) is obtained. If a common part is extracted by a logical operation for obtaining a logical product of both edge images, an edge image (candidate image) including the edge of the moving object Ob in the edge image E (T) is obtained. In addition, candidate images can be generated by variously combining four or more edge images.

  In the candidate image, the logical operation is performed on the binary edge image instead of obtaining the difference from the grayscale image, and the moving object Ob is not extracted from the two images but using three or more edge images. Since the moving object Ob included in the edge image at the specific time is extracted, the same moving object Ob does not appear in two positions in the candidate image, and only the changed region including the moving object Ob is included. Can be extracted. As described above, since the candidate image output from the logic synthesis unit 23 includes noise in addition to the moving object Ob, labeling is performed for each region (connected region) where pixels are connected. Here, if a circumscribed rectangle D1 is set for each connected region as shown in FIG. 2 (e) and labeling is performed on the circumscribed rectangle D1, compared with the case where a label is assigned to each pixel. The amount of data can be reduced.

  The candidate image as shown in FIG. 2D output from the logic synthesis unit 23 in the moving region extraction unit 2 and the direction code image stored in the image memory 21 are the frequency distribution creation unit provided in the region analysis unit 3. 31. The region analysis unit 3 has a function of evaluating whether the region extracted by the moving region extraction unit 2 is a region corresponding to a person or a disturbance other than a person. In the area analysis means 3, first, in the frequency distribution creation section 31, the direction code image stored in the image memory 21 is obtained for each area labeled in the candidate image obtained as the output of the logic synthesis section 23. With reference to the direction code of the pixel on the edge, a frequency distribution related to the direction code is generated for each labeled region. Here, the frequency distribution is normalized by the total number of pixels on each edge of interest. In addition, the direction codes do not use eight types of direction codes, but direction codes that are opposite to each other in the same direction are grouped into the same direction code, and a frequency distribution is generated for the four types of direction codes. That is, a direction code corresponding to 0 degrees and 180 degrees, a direction code corresponding to 45 degrees and 225 degrees, a direction code corresponding to 90 degrees and 270 degrees, a direction code corresponding to 135 degrees and 315 degrees, and The four types of direction codes are used. FIG. 3 shows an example of the frequency distribution generated by the frequency distribution creation unit 31.

  In the region analysis means 3, the frequency distribution for each region generated in the frequency distribution creating unit 31 is input to the disturbance removing unit 32, and it is determined whether or not the disturbance is based on the shape of the frequency distribution. That is, the disturbance removing unit 32 sets a normal range based on an upper limit value and a lower limit value for each direction code with respect to the frequency of each direction code when the region corresponds to a person, and the frequency distribution obtained for each region. When one of the degrees of each direction code deviates from the normal range, the area is regarded as a disturbance other than a person. That is, when the frequency of the direction code in the region deviates from the normal range, the moving object in the region is inclined in a specific direction and is regarded as noise other than a person. This is because the edge corresponding to a person has more curved parts than the straight line, and the edge corresponding to a person has a more complicated shape, so the frequency of all values of the direction code appearing in the pixels above the edge. This is based on an empirical rule that the edges of noise due to shadows and flicker generated by the camera 11 often show a distribution that is biased in a specific direction, while it is relatively high. In short, the disturbance removing unit 32 uses the frequency distribution related to the direction code of the pixel on the edge in each region as a feature amount, determines whether the moving object is a region corresponding to a person or noise other than a person, and The determined area is removed without being given to the distribution comparison processing unit 33 in the next stage.

  The region evaluated not to be noise by the disturbance removing unit 32 is given to the distribution comparison processing unit 33 to evaluate whether or not the region includes a person. The distribution comparison processing unit 33 uses the reference data storage unit 34 registered in advance as the reference data for the frequency direction distribution of the edge direction codes related to the person, and the frequency for each area remaining without being removed by the disturbance removal unit 32. The distribution is compared with the frequency distribution of the reference data stored in the reference data storage unit 34, and the similarity between the two is evaluated by the following calculation.

That is, the frequency stored in the reference data storage unit 34 with the frequency for each direction code as H1i (i = 1, 2, 3, 4) regarding the frequency distribution related to the region evaluated as not noise by the disturbance removing unit 32. when the frequency of each direction code with respect to the distribution and H2i (i = 1,2,3,4), the evaluation value ^{e 2} of similarity calculated by the number 1.

The evaluation value e ² obtained by Equation 1 is compared with an appropriately set threshold value, and when the evaluation value e ² is equal to or less than the threshold value, the similarity is high, so that the region obtained from the candidate image corresponds to a person. It determines that the region, when the evaluation value e ² exceeds the threshold value determines that the area obtained from the candidate image from a low degree of similarity is the disturbance of non-human.

As described above, the area of the moving object Ob is separated from the background by performing a logical operation between frames with respect to the time-series edge image, and the frequency distribution of the direction code of the edge included in the area of the moving object Ob is further characterized. Since an area having a high possibility of being a person is extracted by evaluating the evaluation value e ² with the reference data that is the frequency distribution of the direction code relating to the person, the moving object Ob is relatively easy to calculate. Whether or not is a person can be determined.

  Comparing the configuration of the present embodiment with the conventional configuration using the aspect ratio and area of the circumscribed rectangle, in the conventional configuration, when a plurality of people form one integrated region, the integrated region is determined not to correspond to a person. In contrast, in the configuration of the present embodiment described above, even if a plurality of persons are included in the region output from the disturbance removal unit 32, the direction code of the pixel on the edge When the frequency distribution is normalized, the same tendency as in the case of one person is shown, so that it is possible to evaluate whether the person is a person or a person other than the person in the area.

Furthermore, in this embodiment, the evaluation value e ² of the similarity of the frequency distribution is used as the square sum of the frequency difference for each direction code (that is, the square of the distance between the frequency distributions to be compared). Compared to template matching, the amount of reference data is small and the amount of comparison operation is also small. In template matching, if the target object does not deform, the correlation increases when the shape matches the template. However, for objects that change the shape in the image, such as people, match the template. It is difficult to obtain a large correlation. In template matching, in order to match the size of the target object in the image with the size of the template, enlargement / reduction processing is required, or multiple templates with the same shape but different sizes are required. Or On the other hand, since the frequency distribution of the direction code is used in this embodiment, even if the shape of the target object in the image changes, the frequency distribution does not change greatly, and the correlation with the reference data is easy. Can be evaluated.

  1 shows an example in which the image memory 21 is provided in the moving area extracting means 2, but there is no particular limitation on the location where the image memory 21 is provided, and the image memory 21 is provided in the area analyzing means 3 or the moving area extracting means 2 It may be provided in both the area analysis means 3 or only the image memory 21 may be provided separately.

  Further, in the configuration shown in FIG. 1, the direction code is obtained by the differential processing unit 22 provided in the moving area extracting means 2, and the direction code is not obtained by the area analyzing means 3 that actually uses the direction code. However, this is because the direction code is obtained from the differential value, and it is computationally efficient to obtain the direction code when the differential processing unit 22 obtains the differential value. However, the direction code may be obtained in the region analysis means 3 separately from the calculation for obtaining the differential value used for extracting the edge.

Furthermore, in the above-described example, one type of reference data is stored in the reference data storage unit 34. However, depending on the orientation of the person with respect to the camera 11 and the position where the person exists in the field of view of the camera 11, the reference data is used. Changes occur in the frequency distribution. Therefore, a plurality of types of frequency distributions are stored as reference data in the reference data storage unit 34, and the similarity between the frequency distribution for each region output from the disturbance removal unit 32 and each reference data is obtained. May be. In this case, when the evaluation value e ² for any of the reference data is below the threshold value is regarded as the area corresponding to the area in humans.

  In the above example, the frequency distribution regarding the four kinds of direction codes is used as the feature amount of each area of the moving object Ob obtained from the edge image. However, for simplicity, the frequency distribution in the two directions of the horizontal direction and the vertical direction of the image is used. Only the direction code may be used as the feature amount. In this case, it is possible to determine whether or not the area corresponds to a person by using the ratio of the direction codes in two directions for each area and the size of the ratio. For example, when the frequency of the direction code in two directions is H11 and H12, the size of H11 is compared with k · H12 (k: magnification constant), and when H11 <k · H12, It is determined that the area corresponds to. When evaluating whether each region output from the disturbance removal unit 32 is a region corresponding to a person as described above, the distribution comparison processing unit 33 and the reference data storage unit 34 are unnecessary.

(Embodiment 2)
In the present embodiment, a process for associating regions labeled between time-series edge images and removing uncorrelated regions as disturbances (noise) that occur only once, such as changes in light, will be described. In other words, many disturbances that are generally problematic in the field of view are light insertions and changes, and in the edge image, the shape of the region corresponding to this type of disturbance is compared to the shape of the region corresponding to a person. In this embodiment, the disturbance is removed using this property.

  Now, time-series edge images E (1), E (2), E (3), and E (4) used for extracting the region of the moving object Ob in the logic synthesis unit 23 are given as shown in FIG. It shall be. The illustrated edge images E (1), E (2), E (3), and E (4) are regions P11 to P14 corresponding to a person and regions corresponding to disturbance caused by light reflected on a window or the like. An example including N12 and N13 is shown. In such edge images E (1), E (2), E (3), and E (4), the regions P11 to P14 need to be associated with each other, and the regions N12 and N13 need to be removed without being associated with each other. is there.

Therefore, in this embodiment, each edge image E (1), E (2), E (3), E (4) is used in pairs in time order (each pair of edge images E ( 1), E (2), E (3), and E (4)), and the frequency distribution of the direction code is compared for each region. That is, first, the frequency distribution is obtained for the direction code of the region P11, the frequency distribution is also obtained for the direction codes of the region P12 and the region N12, and the similarity (evaluation value e ² used for comparison with the reference data in the first embodiment). Is calculated in the same manner as above. Here, the distance between the region P11 included in each edge image E (1) and the regions P12 and N12 included in E (2) may be obtained and added to the constraint condition for association. That is, a restriction condition is set for a distance range in which a moving object to be associated moves between a pair of adjacent edge images E (1) and E (2). Judge that it is not possible. If such a constraint condition is set, even if there are a plurality of regions in each of the edge images E (1) and E (2), it is possible to reduce the number of combinations of regions whose frequency distributions should be compared. This leads to faster processing. By the above-described processing, the area P11 and the area P12 can be associated with each other.

  Next, when the same processing is performed on the edge images E (2) and E (3), the region N12 and the region N13 are not associated with each other because the frequency distribution of the direction code is significantly different. Association with P13 is made. Further, by performing the same processing on the edge images E (3) and E (4), the region P13 and the region P14 are associated with each other.

  As described above, the regions P11 to P14 are associated with each other in each of the edge images E (1), E (2), E (3), and E (4), and the region is like a single noise. Since N12 and N13 are not easily associated with each other, they can be removed as disturbances. In other words, time-sequentially adjacent edge images E (1), E (2), E (3), and E (4) are evaluated using the degree of similarity of the frequency distribution, and an area corresponding to the same object is associated. When a region that can be determined to correspond to a person is obtained by comparing the region that can be associated with the reference data, the region is determined to be a region that corresponds to a person, and the region between images is associated This increases the possibility of removing noise.

  For example, the frequency distribution of the direction code in a region where the shape is not constant, such as light, may approximate the frequency distribution of the direction code in a region occupied by a person. This kind of disturbance can be removed by performing such association.

  Further, the distribution comparison processing unit 33 compares each region associated in the time-series edge image with reference data, thereby counting the number of edge images in which regions corresponding to people are continuously obtained. However, when this number is equal to or greater than a predetermined threshold within a predetermined time, it is desirable to determine the area as an area corresponding to a person. For example, if the threshold is set to 3, in the example shown in FIG. 4 described above, the four edge images E (1), E (2), E (3), and E (4) correspond to people. Since the regions P11 to P14 exist continuously, the above-described condition is satisfied, and the regions P11 to P13 in the three edge images E (1), E (2), and E (3) It can be determined that the area corresponds to.

  In the example shown in FIG. 5, there are areas N11 and N12 associated with the two edge images E (1) and E (2) for the five edge images E (1) to E (5). There are regions P11 to P15 associated with the five edge images E (1) to E (5). Now, attention is focused on the first two edge images E (1) and E (2). In the conventional technique, there is a technique for associating an area where the distance between representative points (such as the center of gravity) of each area is minimum between adjacent edge images. In this case, in the illustrated example, the area P11 of the edge image E (1) is associated with the area P11. The shorter distance between the region P12 and the region N12 of the edge image E (2) is associated. That is, there is a possibility that the region N12 is associated with the presence of noise even though the region P12 must be associated with the region P11. On the other hand, in the present embodiment, the region P11 and the region P12 or the region N12 are associated with each other regardless of the distance depending on the similarity of the frequency distribution of the edge direction values, so that regions having different edge characteristics are not associated with each other. It becomes possible to correctly associate the region P11 and the region P12. Moreover, since it is determined whether or not the associated areas P11 and P12 are people by comparing with the reference data, it is possible to remove the noise and reliably detect the area corresponding to the person.

  Further, as described above, the number of edge images in which the regions corresponding to each other and the regions corresponding to the person are continuously obtained within the time during which four edge images are obtained (the above-described predetermined time) are obtained. When the number of images is three or more (the above-mentioned threshold) or more, it is desirable to determine that the area corresponds to a person. For example, in the illustrated example, it is determined that the regions P11 to P15 correspond to people, and the regions N12 and N13 are obtained continuously only from two edge images, and thus are removed as disturbances.

  By the way, when tracking and matching areas for multiple edge images, the area corresponding to the person is hidden behind the shadow of the pillar, etc., and the area cannot be associated, or the person does not move at all And may be removed together. In the example shown in FIG. 6, a state in which an area corresponding to a person cannot be detected in two edge images E (4) and E (7) out of seven edge images E (1) to E (7). Show.

  In this case, it is possible to cope with the same processing as when a region having a non-constant shape such as light is generated. That is, for each area associated in the time-series edge image, the distribution comparison processing unit 33 compares the area with the reference data, thereby counting the number of edge images from which the area corresponding to the person is obtained. When the number of sheets is equal to or greater than a predetermined threshold within a predetermined time, the area is determined as an area corresponding to a person.

  For example, in the example shown in FIG. 6, areas associated with seven edge images E (1) to E (7) in the edge images E (1) to (3), E (5), and E (6). P11 to P13, P15, and P16 exist, but there is no area to be associated in the two edge images E (4) and E (7). However, within the time period for obtaining five edge images (the above-mentioned predetermined time period), the number of edge images that are associated with each other and that correspond to a person is two (the above-mentioned threshold value). When it is above, by determining that the area corresponds to a person, the areas P11 to P13 can be determined to correspond to a person, and the areas P15 and P16 can be determined to correspond to a person. . Here, when an area corresponding to a person is obtained using reference data in two or more edge images among five edge images, the area is determined to be an area corresponding to a person. Even if three of the five edge images are included in the image in which the region corresponding to the person is not detected by determining the similarity to the data, the presence of the person can be detected. Other configurations and operations are the same as those of the first embodiment.

(Embodiment 3)
In the present embodiment, in addition to the above-described processing, conditions relating to the camera 11 are set, thereby facilitating extraction of a region corresponding to a person.

  Now, as shown in FIG. 7, it is assumed that the camera 11 is installed at a position of height h so that the depression angle of the optical axis is θ. The angle of view (viewing angle) of the camera 11 is φ. Here, when considering an orthogonal coordinate system with the floor F immediately below the center of the optical system provided in the camera 11 as the origin, the coordinates of the center of the optical system of the camera 11 are (0, h), and the floor F The visual field limit positions L1 and L2 are L1 (h / tan (θ + φ / 2), 0) and L2 (h / tan (θ−φ / 2), 0), respectively. When the floor surface F between the limit positions L1 and L2 of the visual field is equally divided into four sections, and the angle at which each section is viewed from the camera 11 is obtained, the portion near the camera 11 has a larger angle, and the angle increases as the distance from the camera 11 increases. Becomes smaller. That is, even if there is no change in the size of the imaged object, the apparent size in the image becomes larger as the part is closer to the camera.

The lengths of the four sections on the floor F shown in FIG. 7 are equal to h {1 / tan (θ−φ / 2) −1 / tan (θ + φ / 2)} / 4. The coordinates of the other end point of the section having the limit position L1 of the visual field as one end point are (a, 0), and the coordinates of the other end point of the section having the limit position L2 of the visual field as one end point are (b, 0). Then, the following relationship is established in the ratio of the arc lengths Sa and Sb that allow for both sections.
Sa: Sb = (θ + φ / 2) −tan ⁻¹ (h / a): tan ⁻¹ (h / b) − (θ−φ / 2)
It is assumed that the camera 11 looks down from above, and θ + φ / 2 ≦ 90 ° is established. Here, the ratio of the lengths Sa and Sb is the ratio of the lengths of the arcs that allow the respective sections to be estimated, but can be approximated to the ratio of the lengths of the sections of the floor surface F projected onto the imaging surface of the camera 11. (Shown in an approximate state in FIG. 7), each region of the field of view of the camera 11 is divided by the ratio of each section on the imaging surface of the camera 11 obtained as described above. In other words, the length of each section of the floor surface F projected onto the imaging surface of the camera 11 corresponds to the size of the image of the person standing on the floor surface F in the imaging surface. The field of view of the camera 11 is divided into a plurality of sections according to the above dimensions. For example, the vertical direction in the visual field of the camera 11 is divided into four, and the visual field is divided by the ratio of each section obtained as described above. That is, the area formed below the area formed above the visual field is wider. Further, the square area is formed by applying the vertical length in the divided visual field to the horizontal direction. FIG. 8A shows an example of dividing the rectangular region D2 formed by dividing the field of view by this method. In the illustrated example, a region having a certain width is set in the field of view regardless of the distance from the camera 11 in the horizontal direction. However, if the same division method as in the vertical direction is applied to the horizontal direction, The lateral width in the center is wider than the lateral width.

  In the present embodiment, the field of view is divided into a plurality of regions D2 as described above, and the moving region extracting means 2 can select whether or not to monitor each region D2. In FIG. 8B, a hatched area D2 indicates an invalid area where monitoring is not performed, and a hatched area D2 indicates an effective area. Here, in order to designate the region D2, a storage table is provided in the moving region extraction means 2, a range of pixels for each region D2 is defined in the storage table, and from the moving region extraction means 2 for each region D2. It is possible to select whether or not to perform subsequent processing. Whether or not to monitor each area D2, that is, when designating whether the area D2 is valid or invalid, designates the area D2 with a code assigned to each area or displays the area on the screen and displays the area D2 with a pointing device. It may be specified, or the specified area D2 may be specified as invalid by a switch operation. The means realized by the program having the function of designating validity / invalidity of the area D2 in this way is referred to as monitoring area setting means.

  In determining the region D2, in addition to geometrically determining by the simple method as described above, a simulation is performed in consideration of the aberration of the optical system, and the region D2 is strictly determined so that the dimensions in the real space are equal. You may divide into.

  As described above, when the region D2 is determined geometrically, it is necessary to give the depression angle of the camera 11, so an angle scale is provided in the tilt mechanism for adjusting the orientation of the camera 11, and the camera 11 What is necessary is just to give the value which looked at the angle scale when adjusting direction as a depression angle by manual input. When automating the depression angle input, an angle sensor such as a rotary encoder may be arranged in the tilt mechanism so that the output of the angle sensor is given as the depression angle.

  The height position of the camera 11 is manually input using a numerical key or a thumbwheel switch, or automatically by using a tape-type contact distance sensor or an optical non-contact distance sensor. Give height dimension.

  In this embodiment, the monitoring area setting means divides the field of view of the camera 11 into a plurality of areas D2 so that the dimensions on the floor surface F are equal, and designates whether or not monitoring is performed for each area D2. If only the region D2 that is likely to cause a disturbance such as a window is disabled, noise can be removed without affecting the region corresponding to the person, and the region corresponding to the person can be accurately detected. be able to.

  In addition, if the reference data according to the depression angle of the camera 11 is set, the detection accuracy of the area corresponding to the person can be increased by using appropriate reference data according to the depression angle of the camera 11. Other configurations and operations are the same as those of the first embodiment.

(Embodiment 4)
In the third embodiment, the field of view of the camera 11 is divided into a plurality of rectangular regions D2, but in this embodiment, a region for monitoring a person (effective region) and a region that is not monitored according to an object present in the field of view. (Invalid area). In other words, an arbitrarily shaped monitoring area is defined within the field of view of the camera 11.

  As described in the first embodiment, the moving area extracting means 2 and the area analyzing means 3 are realized by a computer, and the function of defining the monitoring area is also realized by a program executed by the computer. Therefore, the human body detection device of the present embodiment has a monitoring mode for monitoring a moving object (person) and an area setting mode for defining a monitoring area. The switching between the monitoring mode and the area setting mode and the instruction to start the setting in the area setting mode are performed using an input unit such as a switch.

  If the computer includes a display device and can use a pointing device such as a mouse, a monitoring area is set on the screen of the display device. Now, as shown in FIG. 9A, it is assumed that there is a part that is likely to generate a disturbance such as the window W or a part that does not need to detect a person in the field of view captured by the camera 11. In this case, an image (grayscale image or color image) Ib captured by the camera 11 is displayed on the display device as shown in FIG. 9A. Therefore, the image is captured by the camera 11 as shown in FIG. 9B. The valid area D3 and the invalid area D4 are set while confirming the position in the image. That is, the image Ib picked up by the camera 11 is displayed on the display device, and the user sets the boundary line Lb between the effective area D3 and the ineffective area D4 using a pointing device. The boundary line Lb may be approximated by a polygonal line at multiple points T1 to T9. In the example shown in FIG. 9B, the inside of the boundary line Lb is set as the effective area D3, and the outside of the boundary line Lb is set to the invalid area D4 where the window W exists. It is said. The means realized by the program having the function of extracting the effective area D3 as the monitoring area in this way is referred to as monitoring area setting means.

  As described above, since the monitoring area setting means designates the effective area D3 to be monitored and the invalid area D4 that is not to be monitored in the image Ib, an area that is highly likely to cause disturbance in the field of view of the camera 11 is selected. The invalid area D4 can be excluded, and as a result, an area corresponding to a person can be detected with high accuracy.

  By the way, as the monitoring area setting means, instead of designating the boundary line Lb between the effective area D3 and the invalid area D4 on the screen of the display device in the area setting mode, The effective area D3 and the invalid area D4 may be separated by a person actually moving. When this method is adopted, it is necessary to track the position where the person has moved within the field of view of the camera 11, so a technique is adopted in which a person carries a light source and tracks the position of the light source. A light source that emits light including a specific wavelength is used so that the light source can be distinguished from other external light, and the camera 11 selectively images only light of a specific wavelength using a filter.

  As described above, the camera 11 is provided with a filter that transmits only light of a specific wavelength so that the camera 11 has sensitivity only to light of a specific wavelength. The filter may be manually attached to the camera 11. However, if the camera 11 is provided with a mechanism for attaching / detaching the filter to automate the attachment of the filter, the filter can be attached / detached without work at a high place. When near-infrared light is used as light of a specific wavelength, a CCD image sensor or CMOS image sensor having sensitivity to near-infrared is used as the image sensor of the camera 11, and a filter that blocks visible light and transmits near-infrared is used for the camera 11. Attach to.

  On the other hand, as a light source carried by a person moving along the boundary line Lb, a dedicated light source that emits near infrared light can be used, or a remote control transmitter that uses near infrared light as a transmission medium can be substituted. Here, the remote control transmitter uses one having a flashing cycle shorter than the time interval between frames of an image captured by the camera 11.

  The work procedure for setting the boundary line Lb between the effective area D3 and the invalid area D4 is summarized as follows. That is, after first switching to the region setting mode, the filter is manually mounted on the camera 11 or the filter is automatically mounted on the camera 11 by switching to the region setting mode. After the filter is attached, the computer is instructed to start and end the setting operation of the boundary line Lb. The setting work can be instructed by using a keyboard or pointing device when using a computer equipped with a keyboard or pointing device. When a dedicated device is configured using a microcomputer, the setting work is instructed to start or end. A push button switch or the like may be provided.

  When the setting operation is started, the inside of the visual field is imaged at a predetermined time interval by the camera 11, and the captured image Ib is stored in the image memory 21. Here, the camera 11 captures images at the same time interval as in the monitoring mode, turns on the light source at a desired position, and links only the lighting of the light source and the storage of the image Ib, thereby only the image Ib when the light source is turned on. You may make it preserve | save in the image memory 21. FIG.

  On the other hand, the person carrying the light source moves along the boundary line Lb between the effective area D3 and the invalid area D4. At this time, if the image Ib picked up by the camera 11 is displayed in a place where the person carrying the light source can visually recognize the position, it is possible to set a more appropriate boundary line Lb. . In particular, when the light source is held by hand, it moves from front to back and left and right with respect to the position of the person's foot, and the height position from the floor also changes. Even if it passes, there may be a deviation from the boundary line Lb to be set within the field of view of the camera 11. Therefore, it is possible to set a desired boundary line Lb by adjusting the position of the light source while checking the image Ib.

  As described above, saving of all images in which the light source passes over the boundary line Lb (here, a plurality of images at positions corresponding to the points T1 to T9 shown in FIG. 9B) is completed. When the end is instructed later, the position where the density value (luminance value) is maximized is extracted for each stored image Ib. That is, in each image Ib, since the brightness | luminance of the position of a light source is considered to be the maximum, the position of the light source in each image Ib is extracted. Here, in order not to extract noise other than the light source in the image Ib, not only the position where the density value is maximized but also an appropriate image filter may be used together. In this way, the position where the density value extracted from each image Ib is maximized is connected in time order, and the closed region is formed by connecting the positions obtained from the first image Ib and the last image Ib in time series. The closed region can be designated as the valid region D3 or the invalid region D4 (designated as the valid region D3 in the example). The effective area D3 set in this way can be finely adjusted using an image displayed on the display device.

  In the above example, only one closed region is formed in the field of view of the camera 11, but a plurality of closed regions may be set. In that case, the end of the closed region can be instructed for each closed region setting, and the end of the region setting can be instructed after the setting of all the closed regions is completed. Other configurations and operations are the same as those of the first embodiment.

(Embodiment 5)
In this embodiment, when detecting a person, the detection accuracy is improved by using not only an image captured by the camera 11 but also a hot-wire human sensor that detects a heat ray emitted from the person. The human body detection apparatus is illustrated.

  That is, as shown in FIG. 10, a heat sensing means 4 having a pyroelectric infrared sensor (hereinafter abbreviated as “pyroelectric sensor”) 41 as a human sensor is provided. The heat sensing means 4 is configured to generate a pulse signal having a certain time width when a thermal change is detected by the pyroelectric sensor 41. The pyroelectric sensor 41 is a differential sensor, and outputs a voltage signal corresponding to the amount of change in the received heat ray. By setting an appropriate threshold for this voltage signal, the pyroelectric sensor 41 outputs a pulse signal from the heat sensing means 4. It becomes possible to output. The range (light receiving area) in which the pyroelectric sensor 41 receives heat rays is set by a combination with an optical element such as a lens or a mirror, and further, the optical element causes uneven sensitivity in the light receiving area. The heat dose received by the pyroelectric sensor 41 is changed with the movement of the heat source (such as a person).

  The light receiving area of the pyroelectric sensor 41 is substantially coincident with the field of view (or monitoring area) of the camera 11 and changes as shown in FIG. 11B occur within the field of view of the camera 11 (light receiving area of the pyroelectric sensor 41). Then, the heat sensing means 4 outputs a pulse signal as shown in FIG. In FIG. 11, (a) is a change in heat dose caused by a sudden change of sunlight or a swing of the curtain R at the window when no person is present in the field of view. One pulse signal is output from the heat sensing means 4. The output case is shown. Further, (b) in FIG. 11 shows a state where a person M appears in the field of view, and (c) through (e) in FIG. 11 show a state where the person is moving or slightly moving (a state where the limb moves). Is shown. The pulse signal shown in FIG. 11A is repeatedly generated while the person M exists in the field of view as shown in FIGS. In other words, the change in heat dose due to disturbance occurs only once, and the change in heat dose due to human detection often occurs intermittently.

  Therefore, a thermal change analysis means 5 for monitoring the pulse signal output from the heat sensing means 4 is provided, and in the time-series thermal change signal analysis unit 51 provided in the thermal change analysis means 5, a certain detection period Td1 (FIG. 11 ( The number of occurrences of the pulse signal for each a) is counted, and it is determined that there is a person in the field of view when the number of occurrences of the pulse signal is equal to or greater than a predetermined threshold. That is, the time-series heat change signal analysis unit 51 is provided as an analysis unit that determines the presence / absence of a person from the output of a human sensor including a pyroelectric sensor 41 provided in the heat sensing unit 4.

  Further, when the thermal change analyzing means 5 determines that the number of pulse signals generated is equal to or greater than a predetermined threshold and that there is a person in the field of view, a certain confirmation period Td2 (detection period Td1 and confirmation period Td2) The presence / absence of a pulse signal in (which may be equal to Td2) is confirmed, and when the generation of one or more pulse signals is confirmed in the confirmation period Td2, the determination result that a person exists in the visual field is maintained. In addition, if no pulse signal is detected in the confirmation period Td2, the determination result that a person exists in the field of view is cancelled.

  The determination result of the thermal change analysis means 5 is input to the integrated calculation unit 6 together with the determination result of the region analysis means 3, and the integrated calculation unit 6 determines whether a person exists in the field of view (or monitoring area) based on the determination results of both. Confirm whether or not. In the integrated arithmetic unit 6, it is the simplest method to obtain the logical product of the judgment results of both, and a person exists in the field of view (or monitoring area) in both the area analysis means 3 and the thermal change analysis means 5. If the result that the person exists in the field of view (or the monitoring area) is output also in the integrated calculation unit 6 when the determination result is obtained, the determination result is more than the case where the presence of the person is determined by only one side. The reliability will be higher. That is, it is possible to determine the position of a person in a specific image by the output of the area analysis unit 3 when the integrated calculation unit 6 obtains a determination result that a person exists in the visual field (or monitoring area). Become. In addition, when intrusion monitoring is performed using the contents captured by the camera 11, it is possible to reliably detect the intrusion of a person into the field of view (or the monitoring area), and prevent false reports.

  Here, only one of the monitoring of the person using the imaging unit 1 and the monitoring of the person using the heat sensing unit 4 is always performed, and when the possibility of the presence of any moving object is detected during the monitoring, This detection operation may be started. For example, the heat sensing means 4 is always operated, and when the heat sensing means 4 generates even one pulse signal, the imaging means 1 starts imaging, or the imaging means 1 is always operated, What is necessary is just to start a human detection by the heat sensing means 4 when the number of pixels that can be regarded as a moving object in the candidate image obtained from the extraction means 2 is equal to or greater than a predetermined threshold. As described above, by operating only one of the image pickup means 1 and the heat sensing means 4 at all times, it is possible to reduce power consumption as compared with the case where both are always operated. As described above, the other operation is started before the possibility of the presence of a person is determined based on the output of one of the imaging unit 1 and the heat sensing unit 4. This is because waiting until sex is judged delays the time until the possibility of the existence of a person is judged.

  In the above-described example, the integrated arithmetic unit 6 employs the logical product of the outputs of the region analyzing unit 3 and the thermal change analyzing unit 5, but the integrated arithmetic unit 6 may use a logical sum. . In other words, if it is determined that there is a possibility that one of the region analysis means 3 and the heat change analysis means 5 exists, a result that a person exists is output. When this configuration is adopted as the integrated calculation unit 6, the possibility of false alarms due to the intrusion of a person into the visual field (or monitoring area) is reduced. For example, when an image captured by the camera 11 is recorded on a recording device such as a video recorder in order to perform intrusion monitoring, the presence of a person can be detected by the output of either the imaging means 1 or the heat sensing means 4. It is possible to record all images when it is determined that there is a possibility, and to record images for all periods in which a person may exist in the field of view without always operating the recording device become. That is, it is possible to reduce the storage capacity as compared with the case where the recording apparatus is always operated while recording all images when there is a possibility that a person exists in the visual field of the camera 11.

  As a technique for storing an image captured by the camera 11 in a recording device, a technique for storing an image at regular intervals and a technique for storing an image during a period in which a person is detected by a separately provided sensor are known. However, since the former technique saves images regardless of the presence or absence of people, there is a possibility that necessary images cannot be obtained or useless images may be taken. However, since the detection is generally performed by one type of sensor, there is a possibility that a false alarm may occur and a necessary image may not be obtained. On the other hand, in the configuration of the present embodiment, an image can be reliably stored during a period in which there is a possibility that a person exists, and all necessary images are stored. Other configurations and operations are the same as those of the first embodiment. In this embodiment, the pyroelectric sensor 41 is used as a human sensor. However, a photoelectric sensor that performs infrared light transmission and reception, an ultrasonic sensor that transmits and receives ultrasonic waves, and the like can be used as the human sensor. is there. In this case, the output of the human sensor is given to an analysis unit corresponding to the thermal change analysis unit to determine the presence or absence of a person in the field of view (or monitoring area).

(Embodiment 6)
In the present embodiment, as shown in FIG. 12, a storage memory 7 is added to the configuration of the first embodiment shown in FIG. The storage memory 7 is provided for the purpose of recording and storing an image obtained over a predetermined period. Further, a path for recording the gray image from the A / D converter 12 provided in the image pickup means 1 in the image memory 21 and a path for transferring the image from the image memory 21 to the storage memory 7 are illustrated. .

  In general, when an image is recorded and stored for this type of purpose, since the image of the entire field of view is stored for the image (grayscale image or color image) captured by the camera 11, the storage capacity is large like a video recorder. Need a recording device.

  In the present embodiment, focusing on the fact that it is only necessary to obtain an image of a region corresponding to a moving object (person) in the image for purposes such as intrusion monitoring, the region analysis means 3 may have a human presence. Only the detected area is cut out from the image captured by the camera 11 and stored in the storage memory 7, and the image stored in the image memory 21 can be transferred to the storage memory 7. The area to be cut out may be an area that can be considered as a moving object Ob as shown in FIG. 2D. However, if the area is within the circumscribed rectangle D1 shown in FIG. Since the area to be cut out can be specified simply by specifying the coordinates, the processing is simplified. In addition, it is desirable that an image stored in the storage memory 7 is associated with a sign such as an imaging date.

  By adopting such a configuration, it becomes possible to store a required image with a significantly smaller storage capacity compared to the conventional configuration. As a result, a relatively small capacity such as the storage memory 7 can be stored. Images can be stored using a recording device. Since the image memory 21 used for the moving area extraction means 2 is a working memory, a volatile memory may be used. Since the storage memory 7 is a storage, a non-volatile memory (flash memory or the like) is used. If a removable memory is used as the storage memory 7, the contents of the storage memory 7 can be read out by another computer.

  By the way, since the frequency distribution of the direction code in the edge image is used to evaluate whether or not the moving object is a person in the image as described above, the image for evaluating the presence or absence of the moving object has a high resolution. Is not required. However, if used for intrusion monitoring, the image stored in the storage memory 7 is required to have a resolution that can identify a face or the like. Therefore, the camera 11 captures an image with a relatively high resolution, temporarily stores this image in the image memory 21, and performs a process of evaluating whether or not the moving object is a person with a low resolution. Therefore, it is desirable to speed up the processing and to read out a high-resolution image from the image memory 21 and store it in the storage memory 7 for an area recognized as having a high possibility that the moving object is a person. By this process, it is possible to suppress an increase in the storage capacity of the storage memory 7 while satisfying the high speed of the process and the high resolution in the required area. Other configurations and functions are the same as those of the first embodiment.

(Embodiment 7)
As shown in FIG. 13, the present embodiment is provided with an image transmission unit 8 for transmitting a required image to another apparatus via a transmission path instead of the storage memory 7 in the configuration of the sixth embodiment. is there. The image transmitted by the image transmission unit 8 is the same as the image stored in the storage memory 7 in the sixth embodiment, and it is determined that the moving object is likely to be a person among the images captured by the camera 11. An image of only the specified area is cut out and transferred to another apparatus. The timing for transferring the image is a point in time when it is determined that the moving object is likely to be a person, and it is irregular. Therefore, it is necessary to add the imaging date and time for transfer.

  As described above, in the configuration of the present embodiment, only an image of a region considered to be a person is transferred to another device. Therefore, compared with a case where an image captured by the camera 11 is always transferred to another device, the traffic on the transmission path is reduced. It can be greatly reduced. Moreover, it is possible to transfer the image to the other device without reducing the resolution of the image in the area of interest. As a result, an image captured by the camera 11 can be transferred even if the device has a small data storage capacity such as a wireless portable terminal (such as a mobile phone having an image transmission / reception function) as another device.

  As the other device, in addition to the above-described wireless portable terminal, a monitor display device, a storage device for storing an image, a computer having a communication function, and the like can be used, and the image transmission unit 8 has communication specifications with the other device. It may be configured accordingly. For example, the image transferred from the image transmission unit 8 to another device may be an analog image or a digital image, and the transmission path may be wired or wireless. Further, as the processing in the image transmission unit 8, image thinning may be performed according to the specifications of other devices and transmission paths. In particular, when transferring a moving image to another apparatus, if an appropriate frame is extracted according to the communication speed of the transmission path or the processing speed for the image in the other apparatus, the image is not reduced without reducing the resolution. Can be transferred.

  The configuration of the present embodiment is particularly effective in intrusion monitoring. Instead of constantly monitoring the visual field (or monitoring area) and transferring the image to another apparatus, the image is only displayed during a period when it is determined that a person exists. Since the image to be transferred to the device and the image to be transferred is only an area where a person may exist, the traffic on the transmission path to other devices can be greatly reduced, and the cost required for communication can be reduced. In addition, the buffer capacity for transmission / reception can be reduced, and the resolution of the image is not lowered, so that the face of the intruder can be identified. Other configurations and functions are the same as those of the sixth embodiment.

(Embodiment 8)
Since each of the above-described embodiments can extract a region where a person may exist, the presence or absence of an intruder can be monitored using the field of view of the imaging unit 1 as a monitoring region. When used for such a purpose, as shown in FIG. 14, an image display means comprising a display device such as a CRT or a liquid crystal display in order to enable monitoring by an observer of the image taken by the image pickup means 1. 40 is provided. The image display means 40 displays an image based on the grayscale image stored in the image memory 21. The image output means 41 controls what kind of image is displayed on the image display means 40. Further, there is provided a detection signal output means 42 for outputting a detection signal when there is an area evaluated by the area analysis means 3 described above as an area corresponding to a person. The image output means 41 reads from the image memory 21 the area evaluated by the area analysis means 3 as a region corresponding to a person, and displays a partially enlarged image enlarged at an enlargement ratio in accordance with the screen size of the image display means 40. It is displayed on the screen of the display means 40.

  The enlargement ratio in accordance with the size of the screen of the image display means 40 means an enlargement ratio such that the height of the area corresponding to a person is about two thirds of the height of the screen of the image display means 40, for example. To do. However, this enlargement ratio is a guideline and may be set so that a necessary portion does not protrude from the screen of the image display means 40. Furthermore, the enlargement ratio changes according to the position of the intruder in the field of view of the image pickup means 1, and if the intruder's posture does not change, the enlargement ratio increases as the distance from the image pickup means 1 to the intruder increases. Become. However, if the distance range from the imaging means 1 is limited for the region for generating the partially enlarged image, the purpose of grasping the characteristics of the intruder can be achieved only by enlarging the intruder. The magnification rate may be constant. In this case, the enlargement ratio may be set as a specified value by a program, or the supervisor may set it by operating an operation unit (not shown). Depending on the position of the area corresponding to the person in the field of view of the image pickup means 1, reduction may be required instead of enlargement, but even in the case of reduction, the ratio to the original size will be referred to as the enlargement ratio. Since the aspect ratio of the area corresponding to the person is not constant and the aspect ratio of the screen of the image display means 40 is constant, only the height dimension of the area corresponding to the person is adjusted according to the size of the screen. Then, an area having a width dimension corresponding to the aspect ratio of the screen with respect to this height is extracted and used for the partially enlarged image.

  In the present embodiment, when the detection signal output unit 42 outputs a detection signal, the image output unit 41 starts tracking the region evaluated by the region analysis unit 3 as a region corresponding to a person. A partially enlarged image enlarged at an enlargement rate that matches the size of the screen is displayed on the image display means 40. After that, when the region evaluated as the region corresponding to the person is not detected by the region analysis unit 3, the display of the partially enlarged image is canceled and the entire image that is the entire visual field of the imaging unit 1 is displayed on the image display unit 40. Return to. That is, as shown in FIG. 15A, the entire image X1 is always displayed, and when a region Px corresponding to a person is detected, a partial enlargement including the region Px is performed as shown in FIG. 15B. The image X2 is displayed on the image display means 40.

  By the way, the partial enlarged image is displayed at an enlargement ratio that matches the size of the screen of the image display means 40, and the area corresponding to the partial enlarged image moves within the field of view of the imaging means 1 as the intruder moves. It is desirable to indicate to the monitor which portion of the enlarged image is in the field of view of the imaging means 1. Therefore, the image display means 40 is composed of two display devices, and the entire image that is the entire field of view of the imaging means 1 is displayed on one display device, and the partially enlarged image is displayed on the other display device. An operation unit (not shown) that can be operated by a monitor is provided using a display device, and the display state of the entire image and the display state of the partially enlarged image are switched by operation of the operation unit.

  When two display devices are used, although the cost is high, the entire image and the partially enlarged image can be viewed at the same time without performing a special operation, and the monitoring operation is facilitated. Further, if one display device is shared for displaying the entire image and the partially enlarged image, the operation of switching the images is required, but the cost can be reduced. Moreover, since the partially enlarged image is automatically displayed on the image display means 40 when an intruder enters the field of view of the imaging means 1, the image displayed on the image display means 40 is switched to the partially enlarged image. The characteristics of the intruder can be confirmed, and then the position of the intruder can be confirmed by switching to the whole image.

  In order to display the partial enlarged image and the whole image by the image display means 40, as shown in FIG. 15C, the image is displayed so that the whole image X1 is displayed on a part of the screen displaying the partial enlarged image X2. The output means 41 may be configured. That is, the entire image X1 is displayed by superimposition on the screen of the partial enlarged image X2. If this configuration is adopted, the invader's characteristics are confirmed on the screen using the partially enlarged image X2 while using one display device, and at the same time, the entire image X1 is used to enter anywhere in the field of view of the imaging means 1. It becomes possible to know if there is a person. That is, it becomes possible to provide monitoring work at a low cost as well as to facilitate the monitoring work.

  As described above, the entire image is displayed on the image display unit 40 at all times, and when the region analyzed by the region analysis unit 3 is detected as a region corresponding to a person, the partially enlarged image is enlarged to the size of the screen. Since it is displayed with the combined enlargement ratio, the position where the intruder exists may not be grasped immediately after the screen changes. Therefore, when the image output means 41 switches from the state in which the entire image is displayed on the screen of the image display means 40 to the partially enlarged image, as shown in FIG. 16, it matches the size of the screen of the image display means 40. It is desirable to control the enlargement rate of the partial enlarged image X2 to be gradually increased with time until the partial enlarged image X2 is displayed at the same enlargement rate. That is, when a region Px corresponding to a person is generated in the entire image X1, this region Px is displayed in the entire image X1 by superimposition, and the region Px is started from the position where the region Px was first displayed. Gradually increases the area occupied by the entire image X1 with time, and finally displays the partially enlarged image X2 having a desired enlargement ratio on the full screen. By this operation, the area Px expands not from the center of the screen but from the position where it was initially displayed. Therefore, the partial enlarged image X2 is displayed as if it was zoomed up from the entire image X1, and the intruder's part in the whole is displayed. It becomes easy to grasp the existing position. Needless to say, zooming in by digital signal processing is performed because the partial enlarged image X2 uses an image read from the image memory 21.

  When there are a plurality of regions that are evaluated to correspond to a person within the field of view of the imaging unit 1, that is, when there are a plurality of intruders, a plurality of partially enlarged images are generated. In such a case, the image output means 41 automatically switches each partial enlarged image to the screen of the image display means 40 at regular intervals and displays it in order, thereby displaying all the partial enlarged images on one display. Display on the device. The display order may be the raster scan order of the display device. That is, the coordinate position of the upper left corner of each area may be searched in the order of raster scan, and the partially enlarged images corresponding to the detected areas may be displayed in order.

  By automatically displaying a plurality of partially enlarged images in order, it is possible to confirm the characteristics of each of a plurality of intruders and easily identify whether or not there are suspicious persons among a plurality of persons. Since the region corresponding to the partially enlarged image moves with the movement of the intruder, the partially enlarged image corresponding to the position of the intruder at the time of displaying the partially enlarged image is displayed, and also in accordance with the partially enlarged image. It is desirable to display the entire image. When displaying multiple partially magnified images, zoom in the partially magnified image from the display state of the entire image, and return to the state in which the entire image is displayed after a certain time to zoom in on the partially magnified image in another area. It is also possible to repeat the above. However, if such a display is somewhat difficult to see, the partially enlarged images may be sequentially switched and a display that displays the entire image on a part of the screen may be selected.

  Furthermore, instead of sequentially displaying the partial enlarged images on the screen of the image display means 40 at regular intervals, the screen of the image display means 40 is displayed for the number of areas Px1 and Px2 corresponding to people as shown in FIG. The partial enlarged images X21 and X22 corresponding to the areas Px1 and Px2 may be displayed in the respective sections. That is, a partial screen is generated by dividing the screen of the image display means 40 by the number of intruders present in the field of view of the imaging means 1, and the partial enlarged images X21 and X22 are displayed on the partial screens, respectively. This configuration is effective when the number of intruders is relatively small, and it is desirable that the partially enlarged images are sequentially switched and displayed when there are a large number of people. In addition, the upper limit of the number of divisions of the screen of the image display means 40 is limited, a partial screen divided into the number of persons within the upper limit of the number of divisions is generated, and the upper limit for the number of persons exceeding the upper limit of the number of divisions. The screens divided by the number of divisions may be switched to display the number of people. If the screen of the image display means 40 is divided and a plurality of persons are displayed on one screen, the actions of a plurality of intruders can be listed in one screen, and the characteristics and actions of the plurality of intruders can be grasped at a time. can do.

  Depending on the purpose of use of the intruder, it may be necessary to track only one of the intruders instead of tracking all intruders when a plurality of intruders are detected. In such a case, only the intruder detected first in the field of view of the imaging means 1 may be tracked, and a partially enlarged image of only this intruder may be displayed. In addition, when the monitored image is stored for later use, an image only for a period from the start of displaying the partially enlarged image to the end of displaying the partially enlarged image on the image display means 40 is displayed. Since the purpose can be achieved by recording, the storage capacity of the storage medium can be reduced.

1 is a block diagram illustrating a first embodiment. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. FIG. 9 is an operation explanatory diagram of the second embodiment. FIG. 9 is an operation explanatory diagram of the second embodiment. FIG. 9 is an operation explanatory diagram of the second embodiment. FIG. 10 is an operation explanatory diagram of the third embodiment. FIG. 10 is an operation explanatory diagram of the third embodiment. FIG. 10 is an operation explanatory diagram of the fourth embodiment. FIG. 10 is a block diagram illustrating a fifth embodiment. It is operation | movement explanatory drawing same as the above. FIG. 10 is a block diagram illustrating a sixth embodiment. FIG. 10 is a block diagram illustrating a seventh embodiment. FIG. 10 is a block diagram illustrating an eighth embodiment. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pickup means 2 Moving area extraction means 3 Area analysis means 4 Thermal sensing means 5 Thermal change analysis means 6 Integrated calculation part 7 Storage memory 8 Image transmission part 21 Image memory 40 Image display means 41 Image output means 42 Detection signal output means

Claims (13)

A background is obtained by combining three or more time-series edge images using an image pickup means for picking up a predetermined field of view and an edge image that is a binary image obtained by extracting edges from each image picked up at different times by the image pickup means. A moving area extracting means for extracting an area corresponding to the moved object in the edge image at the time of interest by performing a logical operation to be removed, and a frequency distribution of the direction code of the pixel on the edge for the area extracted by the moving area extracting means And using the similarity between the obtained frequency distribution and the reference data that is the frequency distribution of the direction code of the pixel on the edge obtained in advance for the human edge image, the region extracted by the moving region extracting means A human body detection apparatus using an image, comprising: region analysis means for evaluating whether or not the region corresponds to the region.   The frequency distribution is normalized by the total number of pixels on each edge of interest, and the evaluation value of the similarity uses a sum of squares of the frequency difference for each direction code. 2. The human body detection apparatus using an image according to claim 1, wherein an area extracted by the moving area extraction unit is determined as an area corresponding to a person when the value is equal to or less than a predetermined threshold.   The area analysis means sets a normal range based on an upper limit value and a lower limit value for the frequency of each direction code of the frequency distribution obtained for the area extracted by the moving area extraction means, and the frequency deviates from the normal range. 3. The human body detection apparatus using an image according to claim 1, wherein a region where a frequency distribution including a code is obtained is regarded as a disturbance other than a person.   The area analysis means obtains a frequency distribution normalized by the total number of pixels on each edge for the direction code of the pixels on the edge for each area extracted by the moving area extraction means for a plurality of time-series edge images, Next, by evaluating the similarity of the frequency distribution between each pair of adjacent edge images in time series, areas corresponding to the same object are associated between different edge images, and the reference data The human body detection apparatus using an image according to any one of claims 1 to 3, wherein the evaluation is performed using the similarity.   The field of view of the imaging unit is divided into a plurality of regions at a ratio according to the size of the image of the person existing in the field of view of the imaging unit in each part of the imaging surface, and an effective region and an invalid region are provided for each region. 5. The monitoring area setting means having a function of designating the distinction between the effective area and the effective area as a monitoring area for detecting the presence or absence of a person is added. Human body detection device using images of   A region setting mode for setting a monitoring region for detecting the presence or absence of a person can be selected, and in the region setting mode, the imaging means has a light of a specific wavelength from a light source moved along the boundary line of the monitoring region within the field of view. Only one of the inside and the outside of the closed area obtained by linking the position where the density value becomes the maximum in a plurality of images obtained in time series from the imaging means in the area setting mode in the area setting mode is the effective area. 5. An image according to any one of claims 1 to 4, further comprising monitoring area setting means for setting the other as an invalid area and the effective area as a monitoring area for detecting the presence or absence of a person. The human body detection device used.   An image memory for temporarily storing an image for each image picked up by the image pickup device, and when an area corresponding to a person is extracted by the area analysis unit, the image stored in the image memory is cut out and saved for the area. A human body detection device using an image according to any one of claims 1 to 6, further comprising a storage memory.   An image memory for temporarily storing an image for each image picked up by the image pickup device, and when an area corresponding to a person is extracted by the area analysis unit, the image stored in the image memory for the area is cut out to the other device The human body detection apparatus using the image of any one of Claims 1 thru | or 6 provided with the image transmission part forwarded to.   A detection signal output unit that outputs a detection signal when there is an area that is evaluated as a region corresponding to a person in the region analysis unit, and the region analysis unit corresponds to a person when a detection signal is output from the detection signal output unit And an image output means for starting the tracking of the evaluated area and the evaluated area and displaying a partially enlarged image obtained by enlarging the area as compared with other areas on the screen of the image display means. A human body detection device using the image according to any one of claims 1 to 8.   The image output means displays the partial enlarged image on the screen of the image display means at an enlargement ratio that matches the size of the screen of the image display means, and the field of view of the imaging means on a part of the screen of the image display means The whole body image which is the whole is displayed, The human body detection apparatus using the image of Claim 9 characterized by the above-mentioned.   The image output means displays the partially enlarged image at an enlargement ratio in accordance with the screen size of the image display means from a state in which the entire image that is the entire field of view of the imaging means is displayed on the screen of the image display means. 10. The image according to claim 9, wherein the magnification of the partially magnified image is gradually increased as time elapses, starting from the region corresponding to the person and the region initially evaluated by the region analysis means until the state. The human body detection device used.   When there are a plurality of regions evaluated as regions corresponding to the person in the region analysis unit, the image output unit displays the partial enlarged image corresponding to each region on the screen of the image display unit at regular intervals. The human body detection device using an image according to claim 9, wherein the display is switched and displayed.   When there are a plurality of regions that are evaluated as regions corresponding to a person in the region analysis unit, the image output unit divides the screen of the image display unit into sections corresponding to the number of regions and corresponds to each region. The human body detection device using an image according to claim 9, wherein the partial enlarged image to be displayed is displayed in each section.

Images