JP2012033142A - Number-of-person measuring device, number-of-person measuring method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a number-of-person measuring device for calculating the number of persons passing an area displayed in a video by estimating the number of persons displayed in the video and the velocity as a group.SOLUTION: A number-of-passing-person measuring device for inputting a motion vector of each pixel in an image included in a video and a foreground area in the image and measuring moving persons, comprises: projection means for projecting the motion vectors in the foreground area to one-dimension to generate a projection image; and extremal point extraction means for extracting extremal points among pixels included in the projection image generated by the projection means to detect the moving persons.

Description

  The present invention relates to a person counting device, a person counting method, and a program for measuring the number of people and the number of people passing through an image processed by an image processing device.

  A method is known in which the number of people passing through a region shown in a video is measured by detecting and tracking people from a video composed of time-sequential frames (for example, Non-Patent Document 1). . Also, a method of detecting retrograde by observing the movement of a moving area from a video composed of time-sequential frames (Non-Patent Document 2) is known.

Oliver Sidla, Yuriy Lypetskyy, Norbert Brandle, and Stefan Seer, “Pedestrian detection and tracking for counting applications in crowded situations”, Proc.IEEE International Conference on Video and Signal Based Surveillance (AVSS06), p. 70, 2006 Hiroyuki Arai, Takayuki Anno, Midori Mizukami, Miki Haseyama, “Method of Detecting Retrogrades from Video”, IEICE Technical Report, pp. 29-34, February 2006

  However, in the passing person counting method in Non-Patent Document 1 described above, the accuracy of counting the number of people is greatly affected by the tracking performance, and it may be difficult to obtain sufficient number of people counting accuracy particularly in a crowded situation. Further, the retrograde person detection method in Non-Patent Document 2 described above does not necessarily observe the movement of all the people shown in the video, and therefore cannot measure the number of people passing by.

  In the present invention, in view of the problems of the conventional methods as described above, the number measuring device for determining the number of people passing through the area shown in the video by estimating the number of people shown in the video and the speed as a group. The purpose is to provide a method and program for counting people.

  The present invention is a person counting device for inputting a motion vector of each pixel in an image included in an image and a foreground area in the image and measuring a moving person, and the motion vector in the foreground area is 1 Projection means for projecting to a dimension to generate a projected image; and extreme point extraction means for detecting a moving person by extracting extreme points from pixels included in the projected image generated by the projection means. It is characterized by that.

  The present invention is characterized in that the extreme point extraction means extracts extreme points from the pixels included in the projection image generated by the projection means and detects the position of the moving person on the projection axis.

  The present invention is based on an image corresponding to a person included in the image, by mapping from the coordinates in the image to pixels on the real space coordinates indicating the position of the person in the real space. It has a real space mapping means for generating an image, and the projection processing means generates a projected image from the real space image generated by the real space mapping means.

  In the present invention, the real space mapping means adds the predetermined value to a pixel corresponding to a range in a two-dimensional space that can be a standing position of a human image including the target pixel on the projected image, and performs the mapping. By performing, the real space image is generated.

  In the present invention, the real space mapping means adds a predetermined value to a pixel corresponding to a range in a logarithmic space that can be a standing position of a human image including a target pixel on the projected image, and performs a first mapping. The real space image is generated by performing processing and mapping the first mapping processing result on a two-dimensional space.

  The present invention is based on cumulative image generation means for generating an accumulated image by accumulating the extreme value image extracted by the extreme value point extracting means at every elapsed time, and on the accumulated image generated by the accumulated image generating means. The apparatus has a number calculating means for measuring the number of moving persons.

  The present invention is based on cumulative image generation means for generating an accumulated image by accumulating the extreme value image extracted by the extreme value point extracting means at every elapsed time, and on the accumulated image generated by the accumulated image generating means. And representative speed calculating means for calculating the representative moving speed of the moving person from the moving speed of each extreme point.

  The present invention provides a number calculation unit for detecting the number of people included in the image, a number of people detected by the number of people calculation unit, and a representative speed calculated by the representative speed calculation unit within a predetermined time. Passing number calculation means for calculating the number of persons passing through the area.

  The present invention detects a foreground portion from an input video and generates a foreground image, and a range in a two-dimensional space that can be a standing position of a human image including a target pixel on the foreground image. By adding the predetermined value to the corresponding pixel and performing the mapping, the real space image generating means for generating the real space image and the sum of the pixel values of the real space image are taken into the real space region. An in-region number calculating means for calculating the number of existing persons is provided.

  The present invention corresponds to a foreground image generation means for detecting a foreground portion from an input video and generating a foreground image, and a range in a logarithmic space that can be a standing position of a human image including a target pixel on the foreground image. A real space for generating the real space image by adding a predetermined value to the pixel to be processed and performing a first mapping process, and performing a second mapping process on a two-dimensional space from the first mapping process result It is characterized by comprising image generation means and area area number calculation means for calculating the number of persons existing in the real space area by taking the sum of the pixel values of the real space image.

  The present invention uses a computer, which is a person counting device that inputs a motion vector of each pixel in an image included in an image and a foreground region in the image, and measures a moving person, and the projection means of the computer Projecting a motion vector in the foreground region in a one-dimensional manner to generate a projected image, and an extreme point extraction unit of the computer is an extreme value among pixels included in the projected image generated by the projection unit Detecting a moving person by extracting points.

  The present invention is characterized by causing a computer to function each means of the people counting device according to any one of claims 1 to 10.

  As described above, according to the present invention, it is possible to measure the number of passing people without individually tracking each individual person shown in the video. This makes it possible to measure the number of people passing through even in situations where it is difficult to track individuals. In addition, by extracting only people who are moving in the direction of observing people flow, it is possible to measure the number of people passing in each direction even when people move in various directions.

It is a schematic block diagram which shows the structure of the passing number measurement apparatus by one Embodiment of this invention. 3 is a functional block diagram showing functions of the passing number measuring device 1. FIG. It is a figure explaining the process which produces | generates a projection image. It is a figure explaining the process which produces | generates a cumulative image. 4 is a flowchart for explaining the operation of the passing number measuring device 1. It is a figure which shows the example which calculated the motion vector. It is a functional block diagram which shows the structure of the passing number measuring device 1 in 2nd Embodiment. It is a functional block diagram which shows the structure of the passage number measuring device 1 in 3rd Embodiment. It is a functional block diagram which shows the structure of the passage number measuring device 1 in 4th Embodiment. It is a functional block diagram which shows the structure of the passing number measuring device 1 in 5th Embodiment. 4 is a flowchart for explaining the operation of the passing number measuring device 1. It is a figure explaining the method of derivation | leading-out of the range of mapping. It is a figure explaining the method of derivation | leading-out of the range of mapping. It is a figure explaining an example of the method of implement | achieving mapping. It is a figure explaining an example of the method of implement | achieving mapping. It is a figure which shows an example which mapped to the real space image. It is a figure explaining the process which produces | generates a projection image. It is a figure explaining the process which produces | generates a cumulative image. It is a functional block diagram which shows the structure of the passage number measuring device 1 in 6th Embodiment. It is a functional block diagram which shows the structure of the passing number measurement apparatus 1 in 7th Embodiment. It is a functional block diagram which shows the structure of the passing number measuring device 1 in 8th Embodiment. It is a functional block diagram which shows the structure of the passing number measuring device 1 in 9th Embodiment. It is a flowchart which shows the processing operation of the apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the passage number measuring device 1 in 10th Embodiment. It is explanatory drawing which shows the processing operation of the passage number measuring device 1 in 10th Embodiment.

<First Embodiment>
Hereinafter, a passing person counting device according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a passing person counting device according to an embodiment of the present invention. The passing person measuring device 1 is connected to a camera 2 which is an imaging device and a display device 3 such as a liquid crystal display device, and includes an input device 11, a ROM (read-only memory) 12, a CPU (central processing unit) 13, A person who has a random access memory (RAM) 14, an interface (I / F) 15, an external storage device 16, a recording medium driving device 17, and a recording medium 18, and passes through an area shown in an image captured by the camera 2. It is a computer that measures the number of

  In the passing person counting device 1, the input device 11 is an input device such as a mouse or a keyboard. The ROM 12 stores predetermined programs and data. The CPU 13 reads out and executes a program stored in the ROM 12 and controls each unit in the passing number measuring device 1. The RAM 14 is accessed by the CPU 13 and temporarily stores various data. The I / F 15 connects the camera 2 and the passing person counting device 1 and receives data output from the camera 2. The external storage device 16 is a storage device connected to the outside of the passing number measuring device 1, and is, for example, a hard disk. The recording medium driving device 17 drives the recording medium 18 according to instructions from the CPU 13 and stores various data in the recording medium 18. The recording medium 18 stores various data.

  The camera 2 is fixed at a predetermined position, images a target area where the number of passing people is measured, and outputs the imaging result to the passing number measuring device 1. The camera 2 inputs captured images (frame images) to the passing person counting device 1 in time series. For example, the input data is, for example, a still image series or a video stream. The display device 3 displays various data on the screen in accordance with instructions from the passing person counting device 1.

  Next, an example of an embodiment of the present invention will be further described. FIG. 2 is a functional block diagram showing the functions of the passing number measuring device 1. The motion vector extraction unit 101 inputs an image from the camera 2 and outputs a motion vector. The motion vector extraction unit 101 detects an optical flow (see, for example, Non-Patent Document 3) from the current frame of the video and the previous frame, and detects the motion vector and the optical from the current input image captured by the camera. The time t (number of frames) during which the flow detection is continued is calculated. For example, when u is from left to right and v is from top to bottom, this motion vector is represented by the pixel position (u, v) and the motion direction (U, V) of the optical flow detection start point. The

  Non-Patent Document 3; “Image Processing Standard Text Book”, Association for Promotion of Image Information Education, p. 79-280, 1997

  The foreground detection unit 102 inputs an image from the camera 2 and generates a foreground image in the current input image. Here, the foreground image is an image in which a moving object exists in the input image, that is, a point that is the foreground is 1, and a point that is not, that is, a point that is the background is 0. Various methods are known for detecting the foreground image, but any method may be applied. Examples of the foreground image detection method include Non-Patent Documents 4 and 5 below.

Non-Patent Document 4: Hanabe Sai, Toshikazu Wada, Takashi Matsuyama, “Background Difference Method Robust against Lighting Changes”, Information Processing Society of Japan Research Report (1998-CVIM115), Vol.1999, No.29, pp.17 -24, 1999.3
Non-Patent Document 5: Kedar A. Patwardhan, Guillermo Sapiro, Vassilios Morellas, “Robust Foreground Detection in Video Using Pixel Layers”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 4, April 2008

  The projection processing unit 103 receives the motion vector output from the motion vector extraction unit 101 and the foreground image output from the foreground detection unit 102, and generates and outputs a projected motion vector. The projection processing unit 130 sets the point where the direction of the motion vector is within a certain range and the absolute value is equal to or greater than a certain threshold and the value at the same coordinate of the foreground image is 1, and the other point is 0. A one-dimensional projection in which a coordinate image x is taken in a certain direction and the sum of the values in each pixel on the binary image having the same position in the x direction is taken, as shown in FIG. Generate an image.

  In FIG. 3, the horizontal direction is taken as the coordinate axis x, and the sum of the values of the pixels having the same position on the x axis is calculated. This calculation is performed at each position on the x-axis. Here, as an example, the case where the data indicated by the symbol (a) is obtained as the projected image is shown.

  Thus, in this embodiment, the process of obtaining the sum of the values of pixels having the same position in the x direction in the image is referred to as a projection process for the coordinate axis x. Here, when generating a binary image, a process such as a morphological operation for removing the influence of local false detection of a motion vector may be performed.

  Here, the direction of the coordinate axis x is the horizontal direction, but the present invention is not limited to this. In addition, for example, the x-axis may be taken in the direction of the human flow to be measured, such as the u direction when observing the human flow in the left-right direction and the v direction when observing the human flow in the vertical direction. In determining the range of the direction of the motion vector and the threshold value, the method is not limited here. For example, the following method can be considered.

(1) The range of the direction of the motion vector is a range in which the x-direction component of the vector faces the direction in which passage is desired to be measured. For example, when the x axis is set equal to the u axis and rightward passage is measured, the x component of the motion vector is in a positive range.
(2) The threshold value of the absolute value is set to a minimum value at which the influence of movement detected by a person other than the movement of a person or an object whose passage is to be grasped can be discarded.

  The extreme point extraction unit 104 receives the projected motion vector output from the projection processing unit 103, and calculates an extreme point image in which 1 is a point where the value of the projected image takes a maximum value and 0 is a point other than that. Output.

  The cumulative image generation unit 105 receives the extreme point image output from the extreme point extraction unit 104, and generates and outputs a cumulative image that is an image indicating the position of the extreme point. FIG. 4 is a diagram for explaining the accumulated image. In this figure, the cumulative image generation unit 105 generates a cumulative image based on the local maximum image generated by the extreme point extraction unit 104. Here, the extreme point image is stored in the memory in correspondence with the time t by the extreme point extraction unit 104 (FIGS. 4A and 4B). The cumulative image generation unit 105 generates a two-dimensional cumulative image (x, t) indicating the locus of the extreme points (FIG. 4C). Here, a two-dimensional cumulative image is generated by accumulating an extreme value image in which 1 is assigned to a pixel at a position on the coordinate axis x representing the maximum value and 0 is assigned to a pixel that is not, according to time t. In FIG. 4, the pixels having the maximum value and the pixels not having the maximum value are shown in different colors.

  Here, the size of the accumulated image in the x direction is generated so as to match the size of the extreme point image in the x direction. The size of the accumulated image in the t direction is a range in which a size sufficient to detect a locus candidate can be obtained at the time of the Hough transform described later, and is as small as possible.

  The representative speed calculation unit 106 receives the cumulative image output from the cumulative image generation unit 105, and generates and outputs a representative movement speed of people in the image. There are various ways of generating the representative moving speed, which will be described later.

  The in-screen number calculation unit 107 inputs an image from the camera 2, calculates the number of people on the screen, and outputs the calculation result. This in-screen number calculation unit 107 calculates the number of people who are moving in the direction of observing the human flow from the images captured by the camera 2 among the people shown in the current input image.

  The passing number calculating unit 108 calculates the passing number based on the representative moving speed calculated by the representative speed calculating unit 106 and the number of people shown in the image calculated by the in-screen number calculating unit 107.

  Next, the operation of the passing person measuring apparatus 1 having the above-described configuration will be described. FIG. 5 is a flowchart for explaining the operation of the passing number measuring device 1. First, when processing is started and image data is output from the camera 2, the motion vector extraction unit 101 detects an optical flow from the current frame and the previous frame output from the camera 2, A motion vector and a time t during which the optical flow is continuously detected are calculated from the current input image taken by the camera (step S1). Here, with respect to the image data from the camera 2, not the entire image but a part of the image may be extracted and processed.

  The method of calculating the optical flow is not limited here, but it is preferable to select a method in which the flow is stably generated depending on the scene. For example, a method based on luminance gradient or a method based on region matching can be considered, but a method based on luminance gradient is more effective when the degree of congestion is large, and a method based on region matching is more effective when the degree of congestion is small. It is. An example in which a motion vector is calculated from an actual video is shown in FIG. FIG. 6A shows an example in which a frame image at a certain moment of image data output from the camera 2 is extracted. In FIG. 6A, for example, an image is divided into 8 × 8 pixel blocks and obtained using a method based on region matching. When the absolute value of the horizontal motion vector in this frame is represented by the brightness of the pixel, it is as shown in FIG.

  On the other hand, the foreground detection unit 102 generates a foreground image in the current input image (step S2). When the motion vector and the foreground image are generated, the projection processing unit 103 has the direction of the motion vector within a certain range, the absolute value is equal to or greater than a certain threshold value, and the value at the same coordinate of the foreground image is 1. A one-dimensional projected image is generated by generating a binary image in which the point is 1 and the other points are 0 (step S3). FIG. 6C shows a projection image for the u axis generated from the motion vector shown in FIG.

  When the projected image is generated, the extreme point extraction unit 104 generates an extreme point image in which the point where the value of the projected image takes the maximum value is 1 and the point where it is not is 0 (step S4). Here, the number of maximum values indicates the number of persons in the image when a plurality of persons are not present at the same position in the x direction. The point at which the value of the projected image takes the maximum value corresponds to the human candidate position in the x direction. Here, before extracting the extreme value, a process such as applying a smoothing filter to the projected image may be performed.

  When the extreme value image is generated, the cumulative image generation unit 105 stores the extreme point image in the memory in correspondence with the time t, and a two-dimensional cumulative image (x, t) indicating the locus of the extreme point. Is generated (step S5). FIG. 6D is an example of an extreme point trajectory image generated from an actual video. The upper end of the image corresponds to FIG.

  When the accumulated image is generated, the representative speed calculation unit 106 obtains the representative moving speed of the people shown in the image from the accumulated image (step S6). There are various ways of obtaining the representative moving speed. For example, the following three methods (Method 1 to Method 3) are given as an example.

(Method 1)
First, θ-ρ Hough transform is performed on the accumulated image, and a point on the θ-ρ space where the pixel value (number of votes) is equal to or greater than a certain threshold is extracted (step S6-1-1). In determining the threshold value, the method is not limited here. For example, depending on how many extreme points are arranged on a straight line on the accumulated image, the straight line is regarded as the locus of the extreme points. To decide. In the cumulative image of FIG. 6D (for example, vertical 60 pixels × horizontal 80 pixels), when the Hough transform is performed with the step size of θ being 1 degree and the step size of ρ being 1 pixel, the pixel value threshold is about 10 to 15 Is preferred. This is a value experimentally determined from the viewpoint that there are few detection failures of straight line candidates corresponding to the locus of extreme points and false detection of straight line candidates not corresponding to the locus of extreme points. Note that the range of θ may be limited in the Hough transform in order to prevent erroneous detection of straight line candidates that are not the locus of extreme points and to shorten the calculation time. For example, when it is assumed that the moving speed of a person is 3 [pixel / frame] or less, 90 ° ≦ θ ≦ 109 ° can be set in the case of observation of a rightward movement.

  Next, the straight line on the accumulated image corresponding to the extracted point on the θ-ρ space is used as the locus of the extreme point, and the slope of the straight line is used as the moving speed of the extreme point on the image (step S6-1). -2). The slope of the straight line represents how many pixels x the corresponding extreme point moves per frame.

  Next, the representative moving speed of the people shown in the image is obtained from the estimated moving speed of each extreme point (step S6-1-3). The method of calculating the representative moving speed is not limited here, but for example, a method of taking the average value of each moving speed or a method of taking the median value can be considered, and a method that leads to more accurate estimation of the number of people is selected depending on the scene preferable. The method of taking an average value is more effective when the number of people is such that the locus of extreme points corresponds to one person in the video, and the method of taking the median value is more effective when the number of people is not so.

(Method 2)
First, similarly to step S6-1-1, a point on the θ-ρ space where the pixel value is equal to or greater than a certain threshold is extracted (step S6-2-1). Next, a projection process is performed on the coordinate axis ρ in the binary image (θ, ρ) in which the point extracted in step S6-2-1 is 1 and the other points are 0 (see, for example, step S3 and FIG. 3). Then, an angular distribution of the extreme point locus is generated (step S6-2-2).

  Next, a representative angle θ￣ (￣ is attached to θ. The same applies hereinafter) is calculated from the angular distribution of the extreme point locus, and the xt space corresponding to the angle θ￣ on the θ-ρ space is calculated. The inclination at is the representative moving speed of people shown in the image. (Step S6-2-3) The calculation method of the representative angle θ￣ is not limited here. For example, there are a method of taking an average of the angle distribution and a method of selecting an angle at which the distribution is maximized. Also, a process such as applying a smoothing filter to the angle distribution may be performed before calculating the representative angle θ￣.

(Method 3)
First, each pixel value of the θ-ρ image obtained by performing θ-ρ Hough transform on the accumulated image is squared to emphasize points with many votes (step S6-3-1). Next, a projection process for the coordinate axis ρ is performed on the squared θ-ρ image (see step S3, FIG. 3), and an angular distribution of the extreme point locus is generated (step S6-3-2). Note that unlike the case of step S3, the θ-ρ image that is the object of the projection processing here is a multi-valued image. Next, similarly to step S6-2-3, the representative moving speed is calculated from the representative angle (step S6-3-3).

  Next, the in-screen number calculation unit 107 calculates the number of people who are moving in the direction of observing human flow among the people shown in the current input image from the video taken by the camera 2 (Step S1). S7). The method for calculating the number of persons is not limited here. For example, in the method of Non-Patent Document 6 shown below, a method is used in which the foreground image is a binary image generated in step S3. Since the method of Non-Patent Document 6 has a feature that the number of people can be stably estimated even in a situation where there are a large number of people in the image, it is a preferable method for accurately measuring the number of people passing in a situation where there are many people in the image. . Also, by using the binary image generated in step S3 as the foreground image, the number of people moving in the direction of observing the human flow is calculated even when there are people moving in different directions. Can do.

  Non-Patent Document 6: Hiroyuki ARAI, Isao MIYAGAWA, Hideki KOIKE, and Miki ASEYAMA, Members, “Estimating Number of People Using Calibrated Monocular Camera Based on Geometrical Analysis of Surface Area”, IEICE Trans. Fundamentals, Vol. E92-A, No 8, pp. 1932-1938, August 2009

  Further, the number of people moving in the direction of observing the human flow obtained in step S7 may be the number obtained from the maximum value counted in step S4. Also, in a crowded situation where non-plural people move in an overlapping manner in the image and the method according to step S4 counts as one person, it is useful to use the method according to Patent Document 6.

  Next, the passing number calculation unit 108 calculates the instantaneous passing number in the current input image from the representative moving speed calculated in step S6 and the number calculated in step S7 (step S8). The method of calculating the instantaneous passing number is not limited here, but, for example, the representative moving speed is divided by the length in the x direction of the image, and the product of the number of persons moving in the x direction in the image is taken. Shall. For example, assuming that one person moves by the image length in the x direction is regarded as the passage of one person, how many times the person has moved the image length in the x direction between the previous frame and the current frame, Is calculated. As an example, when three persons move by 0.1 times the image width in the x direction during one frame, the instantaneous passing number is 3 × 0.1 = 0.3.

  The passing number calculation unit 108 determines whether or not the passing number of people in all directions has been calculated. If there are a plurality of directions in which the passing is desired and the number of passing numbers in all directions is not calculated, each passing number is calculated. The process from step S3 to step S8 is repeatedly performed with respect to the direction of the number, and the number of passing people is calculated (step S9).

  Next, the passing number calculation unit 108 calculates the cumulative passing number in each direction by cumulatively adding the instantaneous passing number for each direction in which the passing is measured. The cumulative number of passing people is the number of people who have passed through the video in each direction from the frame in which the process is started to the current frame. For example, in the same situation as the example shown in step S8, when the speed of the person is constant, the three persons move from one end of the image to the opposite end over 10 frames. That is, the cumulative number of passing people is 0.3 × 10 = 3 for 10 frames of 0.3, and the cumulative number of people passing by increases by three. The cumulative passing number 3 means that there are 3 persons who have moved by the image length in the x direction in the image, or 6 persons who have moved by 1/2 the image length in the x direction in the image. This is the value shown. In this way, it is possible to calculate an approximate value of the number of people who have moved by the image length in the x direction in an image within a certain time from only the image without using other sensors.

  Then, the passing number calculation unit 108 determines whether or not the end condition is satisfied (step S10). If the end condition is satisfied, the process ends. If not, one new frame of video is acquired (step S11), and the processes from step S1 to step S10 are repeated.

  In the embodiment described above, after step S9, the passing number calculation unit 108 determines whether or not there are a plurality of ranges in the image for which the passage is desired to be measured. The processes from step S1 to step S9 may be performed on the partial images in the range, and when all the partial images in the respective ranges are processed, the process may move to step S10.

  In the embodiment described above, the process of the passing person counting device 1 for the image obtained from the camera 2 may be performed in real time or not.

  In the above-described embodiment, the processing from step S1 to step S2 (motion vector calculation and foreground extraction) is not performed, and the image of the input frame and the motion vector information of each pixel of the image, the image The foreground area information may be used as an input, and the processes in and after step S3 may be performed.

<Second Embodiment>
FIG. 7 is a functional block diagram showing the configuration of the passing number measuring device 1 in the second embodiment. Here, portions corresponding to the respective portions in FIG. 2 are denoted by the same reference numerals, and differences thereof will be described. The projection processing unit 103 receives the image data output from the camera 2, the motion vector of each pixel, and the foreground image of the image. These image data, motion vector, and foreground image may be stored in a storage device, for example, and read out by the projection processing unit 103. The passing number calculation unit 108 calculates the passing number from the cumulative image generated by the cumulative image generation unit 105. For example, the number of moving persons is calculated from the maximum value of the projected image included in the accumulated image.

<Third Embodiment>
FIG. 8 is a functional block diagram showing the configuration of the passing number measuring device 1 in the third embodiment. Here, portions corresponding to the respective portions in FIG. 2 are denoted by the same reference numerals, and differences thereof will be described. The projection processing unit 103 receives the image data output from the camera 2, the motion vector of each pixel, and the foreground image of the image. These image data, motion vector, and foreground image may be stored in a storage device, for example, and read out by the projection processing unit 103. Here, for example, the image data, the motion vector, and the foreground image are known, and the speed of the moving person is calculated by obtaining the representative moving speed based on the known image data, motion vector, and foreground image.

<Fourth Embodiment>
FIG. 9 is a functional block diagram showing the configuration of the passing number measuring device 1 in the fourth embodiment. Here, portions corresponding to the respective portions in FIG. 2 are denoted by the same reference numerals, and differences thereof will be described. The projection processing unit 103 receives the image data output from the camera 2, the motion vector of each pixel, and the foreground image of the image. These image data, motion vector, and foreground image may be stored in a storage device, for example, and read out by the projection processing unit 103. Here, for example, the image data, the motion vector, and the foreground image are known, and based on this, the cumulative value (cumulative value) of the number of people who move the distance (image length distance) of the area projected on the image within a certain time. Calculate the number of people passing through.

<Fifth Embodiment>
Next, the passing number measuring device 1 in the fifth embodiment will be described. FIG. 10 is a schematic block diagram showing the configuration of the passing person counting device according to the embodiment of the present invention. The motion vector extraction unit 300 receives an image from the camera 2 and outputs a motion vector. The foreground detection unit 301 inputs an image from the camera 2 and outputs a foreground image. The real space mapping unit 302 receives the motion vector output from the motion vector extraction unit 300 and the foreground image output from the foreground detection unit 301, and outputs the real space image and the number of persons shown on the screen. The projection processing unit 303 receives the real space image output from the real space mapping unit 302, generates a projected motion vector, and outputs the generated motion vector. The extreme point extraction unit 304 receives the projected motion information output from the projection processing unit 303, calculates and outputs the extreme point of the motion information.

  The cumulative image generation unit 305 receives the extreme point of the motion information output from the extreme point extraction unit 304 and outputs an image indicating the position of the extreme point. The representative speed calculation unit 306 receives the cumulative image output from the cumulative image generation unit 305, generates and outputs the representative movement speed of people in the image. There are various ways of generating the representative moving speed, which will be described later. The passing number calculation unit 307 inputs the number of persons shown in the image output from the real space mapping unit 302 and the representative moving speed of people in the image output from the representative speed calculation unit 306. From the captured video, the number of people moving in the direction of observing the human flow among the people shown in the current input image is calculated and output.

  Next, the operation of the passing person measuring apparatus 1 having the above-described configuration will be described. FIG. 11 is a flowchart for explaining the operation of the passing number measuring device 1. First, when the processing is started and image data is output from the camera 2, the foreground detection unit 301 generates a partial image obtained by extracting a preset range from the current input image output from the camera 2 ( Step S21). The method for determining the range to be extracted is, for example, a method in which a place where a passage is to be counted in the input image is taken as an imaged area, or a portion where a person is clearly not captured (such as a wall or a fixed facility) is excluded. The method of making it into a region etc. are mentioned. This range may be the entire input image.

  Next, the foreground detection unit 301 generates a foreground image in the extracted partial image (step S22). Here, the foreground image is an image in which a moving object exists in a partial image, that is, a point that is a foreground is 1, and a point that is not, that is, a point that is a background is 0. Various methods for detecting the foreground image are known, but any method may be applied. Note that examples of the foreground image detection method include those described in Non-Patent Documents 4 and 5 described above.

On the other hand, the motion vector extraction unit 300 detects the optical flow (see Non-Patent Document 3 above) in the foreground portion from the partial image of the current frame of the video and the partial image of the previous frame, and is captured by the camera. The motion vector and the time t (number of frames) during which the optical flow is continuously detected are calculated from the current partial image (step S23). This motion vector is represented as a motion amount (V _x , V _y ) at each pixel position (x, y), for example, where x is from left to right and y is from top to bottom.

  The method of calculating the optical flow is not limited here, but it is preferable to select a method in which the flow is stably generated depending on the scene. For example, a method based on luminance gradient or a method based on region matching can be considered, but a method based on luminance gradient is more effective when the human density in the imaging region is large, and a method based on region matching is a person in the imaging region. It is more effective when the density is small.

  Next, the real space mapping unit 302 generates a binary image in which the point where the direction of the motion vector is in an arbitrary range and the absolute value is equal to or larger than an arbitrary threshold is 1 and the point where the absolute value is not 0 is 0 ( Step S24). Here, when generating a binary image, a process such as a morphological operation for removing the influence of local false detection of a motion vector may be performed. In determining the range of the direction of the motion vector and the threshold value, the method is not limited here. For example, the following method can be considered.

(1) The range of the direction of the motion vector is a range in which the component in the direction in which the vector passage is desired to be measured faces the direction in which the passage is desired to be measured. For example, in the case of measuring rightward passage, the range is such that the x component of the motion vector is positive.

(2) The threshold value of the absolute value is experimentally determined to be a value that is sufficiently large and as small as possible to discard the influence of the motion detected other than the motion of the person.

  Next, the real space mapping unit 302 creates a real space image representing the position of the person in the real space (step S25). This is a real space corresponding to a range on the two-dimensional real space XY that can be a “standing position of a person image including the pixel” for each pixel whose value of the binary image generated in step S24 is 1. This is realized by adding a predetermined value to the pixels of the image. Hereinafter, this pixel value addition processing is referred to as mapping.

  Here, how to derive the mapping range will be described with reference to FIGS. As a premise, the human model is a rectangular flat plate with a height h and a width w facing the camera (h and w are predetermined. For example, h = 1.7 [m], w = 0.3 [m ]). It is assumed that there is no rotation around the optical axis of the camera.

  Assuming that the pixel at a certain coordinate (x, y) is the center of the bottom (foot) of the person rectangle and the center of the top (top), the corresponding two-dimensional real space XY Considering the standing positions P (x, y) and P ′ (x, y) (see FIG. 13), the coordinates of P (x, y) and P ′ (x, y) are obtained from (x, y), respectively. For example, the expression (12) of Non-Patent Document 6 described above is used).

  At this time, the range of the real space coordinates corresponding to the case where the pixel in question is an arbitrary point on the central axis of the person is a line segment connecting two points P (x, y) and P ′ (x, y). It becomes. Furthermore, if P ′ (x, y) is sufficiently far from the camera position (hereinafter, camera position is the origin) and the width of the person on the image is sufficiently large relative to the pixel width, the pixel in question is the person. The real space coordinate range corresponding to the above arbitrary point, that is, the mapping range to be obtained, is obtained by giving the width of w to this line segment, that is, ± 2 points in the direction perpendicular to the line segment. It can be approximated to a rectangle whose vertices are four points moved by w / 2 (see FIG. 13).

  The pixel value is mapped in the range derived as described above. An example of a specific method for realizing the mapping will be described below with reference to FIGS.

  First, a real space image in which the value of each pixel is 0 is created as a mapping destination space (step S25-1). The size of the real space image is set to a size that can cover the entire mapping range corresponding to each pixel of the binary image generated in step S24. The range to be covered can be calculated by considering the range covering the mapping range corresponding to the pixels at the four corners of the binary image as shown in FIG. For example, the aspect ratio of the captured image is 4: 3, the height from the ground surface of the camera is 7.1 m, the depression angle from the horizontal direction is 17 °, the diagonal angle of view is 46.77 °, and the height of the human model is 1. When the width is 7 m and the width is 0.3 m, the range that the real space image should cover can be -55 [m] to 55 [m] in the X direction and 8 [m] to 163 [m] in the Y direction ( (See Fig. 14 for how to set the X and Y coordinate axes).

  In addition, the step size of the coordinates of the real space image is set to a maximum width sufficient for observing the movement of the person (for example, 10 cm step). If the step size is 10 cm in the example of the above-described coverage range, the size of the real space image is 1101 horizontal × 1551 pixels vertical.

  Next, with respect to the coordinates (x, y) of each pixel whose pixel value is 1 in the binary image, 1 is added to the value of the pixel corresponding to P ′ (x, y) on the real space image (step S25−). 2, FIG. 15 (a)). In FIG. 15, the process from step S25-2 to step S5-4 described later is shown with attention paid to one pixel.

Next, by applying a filter whose length, direction, and coefficient change depending on the distance from the origin of the application pixel to the real space image, the pixels are distributed in the direction of the straight line connecting the origin and the filter application point ( Step S25-3, FIG. 15 (b)). If the distance from the origin to the filter application point is r, the filter direction is the direction of the straight line connecting the origin and the filter application point (this direction is hereinafter referred to as the r direction at that point), and the filter length in the r direction is | P−P ′ | = hr / T _z , width is 1, and each filter coefficient is T _z s ^ (x, y) / (hr) (^ is on s. The same applies hereinafter). Here, T _z is the height from the floor surface to the camera. s ^ (x, y) is a value for weighting the pixels of the binary image, and this weight is given to the pixels on the binary image whose value is 1 when one person appears in the captured image. The sum of the attached values is a value that becomes 1 regardless of where the person appears in the captured image. This is the same as s ^ (x, y) in Non-Patent Document 5.

  Next, in each pixel, the pixel is distributed in the direction perpendicular to the r direction by applying a filter to the real space image in the direction perpendicular to the r direction at the position of the pixel (step S25-4, FIG. 15 ( c)). The length of the filter is w (the width of the person model), the width is 1, and each filter coefficient is 1 / w.

  An example in which mapping is actually performed on a real space image is shown in FIG. When the image in FIG. 16A is mapped to the real space as a motion detection part of the captured image, for example, the image in FIG. 16B is obtained.

  Next, the real space mapping unit 302 calculates the sum of the pixel values of the generated real space image (step S26). When mapping is performed by the method of step S25 described above, the sum of components resulting from mapping from the pixel (x, y) on the binary image among the pixel values of the real space image is s ^ (x, y). . Therefore, the calculated sum is equal to the sum of the pixels of the image obtained by multiplying the binary image calculated in step S24 by ＾ (x, y). This corresponds to the number of persons on the screen moving in the direction of detecting movement.

  Next, as shown in FIG. 17, the projection processing unit 303 takes the coordinate axis u in a certain direction and generates a value of each pixel on the real space image with the same position in the u direction. A one-dimensional projected image taking the sum is generated (step S27).

  In FIG. 17, the horizontal direction is the coordinate axis u, and the sum of the values of the pixels having the same position on the u axis is calculated. This calculation is performed at each position on the u-axis. Here, as an example, the case where the data indicated by the symbol (a) is obtained as the projected image is shown.

  Thus, in this embodiment, the process of calculating the sum of the values of pixels having the same position in the u direction in the image is referred to as a projection process for the coordinate axis u. Here, the direction of the coordinate axis u is not limited. For example, the X direction is used when observing the human flow in the left-right direction, and the Y direction is used when observing the human flow in the front and back directions. It is conceivable to take the same value as the one closer to the direction of the human flow to be measured out of the X direction or Y direction of the real space image.

  When the projection image is generated, the extreme point extraction unit 304, as shown in FIG. 17, sets the point where the value of the projection image takes the maximum value as 1, and sets the point other than 0 as 0. Is generated (step S28). The point at which the value of the projected image takes the maximum value corresponds to the human candidate position in the u direction. Here, before extracting the extreme value, a process such as applying a smoothing filter to the projected image may be performed.

  When the extreme point image is generated, the cumulative image generation unit 305 receives the extreme point image output from the extreme point extraction unit 304 and generates a cumulative image that is an image indicating the position of the extreme point. (Step S29). Here, FIG. 18 is a diagram for explaining the accumulated image. Here, the extreme point image is stored in the memory in correspondence with the time t by the extreme point extraction unit 304 (FIGS. 18A and 18B). The cumulative image generation unit 305 generates a two-dimensional cumulative image (u, t) indicating the locus of the extreme points (FIG. 18 (c)). Here, a two-dimensional cumulative image is generated by accumulating an extreme value image in which 1 is assigned to a pixel at a position on the coordinate axis u representing the maximum value and 0 is assigned to a pixel that is not, according to time t. In FIG. 18, the pixel having the maximum value and the pixel not having the maximum value are shown in different colors.

  Here, the size of the accumulated image in the u direction is generated so as to coincide with the size of the extreme point image in the u direction. The size of the accumulated image in the t direction is set to a minimum value at which a locus candidate can be detected during the Hough transform described later.

  When the accumulated image is generated, the representative speed calculation unit 306 obtains the representative moving speed of the people shown in the captured image from the accumulated image (step S30). There are various ways of obtaining the representative moving speed. For example, the following three methods (method 4 to method 6) are given as examples.

(Method 4)
First, θ-ρ Hough transform is performed on the accumulated image, and a point on the θ-ρ space where the pixel value (the number of votes) is equal to or greater than a certain threshold is extracted (step S30-1-1). At this time, the range of θ may be limited in the Hough transform in order to prevent erroneous detection of straight line candidates that are not the locus of extreme points and to shorten the calculation time. For example, when it is assumed that the moving speed of a person is 3 [pixel / frame] or less, 90 ° ≦ θ ≦ 109 ° can be set in the case of observation of a rightward movement. In determining the threshold value, the method is not limited here. For example, depending on how many extreme points are arranged on a straight line on the accumulated image, the straight line is regarded as the locus of the extreme points. From the viewpoint that there are few detection errors of straight line candidates corresponding to the locus of extreme points and false detection of straight line candidates that do not correspond to the locus of extreme points, it is determined experimentally. There is a way.

Next, the straight line on the accumulated image corresponding to the extracted point on the θ-ρ space is used as the locus of the extreme point, and the inclination of the straight line is used as the moving speed of the extreme point on the image (step S30-1). -2).
The slope of the straight line represents how many pixels u the corresponding extreme point moves in one frame.

  Next, the representative moving speed of the people shown in the captured image is obtained from the estimated moving speed of each extreme point (step S30-1-3). The method of calculating the representative moving speed is not limited here, but for example, a method of taking the average value of each moving speed or a method of taking the median value can be considered, and a method that leads to more accurate estimation of the number of people depending on the scene is selected. preferable. The method of taking an average value is more effective when the number of people is such that the locus of extreme points corresponds to one person in the video, and the method of taking the median value is more effective when the number of people is not so.

(Method 5)
First, similarly to step S30-1-1, a point on the θ-ρ space whose pixel value is equal to or greater than a certain threshold is extracted (step S30-2-1). Next, a projection process is performed on the coordinate axis ρ in the binary image (θ, ρ) in which the point extracted in step S30-2-1 is 1 and the other points are 0 (see step S27, FIG. 17). An angular distribution of the extreme point locus is generated (step S30-2-2).

  Next, the representative angle θ￣ is calculated from the angular distribution of the extreme point locus, and the representative movement of people whose inclination in the ut space corresponding to the angle θ￣ in the θ-ρ space is shown in the captured image The speed is set (step S30-2-3). Although the calculation method of the representative angle θ し な い is not limited here, for example, there are a method of averaging the angle distribution and a method of selecting an angle at which the distribution is maximum. Also, a process such as applying a smoothing filter to the angle distribution may be performed before calculating the representative angle θ￣.

(Method 6)
First, each pixel value of the θ-ρ image obtained by performing θ-ρ Hough transform on the accumulated image is squared to emphasize points with many votes (step S30-3-1). Next, a projection process for the coordinate axis ρ is performed on the squared θ-ρ image (see step S27, FIG. 17), and an extreme point locus angular distribution is generated (step S30-3-2). Note that unlike the case of step S27, the θ-ρ image that is the subject of the projection processing here is a multi-valued image. Next, similarly to step S30-2-3, the representative moving speed is calculated from the representative angle (step S30-3-3).

  Next, when the representative moving speed is calculated, the passing number calculation unit 307 calculates the instantaneous number of passing persons in the current partial image from the representative moving speed calculated in step S30 and the number of persons calculated in step S26. (Step S31). The method for calculating the instantaneous passing number is not limited here. For example, the representative moving speed is divided by the length of the real space image in the u direction, and the product is multiplied by the number of persons moving in the u direction in the image. Shall be. For example, assuming that one person moves by the image length in the u direction is regarded as passing one person, how many people have moved relative to the image length in the u direction between the previous frame and the current frame. Or calculate. As an example, when three persons move by 1/10 of the image length in the u direction during one frame, the instantaneous passing number is 3 × 1/10 = 0.3.

  Next, the passing number calculation unit 307 determines whether or not the passing number of people in all directions has been calculated, and when there are a plurality of directions in which the passing is desired and the number of passing numbers in all directions is not calculated, The process from step S24 to step S31 is repeated for each direction to be measured, and the number of passing people is calculated (step S32).

  Next, the passing number calculation unit 307 cumulatively adds the instantaneous passing number for each direction in which passage is measured, thereby obtaining the cumulative passing number in each direction (step S33). The cumulative number of passing people is the number of people who have passed through the video in each direction from the frame in which the process is started to the current frame. For example, in the same situation as the example shown in step S31, if the speed of the person is constant, the three persons move from one end of the image to the opposite end over 10 frames. That is, the cumulative number of passing people is 0.3 × 10 = 3 for 10 frames of 0.3, and the cumulative number of people passing by increases by three.

  Next, the passage number calculation unit 307 determines whether or not there are a plurality of ranges in which the passage is desired to be measured, and if there are a plurality of ranges, from step S22 to step S33 for the partial image in each range to be measured. Is performed (step S34). When all the partial images in the respective ranges are processed, the process proceeds to step S35.

  Next, the passing number calculation unit 307 determines whether or not the end condition is satisfied (step S15). If the end condition is satisfied, the process ends. If not, one new frame of video is acquired (step S36), and the process from step S21 to step S35 is repeated.

  In this way, it is possible to calculate an approximate value of the number of people who have moved by the image length in the u direction in an image within a certain time without using another sensor.

  In the embodiment described above, the process of the passing person counting device 1 for the image obtained from the camera 2 may be performed in real time or not.

<Sixth Embodiment>
FIG. 19 is a functional block diagram showing the configuration of the passing number measuring device 1 in the sixth embodiment. Here, portions corresponding to the respective portions in FIG. 10 are denoted by the same reference numerals, and differences thereof will be described. The real space mapping unit 302 receives a motion vector and a foreground image, generates a real space image and the number of persons shown on the screen, and outputs it. These motion vectors and foreground images may be stored in a storage device, for example, and read by the real space mapping unit 303. The projection processing unit 303 inputs the real space image output from the real space mapping unit 302. The passing number calculating unit 307 calculates the passing number from the accumulated image generated by the accumulated image generating unit 305. For example, the number of moving persons is calculated from the maximum value of the projected image included in the accumulated image.

<Seventh Embodiment>
FIG. 20 is a functional block diagram showing the configuration of the passing number measuring device 1 in the seventh embodiment. Here, portions corresponding to the respective portions in FIG. 10 are denoted by the same reference numerals, and differences thereof will be described. The real space mapping unit 302 receives a motion vector and a foreground image, generates a real space image and the number of persons shown on the screen, and outputs it. The projection processing unit 303 inputs the real space image output from the real space mapping unit 302. Here, for example, the image data, the motion vector, and the foreground image are known, and the speed of the moving person is calculated by obtaining the representative moving speed based on the known image data, motion vector, and foreground image.

<Eighth Embodiment>
FIG. 21 is a functional block diagram showing the configuration of the passing number measuring device 1 in the eighth embodiment. Here, portions corresponding to the respective portions in FIG. 10 are denoted by the same reference numerals, and differences thereof will be described. The real space mapping unit 302 receives a motion vector and a foreground image, generates a real space image and the number of persons shown on the screen, and outputs it. The projection processing unit 303 takes a coordinate axis u in a certain direction with respect to the real space image generated by the real space mapping unit 302 and takes the sum of the values in each pixel on the real space image having the same position in the u direction. Generate a projected image of. Here, for example, the image data, the motion vector, and the foreground image are known, and based on this, the cumulative value (cumulative value) of the number of people who move the distance (image length distance) of the area projected on the image within a certain time. Calculate the number of people passing through.

<Ninth Embodiment>
In Non-Patent Document 1, the area for measuring the number of people for each moment is set on the image space, and the correspondence between the coordinates of the image space and the real space is not uniquely determined. This method cannot be used as it is when thinking. In order to solve such a problem, a method of measuring the number of people in the real space by mapping the foreground portion on the image into the real space can be considered. However, when creating a mapping table in advance, it is necessary to prepare a mapping function to the real space for each pixel, so the data size of the mapping function is (number of pixels of the captured image) × (number of points in the real space) order. Therefore, it is extremely difficult to secure this data on the memory of the computer. In the ninth embodiment, mapping from image coordinates to real space coordinates is realized with a small amount of memory usage, and analysis is performed in the real space, thereby existing in an arbitrary real space area shown in the video. The number of persons is estimated and is a modification of the fifth embodiment described above.

  Next, a ninth embodiment will be described in detail. In the ninth embodiment, it is an object to measure the number of people per moment existing in an arbitrary real space area shown in the video. The device configuration in the ninth embodiment is the same as the device configuration shown in FIG. In addition, the camera 2 shall use the camera fixed to the predetermined position. Further, in the following description, description will be made assuming a situation in which images (frame images) taken by the camera 2 are input in time series. The input image is a still image series, a video stream, or the like, and the processing does not necessarily have to be performed in real time.

  FIG. 22 is a functional block diagram showing the configuration of the passing number measuring device 1 in the ninth embodiment. In FIG. 22, reference numeral 401 denotes a foreground detection unit that inputs an image from the camera 2, detects a foreground from the input image, and outputs a foreground image. Reference numeral 402 denotes a real space mapping unit that inputs a foreground image, performs a real space mapping process, and outputs a real space image. Reference numeral 403 denotes an in-region number calculation unit that inputs a real space image, calculates the number of people in the region, and outputs in-region number information.

  Next, the operation of the apparatus shown in FIG. 22 will be described with reference to FIG. First, when processing is started, a partial image is generated by extracting a preset range from the current input image. The method for determining the range to be extracted here is, for example, a method in which a part of the input image can correspond to a position in the real space where the number of people is to be counted, a part where a person is clearly not captured (a wall or a fixed facility) Etc.) can be applied. This range may be the entire input image.

  Next, the foreground detection unit 401 detects the foreground part in the extracted partial image, generates a foreground image, and outputs it (step S41). Here, the foreground image is an image in which a moving object exists in a partial image, that is, a point that is a foreground is 1, and a point that is not, that is, a point that is a background is 0. Various methods for detecting the foreground image are known. For example, techniques such as Non-Patent Documents 4 and 5 described above can be applied.

  Next, the real space mapping unit 402 creates and outputs a real space image representing the position of the person in the real space (step S42). This is because the pixels of the real space image corresponding to the range on the two-dimensional real space XY that can be “the standing position of the human image including the pixel” for each of the pixels for which the value of the generated foreground image is 1. This is realized by adding a predetermined value. Hereinafter, this pixel value addition processing is referred to as mapping. The range of mapping is derived as follows.

  As a premise, the human model is a rectangular flat plate having a height h and a width w facing the camera. The height h and the width w are determined in advance, for example, h = 1.7 [m] and w = 0.3 [m]. It is assumed that there is no rotation around the optical axis of the camera.

  Assuming that the pixel at a certain coordinate (x, y) is the center of the bottom (foot) of the person rectangle and the center of the top (top), the corresponding two-dimensional real space XY Considering the standing positions P (x, y) and P ′ (x, y) (see FIG. 12), the coordinates of P (x, y) and P ′ (x, y) are obtained from (x, y). For example, an expression described in Non-Patent Document 6 (for example, Expression (12)) is used.

  At this time, the range of the real space coordinates corresponding to the case where the pixel in question is an arbitrary point on the central axis of the person is a line segment connecting two points P (x, y) and P ′ (x, y). It becomes. Further, when P ′ (x, y) is sufficiently away from the camera position (hereinafter, camera position is the origin) and the width of the person on the image is sufficiently larger than the pixel width (pixel width / person width) Is a mapping range width error), the real space coordinate range corresponding to the case where the pixel of interest is an arbitrary point on the person, that is, the mapping range to be obtained has a width of w on this line segment. In other words, it can be approximated to a rectangle whose vertices are four points obtained by moving two points by ± w / 2 in a direction perpendicular to the line segment (see FIG. 13). The pixel value is mapped in the range derived as described above.

  Here, a specific method for realizing the mapping (step S42) will be described. First, a real space image in which the value of each pixel is 0 is created as a mapping destination space (step S42-1-1). The size of the real space image is set to a size that can cover the entire mapping range corresponding to each pixel of the foreground image generated in step S41. The range to be covered can be calculated by considering a range that covers the mapping range corresponding to the pixels at the four corners of the foreground image (binary image) as shown in FIG. For example, the aspect ratio of the captured image is 4: 3, the height from the ground surface of the camera is 7.1 m, the depression angle from the horizontal direction is 17 °, the diagonal angle of view is 46.77 °, and the height of the human model is 1. When measuring the number of people in the region corresponding to the entire image when the width is 7 m and the width is 0.3 m, the real space image should cover the X direction −55 [m] to 55 [m] and the Y direction 8 [m]. ] To 163 [m]. In addition, the step size of the coordinates of the real space image is set to a maximum width sufficient for observing the movement of the person (for example, 10 cm step). If the step size is 10 cm in the example of the above-described coverage range, the size of the real space image is 1101 horizontal × 1551 pixels vertical.

  Next, with respect to the coordinates (x, y) of each pixel having a pixel value of 1 in the foreground image, 1 is added to the value of the pixel corresponding to P ′ (x, y) on the real space image (step S42-1). -2, Fig. 15 (a)). In this step, 1 is added to the pixel value of the real space image for any foreground pixel, and the pixel is weighted by filtering in a later step.

Next, by applying a filter whose length, direction, and coefficient change depending on the distance from the origin of the application pixel to the real space image, the pixels are distributed in the direction of the straight line connecting the origin and the filter application point ( Step S42-1-3, FIG. 15 (b)). When the distance from the origin to the filter application point is r, the filter direction is the direction of the straight line connecting the origin and the filter application point (r direction), and the filter length in the r direction is | P−P ′ | = hr / T _z , The width is 1, and each filter coefficient is T _z s ^ (x, y) / (hr). Here, T _z is the height from the floor surface to the camera. s ^ (x, y) is a value for weighting the pixels of the foreground image, and this weight is given to the pixels on the foreground image that has a value of 1 when one person appears in the captured image. The sum of the values is such that the person is 1 regardless of where the person appears in the captured image. This is the same as s ^ (x, y) in Non-Patent Document 4.

  Next, in each pixel, the pixel is distributed in a direction perpendicular to the r direction by applying a filter to the real space image in a direction perpendicular to the r direction at the position of the pixel (step S42-1-4, FIG. 15 (c)). The length of the filter is w (the width of the person model), the width is 1, and each filter coefficient is 1 / w. An example in which mapping is actually performed on a real space image is shown in FIG. When the image in FIG. 16A is mapped to the real space as a motion detection part of the captured image, for example, the image in FIG. 16B is obtained. The sum of the pixel values of the real space image created by the method described above represents the number of persons existing in the area included in the image. Further, the sum of pixel values included in an arbitrary partial area of the real space image is the number of persons existing in the real space area corresponding to the partial area.

  Returning to FIG. 23, next, the in-area number calculation unit 403 obtains the estimated value of the number of persons existing in the corresponding real space area by taking the sum of the pixel values of the real space image generated by the real space mapping unit 402. The number of people in the screen is calculated (step S43). When there are a plurality of areas for which the number of people is to be measured, the necessary real space area may be extracted after processing the entire captured image, or a plurality of partial areas are extracted from the captured image, and steps S41 to S43 are performed for each. The above process may be repeated. Then, it is determined whether or not the end condition is satisfied (step S44). If not satisfied, one new frame of video is acquired, and the process returns to step S41 to repeat the process.

<Tenth Embodiment>
Next, a tenth embodiment will be described. In the tenth embodiment, the processing (steps S25 and S42) executed in the real space mapping 302 in the fifth embodiment and the real space mapping unit 402 in the ninth embodiment described above are replaced with processing operations described below. It is a thing.

  First, an x-log Y space image in which the value of each pixel is 0 is created as the first mapping destination space. The size of the x-logY space image is set to a size that can cover the entire mapping range corresponding to each pixel of the foreground image generated in steps S22 and S41. The range to be covered can be calculated by considering the range covering the mapping range corresponding to the pixels at the four corners of the foreground image as in step S42-2-1. For example, the aspect ratio of the captured image is 4: 3, the height from the ground surface of the camera is 7.1 m, the depression angle from the horizontal direction is 17 °, the diagonal angle of view is 46.77 °, and the height of the human model is 1. When the number of people in the area corresponding to the entire image is measured when the width is 7 m and the width is 0.3 m, the range in the log Y direction of the x-log Y space image can be set to 2.07 to 5.10.

  Further, the step size of the coordinates in the log Y direction of the x-log Y space image is determined according to how far the position error can be tolerated. For example, when the increment is 0.01, the error in the pixel distribution range is about 1% at the maximum. If the step size is 0.01 in the above-described example of the coverage range, the size of the real space image in the log Y direction is 304 pixels.

Next, T _z / (hr) is added to the value of the pixel corresponding to P ′ on the x-logY space image with respect to the coordinates (x, y) of each pixel whose pixel value is 1 in the foreground image. T _z / (hr) is the same value as that described in step S42-1-3. Since this value does not change with time, it may be calculated in advance and stored in the memory before the start of the process of repeating imaging, mapping, and number estimation.

Next, the pixels are distributed in the logY direction by filtering the x-logY space image. This corresponds to distributing pixels in the r direction described above. The length of the filter in the logY direction is logr_P-logr_P ′ = log (r_P / r_P ′) = log (T _z / (Tz−h)) and is constant. The width in the x direction is 1. Here, r_P and r_P ′ are r at points P and P ′, respectively. Each filter coefficient is 1.

  Next, an x-Y space image is created as a second mapping destination space. The size of the x-Y space image is equal to the x-log Y space image in the x direction and is equal to the real space image in step S42-1-1 in the Y direction. Each pixel value of the x-Y space image is equal to the pixel value of the corresponding coordinate in the x-log Y space image.

  Next, as a second mapping destination space, a real space image is created by the same method as in step S42-1-1. For each pixel of the x-Y space image, the pixel value is set on the real space image. Add coordinates to the corresponding pixel. Subsequently, the real space image is filtered to distribute the pixels in the X direction (see FIG. 24). The size in the X direction of the filter is the width of the person model, and the size in the Y direction is 1. This process corresponds to approximating the human existence range and distributing pixels in the X direction as shown in FIG. When mapping is actually performed on the real space image by the above method, it is as shown in FIG. 16 as in the fifth and ninth embodiments.

  The method according to the tenth embodiment has the difficulty that an error of the number of people occurs due to the approximation of the pixel distribution range. On the other hand, since the size and direction of the filter are constant regardless of the coordinates, the method according to the ninth embodiment is more processed. Has the advantage of being able to be fast. The sum of the pixel values of the real space image created by the method described above represents the number of persons existing in the area included in the image. Further, the sum of pixel values included in an arbitrary partial area of the real space image is the number of persons existing in the real space area corresponding to the partial area.

  As described above, since the number of persons in an arbitrary area of the real space shown in the video can be directly measured, the range to be observed on the image can be set intuitively.

  In addition, a program for realizing the functions of the passing person counting device 1 in FIGS. 1, 2, 7 to 10, and 19 to 22 is recorded on a computer-readable recording medium and recorded on the recording medium. The number of passing people may be measured by reading the executed program into a computer system and executing it. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  It can be applied to applications where it is indispensable to measure the number of people and the number of people passing through an image processing by using image processing.

  DESCRIPTION OF SYMBOLS 1 ... Passage number measuring device, 2 ... Camera, 3 ... Display apparatus, 101, 300 ... Motion vector extraction part, 102, 301, 401 ... Foreground detection part, 103, 303 ... Projection process part, 104, 304 ... Extreme point Extraction unit, 105, 305 ... Cumulative image generation unit, 106, 306 ... Representative speed calculation unit, 107 ... In-screen number calculation unit, 108, 307 ... Passing number calculation unit, 302, 402 ... Real space mapping unit, 403 ... Area Number of people calculation section

Claims (12)

A person counting device that inputs a motion vector of each pixel in an image included in an image and a foreground area in the image and measures a moving person,
Projection means for projecting a motion vector in the foreground region in a one-dimensional manner to generate a projected image;
Extreme point extraction means for detecting a moving person by extracting extreme points from the pixels included in the projected image generated by the projection means;
A person counting device characterized by comprising: The extreme point extraction means extracts extreme points from pixels included in the projected image generated by the projection means and detects the position of the moving person on the projection axis. People counting device. Based on an image corresponding to a person included in the image, a real space image is generated by mapping the coordinates in the image to pixels on the real space coordinates indicating the position of the person in the real space. Having real space mapping means,
The number measuring device according to claim 1, wherein the projection processing unit generates a projection image from the real space image generated by the real space mapping unit.   The real space mapping means adds the predetermined value to the pixel corresponding to the range on the two-dimensional space that can be the standing position of the human image including the target pixel on the projected image, and performs the mapping, The person counting apparatus according to claim 3, wherein the real space image is generated.   The real space mapping means performs a first mapping process by adding a predetermined value to a pixel corresponding to a range on a logarithmic space that can be a standing position of a human image including a target pixel on the projected image, The person counting apparatus according to claim 3, wherein the real space image is generated by mapping the first mapping processing result on a two-dimensional space. Cumulative image generation means for accumulating the extreme value image extracted by the extreme value point extraction means for each elapsed time and generating a cumulative image;
The person counting according to any one of claims 1 to 5, further comprising: a number calculating means for measuring the number of moving persons based on the accumulated image generated by the accumulated image generating means. apparatus. Cumulative image generation means for accumulating the extreme value image extracted by the extreme value point extraction means for each elapsed time and generating a cumulative image;
Representative speed calculation means for calculating the representative movement speed of the moving person from the movement speed of each extreme point based on the cumulative image generated by the cumulative image generation means;
The person counting device according to any one of claims 1 to 5, characterized in that A number calculation means for detecting the number of persons included in the image;
Based on the number of persons detected by the number of persons calculating means and the representative speed calculated by the representative speed calculating means, a passing number calculating means for calculating the number of persons who pass through the region in the image within a predetermined time;
The person counting device according to any one of claims 1 to 5, characterized in that Foreground image generating means for detecting a foreground portion from the input video and generating a foreground image;
The real space image is generated by performing a mapping by adding a predetermined value to a pixel corresponding to a range in a two-dimensional space that can be a standing position of the human image including the target pixel on the foreground image. Real space image generation means;
A number-of-people counting device comprising: an area number calculating means for calculating the number of persons existing in a real space area by calculating a sum of pixel values of the real space image. Foreground image generating means for detecting a foreground portion from the input video and generating a foreground image;
A first mapping process is performed by adding a predetermined value to a pixel corresponding to a range in a logarithmic space that can be a standing position of a person image including the target pixel on the foreground image, and the first mapping process result Real space image generation means for generating the real space image by performing a second mapping process on the two-dimensional space from
A number-of-people counting device comprising: an area number calculating means for calculating the number of persons existing in a real space area by calculating a sum of pixel values of the real space image. Using a computer that is a person counting device that inputs a motion vector of each pixel in an image included in an image and a foreground area in the image and measures a moving person,
Projecting means of the computer to project a motion vector in the foreground region in a one-dimensional manner to generate a projected image;
The extreme point extraction means of the computer,
Detecting a moving person by extracting extreme points from pixels included in the projected image generated by the projecting means;
This is a method for counting people.   A computer program for causing a computer to function each means of the people counting device according to any one of claims 1 to 10.