JP2010198566A - Device, method and program for measuring number of people - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To measure the number of people included in a crowd based on images obtained by picking up the crowd. <P>SOLUTION: A device includes: a head model setter which inputs the images constituting one frame of a moving image obtained by picking up the plurality of persons and sets the input images while shifting a head model according to the size of a head expected at each part of the images when picking up the persons; a head extraction unit which extracts the heads of the persons contained in the images based on the set head model; a head associating unit which associates the extracted heads with each other among the plurality of images contained in the moving image; and a number of people measuring unit which measures the number of persons contained in the moving image based on the number of loci of the heads associated with each other. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a people counting device, method, and program, and more particularly, to a people counting device, method, and program for measuring the number of people included in a crowd by referring to a moving image obtained by capturing the crowd.

  As a method for detecting and counting a person, there is a method in which a mechanical gate or a detection sensor is provided at the entrance and exit of a target space, and the number of people in the space is grasped based on the number of people passing through.

  In addition, as a method based on image processing, there are a method for detecting and counting an optical flow of a person passing underneath using a camera installed vertically above, or a counting method based on face recognition.

  In patent document 1, the system which measures the number of people staying in a building or a room is described.

  In US Pat. No. 6,057,049, a computer-based system is described for performing scene content analysis for human detection and tracking.

JP 2008-217289 A Special table 2008-542922

  It is difficult to apply the above-described method of providing a mechanical gate or the like to a space with many doorways or an open space. This is because it is necessary to install mechanical gates at all the entrances and exits of the closed space.

  The above-described image processing method is also limited to a wide angle in a normal case because the available angle of view is limited, and can be applied only in a closed space.

  According to the system described in Patent Document 1, it is necessary to arrange a plurality of cameras, and to capture the same space with these cameras, so it is applied when measuring the number of people included in a narrow area such as indoors. Limited range.

  By the way, when many people are crowded, it is particularly difficult to count people based on video. This is because people are often concealed from each other, making it difficult to separate and detect individual people. In the system described in Patent Document 2, since the boundary between the foreground person and the background needs to be extracted as the outline of the blob, when many persons are crowded, person detection is performed. It becomes difficult.

  Therefore, it becomes a problem to be able to measure the number of people included in the crowd based on an image of the crowd. An object of the present invention is to provide a person counting device, method and program for solving such problems.

The number of people counting device concerning the 1st viewpoint of the present invention is
A head model corresponding to the size of the head expected in each part of the image when an image constituting a frame of a moving image obtained by imaging a plurality of people is input and the person is captured with respect to the input image A head model setting unit that sets the position while shifting the position,
A head extraction unit that extracts a human head included in the image based on the set head model;
A head associating unit that correlates the heads extracted between a plurality of images included in the moving image;
A person counting unit that measures the number of persons included in the moving image based on the number of head trajectories associated with each other.

The number counting method according to the second aspect of the present invention is a number counting method by a computer,
The CPU shifts the position of the head model corresponding to the size of the head expected in each part of the image when an image of a person is captured with respect to an image constituting one frame of a moving image obtained by imaging a plurality of people. While setting the process,
A step of extracting a human head included in the image based on the set head model;
A step of associating the heads extracted between the plurality of images included in the moving image with each other;
A CPU measuring the number of persons included in the moving image based on the number of head trajectories associated with each other.

The program according to the third aspect of the present invention is:
A process of inputting an image constituting one frame of a moving image obtained by imaging a plurality of people;
A process of setting the head model according to the size of the head expected in each part of the image when the person is imaged with respect to the input image while shifting the position;
A process of extracting a human head included in the image based on the set head model;
A process of associating heads extracted between a plurality of images included in a moving image,
A process of measuring the number of persons included in the moving image based on the number of head trajectories associated with each other and a computer are executed.

  According to the people counting device, method, and program according to the present invention, it is possible to count persons included in a crowd based on an image obtained by capturing the crowd.

It is a block diagram which shows the structure of the people counting device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the computer which performs the program which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the people counting device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the people counting device which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the number measuring apparatus in a 1st Example. It is a figure for demonstrating operation | movement of the people counting device in a 1st Example. It is a figure for demonstrating operation | movement of the head model setting part in a 1st Example. It is a figure which shows the imaging screen in a 1st Example as an example.

(First embodiment)
A people counting device according to a first embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the people counting apparatus according to this embodiment. Referring to FIG. 1, the people counting device 10 includes a head model setting unit 13, a head extracting unit 20, a head associating unit 16, and a people counting unit 17.

  The head model setting unit 13 inputs an image constituting one frame of a moving image in which a plurality of people are captured, and the head model expected in each part of the image when a person is captured with respect to the input image. A head model corresponding to the size is set while shifting the position. That is, the head model is set with an appropriate size for each part of the image.

  For example, the head model may be an ellipse. The moving image is preferably a moving image obtained by imaging a plurality of people from obliquely above.

  The head extraction unit 20 extracts a human head included in the image based on the set head model.

  The head association unit 16 associates the heads extracted between a plurality of images included in the moving image. Thereby, the trajectory of the head straddling between the frames can be extracted. The head associating unit 16 may associate the extracted heads whose positions are close or those whose features are close to each other.

  The number measuring unit 17 measures the number of persons included in the moving image based on the number of head trajectories associated with each other. The number-of-people counting unit 17 may measure the number of people included in the moving image based on the number of tracks satisfying a predetermined condition. For example, the predetermined condition may be a trajectory that has passed through a predetermined region during a predetermined period.

  The person counting device 10 may further include a coordinate determination unit 12. The coordinate determination unit 12 determines the correspondence between the space coordinates for the imaged space and the screen coordinates of the moving image. At this time, it is preferable that the head model setting unit 13 determines the size of the head model with reference to the determined correspondence.

  The head extraction unit 20 may further include an edge search unit 14. The edge search unit 14 extracts a head by searching for an edge in the contour of the head model. The edge search unit 14 may extract a human head only when an edge is detected in a predetermined ratio or more with respect to the contour of the head model. The head extraction unit 20 may further include a head recognition unit 15. The head recognizing unit 15 determines whether or not the extracted head is a human head based on the learning function.

  FIG. 2 is a block diagram showing a hardware configuration of a computer when the person measuring device is realized by a computer. Referring to FIG. 2, the computer 70 includes a CPU 71, a memory 72, a hard disk 73, an input device 74, and an output device 75.

  Each of these units may be connected to a bus line. The input device 74 may include a mouse and a keyboard. The output device 75 may have a display. The hard disk 73 may store a program. The CPU 71 executes processing in each unit of the person measuring device 10.

  Next, the operation of the people counting device according to the present embodiment will be described with reference to the drawings. FIG. 3 is a flowchart showing the operation of the people counting apparatus according to this embodiment.

  Referring to FIG. 3, the CPU 71 inputs an image constituting one frame of a moving image obtained by imaging a plurality of people (step S1).

  The CPU 71 sets the head model corresponding to the size of the head expected in each part of the image while shifting the position with respect to the input image (step S2).

  The CPU 71 extracts the human head included in the image based on the set head model (step S3).

  The CPU 71 associates the heads extracted between the plurality of images included in the moving image (step S4).

  The CPU 71 measures the number of people included in the moving image based on the number of head trajectories associated with each other (step S5).

(Second Embodiment)
A people counting device according to a second embodiment of the present invention will be described with reference to the drawings. FIG. 4 is a block diagram showing the configuration of the people counting apparatus according to this embodiment.

  Referring to FIG. 4, the person counting device 30 includes a moving image input unit 31, a coordinate determination unit 32, a head model setting unit 33, an edge search unit 34, a head recognition unit 35, a head association unit 36, and a person counting unit. 37. Each of these units operates as follows.

  The moving image input unit 31 inputs video from a camera or a recording device.

  The coordinate determination unit 32 calibrates the correspondence between the space imaged by the moving image input unit 31 and the coordinates in the screen.

  The head model setting unit 33 sets a head model in the screen according to the correspondence obtained by the coordinate determination unit 32.

  The edge search unit 34 searches for an edge located on the contour of the head model set by the head model setting unit 33 and estimates an initial head position that satisfies the condition.

  The head recognizing unit 35 extracts an image around the initial head position estimated by the edge search unit 34, determines whether or not the head is a person using a learning recognizer, and detects the head detection position. Confirm.

  The head associating unit 36 tracks the movement path of the person by connecting the head detection positions detected by the head recognizing unit 35 that are close to each other between the frames.

  The person counting unit 37 counts the persons in the screen from the movement route obtained by the head associating unit 36.

  Next, the overall operation of the people counting device according to this embodiment will be described. FIG. 5 is a flowchart showing the operation of the people counting apparatus according to the present embodiment.

  Referring to FIG. 5, the video input unit 31 inputs video from a camera or a recording device (step S11).

  Next, the coordinate determination unit 32 determines the angle of view for associating the captured spatial coordinate system with the screen coordinates, using each frame in the video input to the moving image input unit 31 (step S12). Here, the method for determining the angle of view is not particularly limited, and a known method may be used.

  Next, the head model setting unit 33 determines a head model to be set in the screen according to the correspondence relationship between the obtained coordinates (step S13). For example, the head model setting unit 33, when an ellipsoid corresponding to a standard head size is installed in the space and projected on the image plane, the contour is imaged on the screen. The ellipse may be a head model.

  Next, the edge search unit 34 searches for an edge located on the contour of the head model set by the head model setting unit 33, and estimates an initial head position that satisfies a predetermined condition (step S14). For example, the edge search unit 34 searches for an edge in the direction that matches the contour edge in the vicinity of each pixel in the image where the model contour exists, and a predetermined threshold value at a certain ratio or more of the contour pixels. When an edge that exceeds 1 is detected, it is estimated that a human head exists at the position in the space, and this may be used as the initial head position.

  Steps S13 and S14 are repeated while moving the position of the head model in an area where a person is considered to exist in the imaged space.

  Next, the head recognition unit 35 extracts an image around the initial head position estimated by the edge search unit 34, recognizes whether or not there is a human head near the head, and determines whether this is the head. The head detection position is determined (step S15). As an example, the head recognizing unit 35 cuts out an image of the initial head position according to the size of the head, normalizes it to a fixed size, and learns this based on a similar head image group. It may be given to the machine to determine whether it is the head. Here, the learning discriminator and the feature quantity used for discrimination are not particularly limited, and known ones can also be used. The head recognizing unit 35 determines only the initial head position determined to have a head by such processing as the head position estimation result.

  The head associating unit 36 inputs the head position estimation result obtained in each frame, and tracks the detected person by linking the positions that are close to each other and appear to be the same person (step) S16). As an example, the head associating unit 36 compares head position estimation results in adjacent frames, searches for a near-position and a feature quantity such as a color distribution obtained from an image of a person region, and The head positions determined to be the same person may be connected to each other, and this may be used as the tracking result of the person.

  The person counting unit 37 inputs the obtained tracking results and counts the detected persons (step S17). As an example, the person counting unit 37 may select and count those trajectories obtained as tracking results that satisfy a predetermined condition. Here, the predetermined condition may be, for example, a trajectory that has passed through a predetermined region during a predetermined period.

  Next, the effect of the people counting device according to the present embodiment will be described. The person counting device according to the present embodiment uses the head shape of a person who is unlikely to be concealed as a clue, and individually detects the size in the screen according to the angle of view. Therefore, according to the people counting device according to the present embodiment, it is possible to count the number of persons based on the image of the congestion situation where many persons in the public space monitoring camera or the like are concealing each other.

  The embodiment is not limited to the above, and can be implemented in various other forms. For example, in the above-described embodiment, the case of a single camera has been described. However, in an environment in which an object is photographed by a plurality of cameras, the same model is projected on each camera image and processed in parallel. You may make it do. In addition, in the situation where a large number of people are mixed, the person counting device may detect and count only people who form a line and proceed in a certain direction.

  Next, the operation of the people counting device according to the embodiment of the present invention will be described using specific examples. Here, a description will be given of an embodiment that is assumed to be applied to person group counting in a station.

  FIG. 6 is a diagram schematically illustrating an environment in which the people counting device operates in the present embodiment. Referring to FIG. 6, the camera 101 is installed so as to include such an environment in the field of view. In this embodiment, it is assumed that the moving image input unit 31 has a camera 101. Note that the number of cameras included in the moving image input unit 31 is not limited to one, and may be plural.

  The imaging area 103 is an area to be monitored, and in this embodiment, is a platform within a station. The target person 102 represents a passenger staying on the platform. The train 104 is a train that is stopped beside the platform 103.

  In such an environment, passengers get on and off the train frequently. Therefore, it is difficult to apply a method for calculating the number of people in the space based on the count at the entrance and exit.

  First, the coordinate determination unit 32 determines an angle of view that associates the position on the image plane input from the camera 101 with the world coordinate system set in the imaging region 103 (step S12).

  The coordinate determination unit 32 can use any known method as a calibration method. As an example, the coordinate determination unit 32 performs some display with a marker or the like so that at least four points on a known plane whose positions set in the space to be photographed are visible in the video of the camera 101. Shoot. From this acquisition frame, the coordinates of each point in the image are acquired. This coordinate acquisition processing is performed by the system user manually acquiring the coordinates of the display marker in the frame, or using specific colors and symbols. The displayed marker may be determined using image processing by an arbitrary method. Using the set of the known point in space and the coordinates in the screen, a projective transformation coefficient between the screen plane and the plane coordinates in space is obtained. Note that such a method is merely an example of a calibration method, and a method of associating a shooting space with an image plane is not limited to such a method.

  Next, the following processing is performed on each frame of the video obtained by photographing the environment in which the passenger is moving in the station platform by the camera 101 whose correspondence between the photographing space and the image plane is known by the above processing.

  First, the head model setting unit 33 sets a head model in the screen (step S13). Referring to FIG. 7, the head model setting unit 33 converts an ellipsoidal model corresponding to a standard head size into a head position in the space for the range 106 to be counted in the space. The height of the head is set on the image plane, and this is projected onto the image plane to determine the position where the outline of the head is imaged on the screen. FIG. 8 shows a head model projected on the screen imaged by the head model setting unit 33.

  In this image, the edge search unit 34 searches for an edge appearing in the image in the vicinity of the pixel where the contour of the head model exists. When the edge search unit 34 detects an edge that exceeds a predetermined threshold at a certain ratio or more in the contour pixels, the edge search unit 34 estimates that a human head exists at the position in the space, and this is the initial value. The head position is set (step S14). The search for the initial head position by the edge search unit 34 is repeatedly performed while moving the position of the head model in the counting target space 106.

  The head recognizing unit 35 recognizes a head image in order to confirm and narrow down whether or not the initial head position thus obtained is a person's head (step S15). The head recognizing unit 35 cuts out an image around the initial head position in accordance with the size of the head, performs normalization that transforms to a predetermined image size (for example, 64 × 64 pixels), and then performs a learning function. It is input to the equipped device, and it is determined whether or not the normalized image is the head.

  An apparatus having a learning function needs to learn separately based on a large number of head images. In addition, there is no restriction | limiting in the feature-value used for the apparatus provided with the learning function used here, and discrimination | determination, A known method can be used.

  Only when it is determined that the normalized image is the head, the head recognition unit 35 determines the initial head position as the head position estimation result. The head recognition unit 35 recognizes a head image for all of the obtained initial head positions, and obtains a plurality of head position estimation results.

  The above processing is performed independently for each frame and is repeatedly performed over a predetermined number of frames.

  The head associating unit 36 uses the head position estimation result over a plurality of continuous frames, the distance between adjacent frames tracking the target person is shorter than a predetermined threshold, and the similarity of head images between frames Are equal to or greater than a predetermined reference value, they are regarded as the same person and are connected. The degree of similarity between the head images can be, for example, a degree of overlapping color distribution frequencies in the images, but is not particularly limited thereto. The head associating unit 36 performs a connection operation for the detected head positions in the plurality of frames to obtain the tracking result of the person (step S16).

  Next, the person counting unit 37 counts the obtained tracking paths of the plurality of persons (step S17). The number-of-people counting unit 37 may count only those whose path characteristics satisfy a predetermined condition. Further, the person counting unit 37 may classify the routes and count each classified route. As an example, the person counting unit 37 may count only persons who move in a specific direction in a specific place such as a position corresponding to a train door.

  Furthermore, the number of people measured by the number-of-people measurement apparatus 30 may be output every predetermined time. Further, the measured number of people may be recorded in a storage device, and the change with time may be visualized and confirmed later.

  The number-of-people counting device, method, and program according to the present invention can be used for counting people in a station or other open space, and can also be used for analyzing a flow line in a store or a floor.

10, 30 Number measuring device 11, 31 Video input unit 12, 32 Coordinate determining unit 13, 33 Head model setting unit 14, 34 Edge searching unit 15, 35 Head recognition unit 16, 36 Head association unit 17, 37 Measurement unit 20 Head extraction unit 70 Computer 71 CPU
72 Memory 73 Hard disk 74 Input device 75 Output device 101 Camera 102 Target person 103 Imaging region 104 Train 105 Head model 106 Measurement target space 201 Imaging screen

Claims (12)

A head model corresponding to the size of the head expected in each part of the image when an image constituting a frame of a moving image obtained by imaging a plurality of people is input and the person is captured with respect to the input image A head model setting unit that sets the position while shifting the position,
Based on the set head model, a head extraction unit that extracts a human head included in the image;
A head associating unit that correlates heads extracted between a plurality of images included in the moving image;
A people counting device, comprising: a person counting unit that measures the number of people included in the moving image based on the number of head trajectories associated with each other. A coordinate determination unit for determining a correspondence relationship between the space coordinates for the imaged space and the screen coordinates of the moving image;
The number measuring device according to claim 1, wherein the head model setting unit determines the size of the head model with reference to the correspondence relationship.   The person counting device according to claim 1, wherein the head extraction unit further includes an edge search unit that extracts a head by searching for an edge in an outline of the head model. .   The person counting according to claim 3, wherein the edge search unit extracts a human head only when the edge is detected in a predetermined ratio or more with respect to a contour of the head model. apparatus.   The head extracting unit further includes a head recognizing unit that determines whether or not the extracted head is a human head based on a learning function. The person counting device according to any one of the above.   6. The person according to claim 1, wherein the head associating unit associates the extracted heads whose positions are close or whose features are close to each other. Measuring device.   The number-of-people measuring device is characterized in that the number of people included in the moving image is measured based on the number of the trajectories satisfying a predetermined condition.   The person counting apparatus according to claim 7, wherein the predetermined condition is a trajectory that has passed through a predetermined region during a predetermined period.   The person counting device according to any one of claims 1 to 8, wherein the moving image is a moving image obtained by imaging a plurality of people from obliquely above.   The person counting apparatus according to claim 1, wherein the head model is an ellipse. A method for counting people using a computer,
The CPU shifts the position of the head model corresponding to the size of the head expected in each part of the image when an image of a person is captured with respect to an image constituting one frame of a moving image obtained by imaging a plurality of people. While setting the process,
A step of extracting a human head included in the image based on the set head model;
A step of associating a head extracted between a plurality of images included in the moving image with each other;
And a step of measuring the number of persons included in the moving image based on the number of head trajectories associated with each other. A process of inputting an image constituting one frame of a moving image obtained by imaging a plurality of people;
A process of setting the head model according to the size of the head expected in each part of the image when the person is imaged with respect to the input image while shifting the position;
Based on the set head model, a process of extracting a human head included in the image;
A process of associating heads extracted between a plurality of images included in the moving image;
A program for causing a computer to execute a process of measuring the number of persons included in the moving image based on the number of head trajectories associated with each other.

Images