JP2006031153A - Person counting device and person counting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a person counting device performing accurate counting of persons even in an image with overlapping of persons, and performing the counting of persons by moving directions. <P>SOLUTION: This person counting device comprises: an image input means 10 inputting an object image; an edge image extraction means 20 extracting edge images from the object image; a person position detection means detecting person positions in the object image based on the edge images; a position tracking means 40 tracking persons who are moving in a series of continuous images containing the object image from the person positions detected by the person position detection means; and a person counting means 50 counting persons who enter and leave a predetermined counting area from the traces of the person positions tracked by the position tracking means. Matching of the person positions among frames is performed to count the persons who enter and leave the counting area. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a person counting device that calculates the number of persons entering and leaving a predetermined area by image recognition.

  In recent years, technology that automatically counts the number of people entering and exiting places such as train stations, stores, museums, and event venues has attracted attention. This is because various applications that automatically count the number of people entering and exiting a specific place and use the result are spreading. Examples of these applications include formulating train driving plans corresponding to the number of passengers at train stations, building store layouts based on the number of customers at stores, and safety measures for entertainment venues corresponding to changes in the number of visitors. There is implementation.

  Patent Document 1 discloses a technique relating to a moving body counting apparatus that counts moving bodies (that is, persons) by image recognition. According to Patent Document 1, an image captured by a depression angle photographed by a CCD camera installed on a ceiling is processed, a moving body region is extracted from a difference between frames, and an aspect ratio, an area ratio, and a peripheral surface ratio of the moving body region are extracted. The moving object is extracted, the moving object is associated between the frames, and the number of moving objects that have moved the target area is counted.

In the above-described conventional example, in counting the number of people, it is considered that the person touches, but the overlapping of the person and the separation from the overlap are not considered. For this reason, in order to perform high-accuracy counting of people, it is necessary to install a camera at a position (for example, a ceiling) where no overlapping of persons occurs in the captured image, and to shoot directly under the camera. For this reason, there is a limitation that it is not possible to count the number of people with high accuracy in a place where there is no ceiling or where a camera cannot be installed at the top. Further, in order to count the number of people in consideration of the movement direction of the person, further ideas are required.
JP-A-8-185521 JP 2003-44853 A JP 2003-173436 A JP 2002-146160 A

  SUMMARY OF THE INVENTION An object of the present invention is to provide a person counting device that can accurately count the number of people in a captured image where people overlap, and that can count the number of people in each moving direction.

  The person counting apparatus according to claim 1, wherein an image input unit that inputs a target image, an edge image extraction unit that extracts an edge image from the target image, a person position in the target image based on the edge image, From the position detection means for outputting as position information, the position tracking means for tracking the position of the person moving in the image sequence including the target image based on the position information, and the locus of the position of the person tracked by the position tracking means And a number counting means for counting the number of persons entering and leaving a predetermined counting area.

  According to this configuration, the position of a person is detected based on an image captured by a camera, the position of a person moving in a series of consecutive images is tracked, and the number of persons who have entered and exited a predetermined counting area can be accurately determined. It can be counted.

  In the person counting apparatus according to the second aspect, the image input means inputs an image composed of continuous frames as a target image, and the edge image extraction means obtains an inter-frame difference of the target image and extracts an edge image.

  According to this configuration, it is possible to detect the person position in the image using the edge image extracted from the inter-frame difference of the target image.

  In the person counting device according to claim 3, the target image is an image taken from either the horizontal direction or the diagonally upward direction, and the person position detecting unit obtains a ridge line connecting the upper end portions of the edge image, Based on this ridgeline, the position of a person in the target image is detected and used as position information.

  In the person counting device according to the fourth aspect, the person position detecting means obtains the maximal point of the ridge line, determines the obtained maximal point as the person position in the target image, and uses it as position information.

  According to these configurations, an image captured from a horizontal direction or an obliquely upward direction can be used as a target image for detecting a person position in an image, not an image captured at a depression angle. Therefore, it is not necessary to suspend the camera for photographing the target image on the ceiling, and it can be conveniently installed at the height of the line of sight on the wall, for example. Further, it is possible to obtain a ridge line from the edge image, determine the maximum point as the position of a person in the target image, and output it as position information.

  In the person counting apparatus according to claim 5, the target image is composed of images of successive frames, and the position tracking unit obtains a correlation between frames with respect to the position information output by the person position detecting unit and The movement position of the person who moves is tracked.

  7. The person counting apparatus according to claim 6, wherein the target image is composed of images of successive frames, and the position tracking means detects the movement direction of the person in the target image and holds it as movement direction information, and the current frame of the target image. Based on the position information and the movement direction information in, the movement position of the person in the next frame of the target image is predicted, and the person in the current frame of the target image is associated with the person in the next frame in consideration of the predicted movement position Thus, the movement position of the person is tracked.

  According to these configurations, since the association between the persons between the frames is performed using the predicted movement position predicted based on the position information and the movement direction information, the movement position of the person can be tracked with high accuracy and efficiency. Can be executed.

  8. The person counting apparatus according to claim 7, wherein the position tracking means determines the number of persons overlapping in the target image based on the position information and the moving direction information, and determines the number of persons released from the overlap. To do.

  According to this configuration, since the number of persons can be counted even if people overlap in the target image, it is possible to provide a person counting device that is not limited by the shooting angle of the target image. Therefore, it is not necessary to suspend the camera for photographing the target image on the ceiling, and it can be conveniently installed at the height of the line of sight on the wall, for example.

  In the person counting apparatus according to claim 8, the number counting means determines a person who has entered and exited a predetermined counting area based on the locus of the position of the person tracked by the position tracking means, and the number of persons who have entered and exited the counting area is determined. Count.

  In the person counting apparatus according to claim 9, the position tracking means detects the movement direction of the person in the target image and holds it as movement direction information, and the person counting means is based on the movement direction information held by the position tracking means. The number of persons who entered and exited a predetermined counting area is counted according to the moving direction.

  According to this configuration, it is possible to provide a highly accurate person counting device capable of counting not only the number of persons entering and exiting a predetermined counting area but also the number of persons entering and exiting according to the moving direction.

  According to the present invention, it is possible to provide a person counting device capable of accurately counting the number of people even in a photographed image where people overlap, and capable of counting the number of people in each moving direction.

  Next, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram of a person counting apparatus according to Embodiment 1 of the present invention. The person counting device of this embodiment includes an image input means 10, an edge image extraction means 20, a person position detection means 30, a position tracking means 40, a person counting means 50, an image memory 60, an input means 70, a control means 80, and a display. Means 90 are provided.

  In FIG. 1, an input means 70 comprises a keyboard and a mouse, and a user of the person counting apparatus inputs operation information of the apparatus. The display means 90 comprises an LCD or the like, and displays the target image, processing status, detection result, etc. to the user of this apparatus. The control means 80 controls each component of the person counting device.

  With reference to FIG. 1 and FIG. 10, the outline | summary of operation | movement of the person counting device of this form is demonstrated below. FIG. 10 is a flowchart of the operation of the person counting apparatus according to Embodiment 1 of the present invention.

  In step S0, the operation of the person counting apparatus of the present embodiment starts.

  In step S <b> 1, the image input unit 10 inputs a series of target images captured by a CCD camera (not shown in FIG. 1) and stores them in the image memory 60.

  In step S <b> 2, the edge image extraction unit 20 takes the difference between the previous frame and the current frame of the target image stored in the image memory 60 and extracts an edge image. Subsequently, the person position detection means 30 performs a ridge line extraction process, a filter process, a maximum point extraction process, and the like on the edge image extracted by the edge image extraction means 20 to detect a person position in the current frame.

  In step S3, the position tracking unit 40 associates the person position of the previous frame of the target image detected by the person position detection unit 30 with the person position of the current frame, and tracks the person position for each frame. For this association, a predicted movement position predicted based on position information and movement direction information given for each person is used. As a result, when person overlap occurs in the target image (hereinafter referred to as “occurrence of occlusion”) or when person overlap is resolved (hereinafter referred to as “cancellation of occlusion”). ) But you can count the number of people accurately.

  In step S4, the person counting means 50 obtains the trajectory of the movement position of the person from the tracking result of the person position for each frame performed by the position tracking means 40, and counts the number of persons entering and exiting a predetermined counting area. To do. At this time, the number of persons entering and exiting the counting area can be counted for each movement direction using the movement direction information given to each person.

  In step S5, a series of processing is terminated.

  Next, details of processing in each step will be described.

  FIG. 2 shows edge image processing, ridge line extraction processing, filter processing, and maximum point extraction processing according to Embodiment 1 of the present invention. 2A is an example of an input target image ((n) frame image) according to Embodiment 1 of the present invention. An image sequence including such a target image is input to the image input unit 10 in FIG. 1 and stored in the image memory 60.

  FIG. 2B shows an edge image extracted from the input target image according to Embodiment 1 of the present invention. Edge image extraction is performed as follows. The difference between the (n) frame image and the (n-1) frame image is calculated for each pixel to obtain a difference image A which is an inter-frame difference. Similarly, the difference between the image of the (n + 1) frame and the image of the (n) frame is calculated for each pixel to obtain a difference image B that is an interframe difference. At this time, the difference image A and the difference image B are binarized by the threshold value t1. Next, when the logical product of the difference image A and the difference image B is obtained, an edge image shown in FIG. 2B is obtained. Note that the threshold t1 is preferably set to a value such that a good edge image can be obtained.

  FIG.2 (c) shows the ridgeline extraction image calculated | required from the edge image in Embodiment 1 of this invention. In this process, the edge image shown in FIG. 2B is searched from the top in the vertical direction, and the point of the edge image obtained first is used as the ridge line, and all the points of the edge image below it are deleted. By this process, the ridgeline shown in FIG. 2C is obtained.

  FIG. 2 (d) shows the filter processing in the first embodiment of the present invention. The image of the ridge line shown in FIG. 2C is regarded as a spatial signal waveform when the horizontal direction of the screen is the x axis and the vertical direction is the y axis, and filter processing is performed on the spatial signal waveform. As the filter, a band pass filter having a pass band suitable for detection of an assumed person size is used. As a result of this processing, a smooth curve shown in FIG.

  FIG.2 (e) is a figure which shows extraction of the maximum point position in Embodiment 1 of this invention. The maximum point of the filtered spatial signal waveform in FIG. 2D is obtained, and that point is detected as the position of the person in the target image. The detected person position is described by the X coordinate and the Y coordinate of the maximum point. The X coordinate represents the position of the person in the horizontal direction of the screen, and the Y coordinate represents the apparent height of the person reflecting the position of the person in the depth direction. The X coordinate and the Y coordinate are part of the position information given to the person.

  The processing until the maximum point is obtained from the input target image described above is performed using the technique disclosed in Japanese Patent Publication No. 2003-44853 (Patent Document 2) or Japanese Patent Publication No. 2003-173436 (Patent Document 3). It can be used and executed. Therefore, further explanation is omitted here.

  Next, a person position tracking process will be described.

  FIG. 3A is an illustration of the maximum point (= person position point) of the (n−1) frame in the first embodiment of the present invention. In the (n-1) frame, person position points P1, P2, P3, and P4 are detected. FIG.3 (b) is an illustration figure of the local maximum point (= person position point) of the (n) frame in Embodiment 1 of this invention. (N) In the frame, person position points Q1, Q2, Q3, Q4, and Q5 are detected.

  FIG. 4 shows an example of the maximum point of the (n-1) frame shown in FIG. 3A and the example of the maximum point of the (n) frame shown in FIG. is there. That is, FIG. 4 is an explanatory diagram of person position tracking in the first embodiment of the present invention. A person position tracking process performed by the position tracking unit 40 of FIG. 1 will be described below with reference to FIG.

  In addition to the position information described above, each person position point is given person movement direction information obtained from the position information in the (n-1) frame and the position information in the (n) frame. Using this position information and movement direction information, the person position in (n) frame can be predicted from the person position in (n-1) frame.

  In FIG. 4, it is estimated that the person position point P1 of the (n-1) frame is located at the tip of the arrow coming from the person position point P1 in the (n) frame. In fact, there is a person position point Q1 of (n) frames in the immediate vicinity. Therefore, it is determined that the same person has moved from the person position point P1 in the (n-1) frame to the person position point Q1 in the (n) frame, and the person position point P1 and the person position point Q1 are associated with each other.

  In addition, it is estimated that the person position point P2 and the person position point P3 in the (n-1) frame are located at the tip of the arrow coming out from each point in the (n) frame. However, in the (n) frame, there is only one person position point Q2 in the vicinity thereof. This means that in the (n-1) frame, two persons observed as two separated local maxima are observed as one local maximum overlapping in the (n) frame. This is the occurrence of occlusion. Therefore, it is determined that the person position point P2 and the person position point P3 in the (n-1) frame overlap as the person position point Q1 in the (n) frame, and the person position point P2 and the person position point P3 are the person position. Corresponding to point Q2.

  The person position point P4 in the (n-1) frame is determined to have been divided into the person position point Q3 and the person position point Q4 after the occlusion is canceled in the (n) frame, and is given to the person position point P4. The position information and the moving direction information for the two persons are associated with the person position point Q3 and the person position point Q4 separately.

  (N) Another person position point Q5 observed in the frame is not associated with any local maximum point in (n-1) frame, and the position is close to the right edge of the screen. ) It is determined that the person newly detected in the frame has entered from the right end of the screen.

  As described above, according to the person counting apparatus of the present embodiment, the person position of the next frame is predicted from the person position of the current frame based on the position information and movement direction information given to each person, and The movement of the person can be accurately associated. This association can be performed accurately even when occlusion occurs or is canceled between frames. By such processing, the person counting apparatus according to the present embodiment can count the number of persons entering and leaving the counting area set in advance in the screen for each moving direction. Details thereof will be described later.

  Next, an algorithm of person position tracking and person counting performed by the person counting apparatus of this embodiment will be described.

  First, the maximal point of the ridgeline in the (n) frame of interest is the person position point Npi (i = 0 to k; k is the number of person positions observed in the (n) frame), and (n−1) in the frame. The maximum point is defined as a person position point Ppj (j = 0 to m; m is the number of person positions observed in (n−1) frames).

  Each person position point is given data having the data structure shown in FIG. That is, the data representing the person position point is registered in the X coordinate “n_x”, the Y coordinate “n_y” of the current position of the person position point, the number of persons registered in the person position point “peak_num”, and the person position point. This is the link destination “* man_link []” to the person data of the person who has been selected.

  The person registered in the person position point is given person data having the data structure shown in FIG. That is, the person data includes the X coordinate “p_x” and the Y coordinate “p_y” of the past position ((n−1) frame) of the person, the X coordinate “n_x” and the Y coordinate “n_y” of the current position ((n) frame). ”Is a flag“ first_direction ”indicating the direction in which the person entered the screen. The flag “first_direction” takes values of “0”, “1”, “2”, and “3” depending on the approach direction in the left / right and up / down directions.

  Further, the number counting means 50 is provided with a number counter for counting the number of people entering and leaving the counting area. The counting area can be set at any place in the screen. Specific examples thereof will be described later.

  FIG. 7 shows the data structure of the people counter in Embodiment 1 of the present invention. That is, the data structure of the number counter includes the number of persons “count_in_L” performed in the counting area from the left, the number of persons “count_in_R” performed in the counting area from the right, the number of persons “count_out_L” exiting the counting area from the left, and It consists of data of the number of people “count_out_R” who have left the counting area from the right.

  The person position tracking and the counting of the number of people start from searching for correspondence between persons belonging to each person position point between frames. This association is divided into the following case (a) and case (b).

  Case (a): (n−1) Correlation of person position point Ppj in frame to person position point Npi in (n) frame.

(1) A person position point existing in the screen in the (n-1) frame and in the screen in the (n) frame is searched for.

  (N-1) The person at the person position point Ppj in the frame is predicted in the frame (n) using the coordinates (p_x, p_y) of the past position of the person data and the coordinates (n_x, n_y) of the current position. A predicted movement position is obtained, a person position point Npi located within a threshold (a1x, a1y) from the predicted movement position is searched, and the closest person position point Npi is determined as a point corresponding to the person position point Ppj.

  Even if the closest person position point Npi is already associated with another person position point Ppj, association based on the currently searched result is performed, and so-called overlapping association is performed.

  (2) Although it exists in the screen in the (n-1) frame, the person position point moved to the outside of the screen is associated in the (n) frame.

  If there is no person position point Npi within the distance of the threshold (a1x, a1y) from the predicted movement position with respect to the person position point Ppj and the predicted movement position with respect to the person position point Ppj is within the threshold (a2x) from the screen edge, It is determined that the person position point in the (n) frame to be associated with the person position point Ppj has gone out of the image.

  (3) The person position point that exists in the screen in the (n-1) frame and is in the same position as the (n-1) frame in the (n) frame is also associated.

  If the predicted movement position with respect to the person position point Ppj is within the screen and is separated from the screen edge by a threshold (a2x) or more and there is no person position point Npi within the threshold (a1x, a1y) from the predicted movement position, the person position The person position point in the (n) frame to be associated with the point Ppj is determined to be stopped at the position of the (n−1) frame. As in this case, when the person is in a stopped state, the difference between frames is zero, and the maximal point of the ridge line does not appear, and therefore the person position cannot be detected in the (n) frame.

  Note that the threshold values (a1X, a1Y) and the threshold value (a2X) may be set to arbitrary values.

  Case (b): (n) For the person position point Npi in the frame (n), which is not associated in case (a), (n-1) Correlation to the person position point Ppj in the frame.

(1) Although it does not exist in the screen in the (n-1) frame, the person position point that exists in the screen in the (n) frame is associated.

  If the person position point Npi is within the threshold (b1x) from the screen edge, the person position point Npi is determined to be a new person position point that has entered the image in the (n) frame, and the person position point Ppj Is not associated.

  (2) The person position points that exist in the screen in the (n-1) frame and also exist in the screen in the (n) frame are associated.

  If there is a person position point Ppj that is overlap-associated within the threshold (b2x, b2y) from the position of the person position point Npi, the person position point Npi is separated from the person position point Ppj associated with the overlap. Determine and associate.

  (3) In the (n-1) frame, the person who has stopped in the screen associates the person position point that has started moving in the (n) frame.

  The person position point Npi is not within the threshold value (b1x) from the end of the image, and there is no person position point Ppj that is associated with the position within the threshold value (b2x, b2y) from the position of the person position point Npi. It is determined that the person who has stopped at the position of the position point Npi has started to move.

  Note that the threshold value (b1X) and the threshold values (b2x, b2y) may be set to arbitrary values.

  The setting of the counting area in the person counting apparatus of this embodiment will be described below.

  FIG. 8A shows the counting area set on the screen 101 showing the person position in the first embodiment of the present invention. This figure is a screen obtained by extracting local maximum points representing a person position from an input target screen photographed by a camera, and person position points P1 to P4 are observed. In this screen 101, the counting area is an area sandwiched between the line segment L1 and the line segment L2, and the persons entering and exiting this area are counted.

  The counting area in the actual space in which the input target screen is captured with respect to the counting area sandwiched between the line segment L1 and the line segment L2 on the screen 101 is determined by the relative positional relationship between the camera and the actual space.

  FIG. 8B shows the counting area of the hole 102 (plan view) corresponding to the counting area of FIG. In this example, the hall 102 has entrances on the left and right, and a person passing through the hall 102 through these entrances is photographed with a camera 103 installed at a height of approximately the line of sight to obtain an input target screen. The number of people passing through is counted. At this time, the line segment L1 and the line segment L2 in FIG. 8A correspond to the line segment (ab) and the line segment (dc) in FIG. 8B, respectively. That is, the counting area (L1-L2) of the screen 101 corresponds to the counting area (abcd) of the hole 102.

  In consideration of such a relationship, the counting area may be set on the screen.

  Next, with reference to FIG. 9, the person position tracking and person counting algorithm performed by the person counting apparatus of the present embodiment will be described more specifically.

  FIG. 9 is an explanatory diagram of processing accompanying the entry and exit of a person in Embodiment 1 of the present invention. In this example, the counting area is set to the same size as the screen. That is, the left and right sides of the screen are the boundaries of the counting area, and the entire screen is the counting area.

  In FIG. 9, in (n) frame, the local maximum point is not observed, and there is no person on the screen.

In the (n + 1) frame, a person position point A1 appears at the left end of the screen, and a person position point A2 appears at the right end of the screen. The coordinate positions of the person position points A1 and A2 are
A1 (A1_n + 1_X, A1_n + 1_Y)
A2 (A2_n + 1_X, A2_n + 1_Y)
And Since the person position points A1 and A2 do not have a movement source associated with each other and are within the threshold (b1X) from the screen end, it is determined that the person has newly entered the screen. Further, from the distance from both ends of the screen, it is determined that the person position point A1 has entered from the left and the person position point A2 has entered from the right.

The following person position point data is given to the person position point A1.
peak_num = 1
* Man_link [] = A1_p1
n_X = A1_n + 1_X
n_Y = A1_n + 1_Y
Further, the following data is given as the person data “A1_p1” linked to the person position point A1.
p_X = 0
p_Y = A1_n + 1_Y
first_direction = 0
Here, since the person position point A1 is a person who has entered the screen from the left side, the X coordinate of the past position is “0” which is the X coordinate of the left side of the screen, and the Y coordinate of the past position is the current position. “A1_n + 1_Y”, which is the same as the Y coordinate. Furthermore, the initial direction is “0” indicating that the vehicle has entered from the left.

The following person position point data is given to the person position point A2.
peak_num = 1
* Man_link [] = A2_p1
n_X = A2_n + 1_X
n_Y = A2_n + 1_Y
Further, the following data is given as the person data “A2_p1” linked to the person position point A2.
p_X = IMAGEX
p_Y = A2_n + 1_Y
first_direction = 1
Here, since the person position point A2 is a person who newly enters the screen from the right side, the X coordinate of the past position is “IMAGEX” that is the X coordinate of the right side of the screen, and the Y coordinate of the past position is the current position. “A2_n + 1_Y”, which is the same as the Y coordinate. Furthermore, the initial direction is “1” indicating that the vehicle has entered from the right.

  Here, since the person position point A1 has entered from the left, the person counter “count_in_L” is incremented by “1”. Similarly, since the person position point A2 has entered from the right, the person counter “count_in_R” is incremented by “1”.

  In the (n + 2) frame, the two person position points move toward the positions of the person position points B1 and B2 toward the center of the screen.

The coordinate positions of the person position points B1 and B2 are
B1 (B1_n + 2_X, B1_n + 2_Y)
B2 (B2_n + 2_X, B2_n + 2_Y)
And

  The person position points A1 and A2 in the (n + 1) frame and the person position points B1 and B2 in the (n + 2) frame are associated with each other. For this purpose, the human position point in the (n + 2) frame is predicted from the position information possessed by the person data of each person position point, and the distance is within a predetermined threshold (a1X, a1Y). ) The frame is associated with the person position point of the frame. That is, the position of the person position point A1 is linearly predicted from the coordinates (p_X, p_Y) of the past position of the person data “A1_p1” and the coordinates (n_X, n_Y) of the current position, and the point “Ap1” ( It is determined that the position is within the threshold (a1X, a1Y) from the position (a1X, a1Y) and has moved to the person position point B1 having the closest coordinates from the predicted position point. Similarly, it is determined that the person position point A2 has moved to the person position point B2 having the closest coordinates within the threshold (a1X, a1Y) from the predicted position point “Ap2”.

As a result of the above association, the following person position point data is given to the person position point B1.
peak_num = 1
* Man_link [] = B1_p1
n_X = B1_n + 2_X
n_Y = B1_n + 2_Y
Furthermore, the following data is given as the person data “B1_p1” linked to the person position point B1.
p_X = A1_n + 1_X
p_Y = A1_n + 1_Y
first_direction = 0 (inherits the value of A1)
The following person position point data is given to the person position point B2.
peak_num = 1
* Man_link [] = B2_p1
n_X = B2_n + 2_X
n_Y = B2_n + 2_Y
Furthermore, the following data is given as the person data “B2_p1” linked to the person position point B2.
p_X = A2_n + 1_X
p_Y = A2_n + 1_Y
first_direction = 1 (inherits the value of A2)
In this (n + 2) frame, the number of people counter does not change.

  In the (n + 3) frame, the two person position points overlap and appear as one person position point C1, and occlusion occurs.

The coordinate position of the person position point C1 is
C1 (C1_n + 3_X, C1_n + 3_Y)
And

  The person position points B1 and B2 in the (n + 2) frame are associated with the person position point C1 in the (n + 3) frame. This is done by predicting the person position point in the (n + 3) frame from the position information possessed by the person data of each person position point, and within the predetermined threshold value (a1X, a1Y), the (n + 3) frame. Is associated with the person position point. That is, the position of the person position point B1 is linearly predicted from the coordinates (p_X, p_Y) of the past position of the person data “B1_p1” and the coordinates (n_X, n_Y) of the current position. Similarly, the position of the person position point B is linearly predicted from the coordinates (p_X, p_Y) of the past position of the person data “B2_p1” and the coordinates (n_X, n_Y) of the current position. As a result, it is determined that the person position points B1 and B2 have both moved to the person position point C1. That is, it is determined that the person position point C1 is the one where the person position points B1 and B2 are moved and the occlusion occurs.

As a result of the above association, the following person position point data is given to the person position point C1.
peak_num = 2
* Man_link [] = C1_p1
= C1_p2
n_X = C1_n + 3_X
n_Y = C1_n + 3_Y
Here, the number of persons registered at the person position point C1 is two, and there are two links to person data. The order of the person data is determined in order of increasing value by comparing the Y coordinates of the past positions of the person data.

The following data is given as the first person data “C1_p1” linked to the person position point C1.
p_X = B2_n + 2_X
p_Y = B2_n + 2_Y
first_direction = 1
Further, the following data is given as the second person data “C1_p2” linked to the person position point C1.
p_X = B1_n + 2_X
p_Y = B1_n + 2_Y
first_direction = 0
In this (n + 3) frame, the number of people counter does not change.

In the (n + 4) frame, the occlusion is canceled and the person position point C1 where the two persons are registered has moved to the person position points D2 and D1. The coordinate positions of the person position points D2 and D1 are
D1 (D1_n + 4_X, D1_n + 4_Y)
D2 (D2_n + 4_X, D2_n + 4_Y)
And

  The person position point C1 and the person position points D1 and D2 are first associated according to the association case (a).

  Using the coordinates (n_X, n_Y) of the current position of the person position point C1 and the coordinates (p_X, p_Y) of the past position of the first person data “1C_p1”, linear prediction of the person position point in the (n + 4) frame is performed. Then, it is determined that the person position point C1 has moved to the person position point D1, which is the closest coordinate to the predicted position.

  At this point, since the person position point D2 has not been associated yet, the person position point D2 is associated with the person position point in the (n + 3) frame according to the association case (b). As a result, in the (n + 3) frame, the second person data “C1_p2” that exists in the vicinity of the person position point D2 and belongs to the person position point C1 that is the person position point (peak_num> 1) that is overlapped and associated with each other. It is determined that the person corresponding to has moved to the person position point D2.

  From the above tracking results, in this case, it is determined that the person position point C1 has moved to the person position points D1 and D2, so it is determined that the occlusion of the person has been released.

The following person position point data is given to the person position point D1.
peak_num = 1
* Man_link [] = D1_p1
n_X = D1_n + 4_X
n_Y = D1_n + 4_Y
Further, the following data is given as the person data “D1_p1” linked to the person position point D1.
p_X = B1_n + 2_X
p_Y = B1_n + 2_Y
first_direction = 1
Here, since the person position point C1 of the previous frame corresponding to the person position point D1 is the person position point (peak_num> 1) that is overlap-associated, the coordinates of the past position of the person position point D1 are not updated.

The following person position point data is given to the person position point D2.
peek_num = 1
* Man_link [] = D2_p1
n_X = D2_n + 4_X
n_Y = D2_n + 4_Y
Further, the following data is given as the person data “D2_p1” linked to the person position point D2.
p_X = B1_n + 2_X
p_Y = B1_n + 2_Y
first_direction = 0
In this (n + 4) frame, the number of people counter does not change.

In the (n + 5) frame, the person position point D2 entering from the left end of the screen leaves the right end of the screen, and the person position point D1 entering from the right end of the screen moves to the person position point E1. The coordinates of the person position point E1 are
E1 (E1_n + 5_X, E1_n + 5_Y)
And

  The person position points D1 and D2 are associated with the person position point E1. From the coordinates (p_X, p_Y) of the past position of the person data “D1_p1” and the coordinates (n_X, n_Y) of the current position, the position of the person is linearly predicted, and the person position E1 at the coordinates closest to the predicted position It is determined that the point D1 has moved. Position prediction is performed linearly from the coordinates (p_X, p_Y) of the past position of the person data “D2_p1” and the coordinates (n_X, n_Y) of the current position, and the person position is within the threshold (a1X, a1Y) from the predicted position. Since there is no point E1 and the right edge of the screen is within the threshold (a2X), it is determined that the person position point D2 has gone out of the screen.

As a result of the above association, the following person position point data is given to the person position point E1.
peak_num = 1
* Man_link [] = E1_p1
n_X = E1_n + 5_X
n_Y = E1_n + 5_Y
Further, the following data is given as the person data “E1_p1” linked to the person position point E1.
p_X = D1_n + 4_X
p_Y = D1_n + 4_Y
first_direction = 1
Here, since the person position point D2 has come out from the right end of the screen, the “count_out_R” of the person counter is incremented by “1”.

  In the (n + 6) frame, the person position point that has entered from the right end of the screen also leaves the left end of the screen, and there is no person on the screen.

  Again, tracking is associated with the person position point E1, but the predicted position in the (n + 6) frame from the person position point E1 is within the threshold (a2x) from the left end of the screen. The position point E1 is determined to have come out from the left end of the screen in the (n + 6) frame.

  Here, since the person position point E1 has come out from the left end of the screen, the “count_out_L” of the person counter is incremented by “1”.

Thus, a series of person position tracking is completed. By these processes, each variable of the people counter becomes
count_in_L = 1
count_in_R = 1
count_out_L = 1
count_out_R = 1
It can be seen that one person entered from the right edge of the screen, one person entered from the left edge of the screen, one person from the right edge of the screen, and one person from the left edge of the screen.

  As described above, according to the person counting apparatus of the present embodiment, for example, a target image is taken from a camera installed at the height of the line of sight on the wall, and even if people overlap in the target image, Not only can the number of persons entering and exiting a predetermined counting area be accurately counted, but the number of persons entering and exiting can also be counted by moving direction.

  The gist of the present invention is to realize a person counting device capable of obtaining an accurate number coefficient even if people overlap, and various applications are possible without departing from the gist of the present invention.

  The person counting apparatus according to the present invention can be used in, for example, a person counting apparatus that counts the number of persons who enter and leave a region by image recognition and its application fields.

Block diagram of person counting device in Embodiment 1 of the present invention (A) Input target image in the first embodiment of the present invention (b) Edge image extracted from the input target image in the first embodiment of the present invention (c) Edge line obtained from the edge image in the first embodiment of the present invention Extracted image (d) Filter processing in Embodiment 1 of the present invention (e) Extraction of local maximum point in Embodiment 1 of the present invention (A) Example of (n-1) frame maximum point (person position point) in Embodiment 1 of the present invention (b) (n) Frame maximum point (person position point) in Embodiment 1 of the present invention ) Explanatory drawing of person position tracking in Embodiment 1 of the present invention Data structure of person position point in Embodiment 1 of the present invention Data structure of person data in Embodiment 1 of the present invention Data structure of number counter in embodiment 1 of the present invention (A) Counting area set on screen showing person position in Embodiment 1 of the present invention (b) Counting area on corresponding hole plan view Explanatory drawing of the process with the entrance and exit of the person in Embodiment 1 of this invention Flowchart of operation of person counting apparatus in embodiment 1 of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image input means 20 Edge image extraction means 30 Person position detection means 40 Position tracking means 50 People counting means 60 Image memory 60
70 Input means 70
80 Control means 80
90 display means 90

Claims (18)

An image input means for inputting a target image;
Edge image extraction means for extracting an edge image from the target image;
A person position detecting means for detecting a person position in the target image based on the edge image and outputting as position information;
Position tracking means for tracking the position of a person who moves in an image sequence including the target image based on the position information;
A person counting device comprising: a person counting means for counting the number of people who entered and exited a predetermined counting area from the locus of the position of the person tracked by the position tracking means. The person counting apparatus according to claim 1, wherein the image input unit inputs an image composed of continuous frames as the target image, and the edge image extraction unit obtains an inter-frame difference of the target image and extracts an edge image. . The target image is an image taken from one of a horizontal direction and a diagonally upward direction, and the person position detection unit obtains a ridge line connecting the upper end portions of the edge image, and based on the ridge line, the The person counting apparatus according to claim 1, wherein a person position in a target image is detected and used as the position information. The person counting device according to claim 3, wherein the person position detecting unit obtains a local maximum point of the ridge line, determines the obtained local maximum point as a human position in the target image, and uses it as the positional information. The target image is composed of images of consecutive frames, and the position tracking unit obtains a correlation between frames for the position information output by the person position detecting unit, and moves a person who moves in the images of consecutive frames. The person counting apparatus according to claim 1, wherein the position is tracked. The target image is composed of images of successive frames, and the position tracking means detects a movement direction of a person in the target image and holds it as movement direction information, and the position information in the current frame of the target image and the Predicting the movement position of the person in the next frame of the target image based on the movement direction information, and considering the predicted movement position and associating the person in the current frame of the target image with the person in the next frame The person counting device according to claim 1, wherein the movement position of the person is tracked. The person according to claim 6, wherein the position tracking unit determines the number of persons overlapping in the target image based on the position information and the movement direction information, and determines the number of persons released from the overlap. Counting device. 2. The person counting unit according to claim 1, wherein the person counting unit determines a person who has entered and exited a predetermined counting area based on a locus of the position of the person tracked by the position tracking unit, and counts the number of persons who have entered and exited the counting area. Person counting device. The position tracking unit detects a moving direction of a person in the target image and holds it as moving direction information, and the number counting unit counts a predetermined number based on the moving direction information held by the position tracking unit. The person counting apparatus according to claim 1, wherein the number of persons entering and exiting the area is counted by moving direction. An image input step for inputting a target image;
An edge image extraction step of extracting an edge image from the target image;
A person position detecting step for detecting a person position in the target image based on the edge image and outputting the position information as position information;
A position tracking step for tracking the position of a person moving in an image sequence including the target image based on the position information;
A person counting method including a person counting step of counting the number of persons entering and leaving a predetermined counting area from the locus of the position of the person tracked in the position tracking step. The person counting method according to claim 10, wherein in the image input step, an image composed of continuous frames is input as the target image, an inter-frame difference of the target image is obtained, and an edge image is extracted. In the edge image extraction step, the edge image is obtained from a target image taken from either a horizontal direction or an obliquely upward direction, and in the person position detection step, based on a ridge line connecting the upper end portions of the edge image. The person counting method according to claim 10, wherein a person position in the target image is detected and used as the position information. The target image is composed of images of continuous frames, and in the person position detecting step, a local maximum point of the ridge line is obtained, the obtained local maximum point is determined as a human position in the target image, and is used as the position information. Item 12. The person counting method according to Item 12. The target image is composed of images of successive frames. In the position tracking step, a correlation between frames is obtained for the position information output in the person position detection step, and the movement of a person moving in the images of successive frames is obtained. The person counting method according to claim 10, wherein the position is tracked. The target image is composed of images of successive frames. In the position tracking step, the movement direction of the person in the target image is detected and held as movement direction information, and the position information in the current frame of the target image and the Predicting the movement position of the person in the next frame of the target image based on the movement direction information, and considering the predicted movement position and associating the person in the current frame of the target image with the person in the next frame The person counting method according to claim 10, wherein the movement position of the person is tracked. 16. The person according to claim 15, wherein in the position tracking step, the number of overlapping persons in the target image is determined based on the position information and the moving direction information, and the number of persons released from the overlapping is determined. Counting method. 11. The person counting step according to claim 10, wherein in the number counting step, a person who enters and exits a predetermined counting area is determined based on a trajectory of the position of the person tracked in the position tracking step, and the number of persons who enter and leave the counting area is counted. Person counting method. In the position tracking step, the movement direction of the person in the target image is detected and held as movement direction information, and in the number-of-persons counting step, based on the movement direction information, The person counting method according to claim 10, wherein the number of persons is counted for each moving direction.

Images