JP2011039792A - Counter for the number of persons - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately count the number of persons by highly accurate person recognition and person tracking. <P>SOLUTION: A counter for the number of persons includes: a measurement part 106 for measuring a distance from a camera up to a measurement target for each pixel; a storage part 107 for storing three-dimensional data including distance information in which a pixel position is related to the distance when mapping a pixel on a plane on which the measurement target is grounded and distance information indicating a distance in a height direction vertical to the plane on the pixel position for each pixel; a clustering part 115 for extracting and clustering only three-dimensional data including the distance information within a threshold from the three-dimensional data and obtaining a person candidate area for each sampling frame; a tracking part 117 for determining the same person candidate area based on correlation value among a plurality of person candidate areas obtained for each sampling frame and tracking a moving locus in the same person candidate area; and a counting part 118 which, when the person candidate area passes a pixel area previously set larger than the person candidate area, counts the number of passages as the number of passed persons. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a number counter that detects, for example, the number of persons passing through an entrance based on captured image information of a camera.

  A conventional number counter counts the number of people by attaching a camera at the entrance of a room and performing person recognition using image processing from the captured image information (see, for example, Patent Document 1). In addition, a person is recognized by a camera to track a person, and a backward person or a staying person is detected (see, for example, Patent Document 2).

Japanese Patent Laid-Open No. 10-49718 JP 2008-197946 A

  However, the conventional number counter generally uses a luminance camera as a camera. Luminance cameras generate images by focusing the illuminated natural light subject image with a lens and forming it on the light receiving surface of an area sensor such as a CCD or CMOS sensor. Information obtained from each pixel is It becomes the luminance information incident on each pixel. For this reason, in the conventional number counter, when recognizing a person, it is difficult to separate the person and the background if the clothes of the person have a low contrast compared to the background such as the floor. Can not do it.

  The present invention has been made to solve the above-described problems, and enables high-precision person recognition and person tracking as compared with the case of using luminance information, thereby counting the number of persons with high accuracy. The purpose is to provide a people counter.

  In order to solve the above-described problem, the number counter according to the present invention measures the distance from the camera to the measurement target for each pixel in the detection area of the camera and for each pixel in the detection area obtained by the camera. A first distance information associating the pixel position and the distance when the pixel is mapped onto a plane on which the measurement object is grounded, and a distance in a height direction perpendicular to the plane at the pixel position. Only the three-dimensional data including the second distance information within the threshold is extracted from the storage unit storing the three-dimensional data including the second distance information shown for each pixel and the three-dimensional data stored in the storage unit. A clustering unit that clusters the extracted data and obtains a human candidate region representing a pixel region including the measurement target for each sampling frame; and the clustering unit A tracking unit that determines the same human candidate region based on a correlation value between a plurality of human candidate regions obtained for each sampling frame and tracks a movement trajectory of the same human candidate region, and a tracking result of the tracking unit And a counting unit that counts the number of passing persons as the number of passing persons when the person candidate area passes through a pixel area set in advance larger than the person candidate area.

  According to the number counter of the present invention, even when the brightness information cannot be identified by the method using the brightness information, such as when the brightness of the person and the background is the same, the measurement object is identified regardless of the brightness of the measurement object by using the distance information. It is possible to count the number of people with high accuracy by more accurate person recognition and person tracking.

The block diagram which shows the number of people counter which concerns on this embodiment. The flowchart which shows operation | movement of the people counter which concerns on this embodiment. The figure which shows the detection area which concerns on this embodiment. The figure which shows an example of person data. The figure which shows the count area | region of a number of people. The figure which shows an example of tracking data. The figure which shows the movement vector of a person candidate area | region. The flowchart which shows the operation | movement of the person detection which concerns on this embodiment. The flowchart which shows the operation | movement of clustering of a person candidate area | region. The figure which shows the person candidate area | region extracted with the height threshold value in the detection area. The figure which shows the person candidate area | region which mapped and clustered to XY plane based on the height threshold value. The figure explaining the maximum height and predetermined distance D of a person candidate area | region. The figure which shows the person candidate area | region extracted by the distance D in the detection area. The figure which shows the person candidate area | region which mapped and clustered to XY plane based on the predetermined distance D. FIG. The flowchart which shows the operation | movement of the person tracking which concerns on this embodiment. The figure which shows an example of a correlation table. The flowchart which shows the operation | movement of the human feature extraction which concerns on 2nd Embodiment. The figure which shows the case where the measurement object contained in a person candidate area | region is a person. The figure which shows the case where the measuring object contained in a person candidate area | region is a small animal. The figure which shows the case where the measurement object contained in a person candidate area | region is a person and a package.

Hereinafter, some embodiments of a people counter according to the present invention will be described in detail with reference to the drawings. In each embodiment, the same reference numerals are assigned to the same parts, and repeated description is omitted.
(First embodiment)
The configuration of the people counter according to the first embodiment of the present invention will be described in detail with reference to FIG.
The number counter 100 according to the present embodiment includes a laser light emission unit 101, a pulse control unit 102, a laser light reception unit 103, a phase detection unit 104, an A / D conversion unit 105, a distance measurement unit 106, and a distance memory. 107, amplitude detection unit 108, A / D conversion unit 109, luminance measurement unit 110, luminance memory 111, mirror 112, X axis control unit 113, Y axis control unit 114, clustering unit 115, The measurement object detection unit 116, the measurement object tracking unit 117, and the number of people counting unit 118 are included.
The laser emission unit 101, the pulse control unit 102, the laser light receiving unit 103, the phase detection unit 104, the A / D conversion unit 105, the distance measurement unit 106, the distance memory 107, and the amplitude detection unit 108 The A / D conversion unit 109, the luminance measurement unit 110, the luminance memory 111, the mirror 112, the X-axis control unit 113, and the Y-axis control unit 114 are collectively referred to as a distance camera.

The pulse control unit 102 generates a pulse-shaped control signal to emit light from the laser light emitting unit 101, and sends the control signal to the laser light emitting unit 101 and the phase detection unit 104.
The laser light emitting unit 101 includes an LED (Light-Emitting Diode) or a laser diode, receives a control signal from the pulse control unit 102, and generates light in pulses. The laser light emitting unit 101 may be a laser that mainly generates infrared light, but is not limited to these LEDs and laser diodes, and may be other light emitting elements.

  The laser light receiving unit 103 receives reflected light from a subject that is a measurement target of light emitted from the laser light emitting unit 101, and obtains a pulsed analog signal (hereinafter also referred to as a reception signal) by photoelectric conversion. Further, as a light receiving element used in the laser light receiving unit 103, for example, a PD (Photo Diode) is conceivable.

  The phase detection unit 104 receives a control signal from the pulse control unit 102 and a reception signal from the laser light receiving unit 103. Then, the phase difference between the received control signal and the received signal is detected, and a phase signal representing the detected phase difference is obtained. The phase signal is composed of an analog signal represented by a DC level.

The A / D converter 105 receives the phase signal from the phase detector 104, digitizes it, and converts it into a digital signal. Analog-to-digital conversion may be performed using a commonly used technique, and detailed description thereof is omitted here.
The distance measurement unit 106 receives the digital signal from the A / D conversion unit 105 and calculates distance information that is information indicating the distance from the distance camera to the measurement target. The distance information can be calculated as follows. First, the digital signal is converted into a numerical value proportional to twice the time it takes for the laser to reach the object to be measured. Next, the value obtained by multiplying this value by the speed of light and dividing by 2 is the distance information.

  The distance memory 107 receives distance information from the distance measurement unit 106. At the same time, X-axis control information and Y-axis control information, which are information indicating pixel positions when mapped to XY plane coordinates with a certain pixel as a reference, are transmitted from an X-axis control unit 113 and a Y-axis control unit 114 described later. receive. Then, the received distance information is stored in association with the X-axis control information and the Y-axis control information. The XY plane coordinates are planes parallel to the floor surface, which is the plane on which the measurement target is grounded. Here, the distance memory 107 is, for example, a RAM, a flash memory, a ROM, an EPROM, an EEPROM, a register, a hard disk, a removable disk, or a CD-ROM. Note that the present invention is not limited to these examples, and any storage medium known in the technical field may be used.

The amplitude detector 108 receives the received signal from the laser light receiver 103, detects the maximum amplitude of the pulse of the received signal, and obtains an analog signal representing the detected value.
The A / D converter 109 receives an analog signal representing the detected value of the maximum amplitude from the amplitude detector 108, digitizes it, and converts it into a digital signal.

The luminance measurement unit 110 receives the digital signal from the A / D conversion unit 109, and calculates luminance information that is information regarding the luminance of the measurement target.
The luminance memory 111 receives luminance information from the luminance measuring unit 110 and also receives X-axis control information and Y-axis control information from the X-axis control unit 113 and the Y-axis control unit 114, and the received luminance information and X-axis control information. And Y-axis control information are stored in association with each other. The luminance memory 111 may be any storage medium known in the technical field, like the distance memory 107.

The mirror 112 reflects the laser emitted from the laser light emitting unit 101 and adjusts the irradiation angle of the laser so as to correspond to all pixel positions (hereinafter also referred to as detection areas) that can be imaged by the camera. The adjustment of the irradiation angle is performed by receiving axis control information from an X-axis control unit 113 and a Y-axis control unit 114 described later.
The X-axis control unit 113 and the Y-axis control unit 114 generate X-axis control information and Y-axis control information for performing angle control corresponding to the detection area on the mirror 112, respectively. Further, the X axis control unit 113 and the Y axis control unit 114 send the X axis control information and the Y axis control information to the distance memory 107 and the luminance memory 111, respectively. The X-axis control information and the Y-axis control information are represented as (X, Y) = (2000, 1000), for example, assuming that the pixels in the detection area are points in the XY plane coordinates.

The clustering unit 115 receives distance information and luminance information at each pixel position from the distance memory 107 and the luminance memory 111, performs clustering for setting a human candidate region for the measurement target, and sets the region start point and region end point. Ask. The human candidate area is a pixel area that includes a measurement object that is estimated to be a person and that is larger than the pixel area that represents the measurement object.
Note that the size of the pixel area of the human candidate area may be an arbitrary pixel area as long as it includes the entire pixel area representing the measurement target. The region start point and the region end point are pixel positions that define a human candidate region. For example, on the XY plane coordinates, when the region start point is (X, Y) = (2000, 1000) and the region end point is (X, Y) = (2300, 1100), the human candidate region is 300 × 100. Represents a rectangular area. Here, the human candidate area is specified by XY plane coordinates, but the human candidate area may be specified by three-dimensional space coordinates. In this case, the region start point and region end point are represented by three parameters. The clustering method of the human candidate area will be described later with reference to FIGS.

The measurement target detection unit 116 receives the region start point and the region end point from the clustering unit 115, performs human detection as a measurement target determination from the human candidate region, and obtains person data for specifying a person.
The measurement object tracking unit 117 performs person tracking by detecting the person data obtained by the measurement object detection unit 116 in time series, and creates tracking data, which is data representing the movement trajectory of the person, for the detected number of people.

  The number counting unit 118 receives tracking data from the measurement target tracking unit 117 for each number of people, and counts the number of people who have passed a predetermined count area. The count area is an arbitrary rectangular area within the detection area. Based on the tracking data of the coordinates of the human candidate area, if the human candidate area passes through this rectangular area, the number of passing people is counted as one person.

Next, the operation of the people counter 100 will be described in detail with reference to the flowchart of FIG. As an example of the detection area in the present embodiment, the operation of the number counter 100 will be described assuming that the area shown in FIG. 3 is sampled as the detection area.
In FIG. 3, a person A, a person B, and a person C to be measured exist in a detection area in three-dimensional space coordinates. In this way, when a person stands upright on the floor surface, the distance from the pixel position where the person is in contact with the floor surface in the height direction (also referred to as the Z-axis direction as shown in FIG. 3) It is calculated using the distance information stored in the distance memory 107. Since the method of calculating the distance in the height direction is a general method, a detailed description thereof is omitted here. Hereinafter, information in which distance information is associated with a distance in the height direction calculated from the pixel position and the pixel position is referred to as three-dimensional data.

First, in step S201, three-dimensional data related to a sampling frame at a certain sampling time is acquired from the distance memory 107.
Subsequently, in step S202, the measurement target detection unit 116 detects a human candidate area representing an area estimated to be a person in the detection area based on the three-dimensional data, and the person data 1 representing the result is detected. Generate. Further, the detected number of people is Nh1. The person detection method performed in step S202 will be described in detail later with reference to the flowcharts of FIGS.
In step S203, the person data 1 is acquired and a list is generated. The list may be stored in the measurement target detection unit 116 or may be stored in an external memory.

Here, an example of a list of person data will be described with reference to FIG.
Lists the person index, the maximum height of the area where the person is detected, the maximum height coordinate, the time, the start point of the area representing the human candidate area, and the end point of the area representing the human candidate area as person data relating to one person Is generated. In FIG. 4, person data of three persons is generated. For example, the person data having an index of 1 has a maximum height (approximate height) of 175 cm, and coordinates where the maximum height is located (1050, 2250). The detected time is “14:10:32”, the region start point representing the human candidate region is (1000, 2000), and the region end point is (1100, 2500). In this case, the human candidate area is 100 × 500. Such person data is generated for the detected number Nh1.

In step S204, it is determined whether or not at least one person has been detected among the detected number Nh1, and if a person is detected, the process proceeds to step S205. If no person has been detected, the process returns to step S201, and is described above. The processing from step S201 to step S204 is repeated.
In step S205, the person counter N is initialized to “0”.

Next, the processing from step S206 to step S208 is the same as the processing from step S201 to step S203. That is, three-dimensional data related to a sampling frame at the next sampling time is acquired from the distance memory 107, a person candidate area is detected, and person data 2 representing the result is generated for the number of detected persons Nh2, thereby generating a list.
In step S209, it is determined whether or not at least one person has been detected among the detected number Nh2. If a person has been detected, the process proceeds to step S211. If no person has been detected, the tracking data is set to “0” in step S210. ”Is then returned to step S201, and the processing from step S201 to step S209 is repeated thereafter.

Next, in step S211, the movement of the person is traced from the correlation value between the person data 1 and the person data 2. Detailed operation of step S211 will be described later in detail with reference to FIG.
In step S212, it is determined whether or not a person has passed the count area. In the pass determination, if a candidate person area outside the count area enters the count area and the person candidate area moves out of the count area again while maintaining the approach direction, it is determined as passing. If it is determined that the person has passed the count area, the process proceeds to step S213, where the person counter N is incremented by one in the person counting unit 118. If it is determined that the person has not passed the count area, the process proceeds to step S214.

An example of the passage determination will be described in detail with reference to FIG.
FIG. 5 is a diagram in which the pixels in the detection area are expressed in XY plane coordinates, and the three-dimensional space coordinates shown in FIG. 3 are viewed from above (Z = 0). In the drawing, an arbitrary rectangular area is provided in the detection area, and the human candidate area moves from the upper coordinates outside the count area to the lower coordinates outside the count area through the count area. This state is determined that the person has passed the rectangular area. In the example of FIG. 5, the detection area is considered in the XY plane coordinates, but the count area may be provided in a three-dimensional space.

In step S214, tracking data including the coordinates of the human candidate area is stored. An example of the tracking data will be described in detail with reference to FIGS. As shown in FIG. 6, the parameters shown in FIG. 4 are recorded in the tracking data in time series. Further, when the measurement target tracking unit 117 is tracking the same person, that is, a person having the same person index, the tracking flag is set to “1”. On the other hand, when the same person is not tracked, the tracking flag is set to “0”.
In the example of FIG. 6, since the person 1 and the person 2 are in a tracking state, the tracking flag is set to “1”, and the tracking flag is set to “0” on the assumption that the person 3 is not tracking. ing. From this tracking data, the movement trajectory of the person can be expressed as shown in FIG. Further, it is possible to obtain a person's movement vector from the approximate straight line of the movement locus, and it is also possible to calculate information as to which direction outside the detection area and at what speed.

In step S215, based on the tracking data shown in FIG. 6, results such as the number of people count, the tracking trajectory, and the moving direction are output.
Finally, in step S216, the person data 1 acquired in step S203 is replaced with the person data 2 acquired in step S208 and updated. Then, in order to obtain the next person data 2 to be compared with the person data 1 from the new sampling frame, the process returns to step S206, and the same processing from step S206 to step S216 is repeated. The operation of the people counter 100 in the present embodiment described above continues until the operation is forcibly ended, for example, until the power of the people counter 100 is turned off.

Next, the flow of human detection in step S202 will be described in detail with reference to the flowchart of FIG.
First, in step S801, a person candidate area Idx that is a variable indicating an index of the person candidate area is initialized to “1”, and a counter Nh that counts persons in the detection area is initialized to “0”. The value of Nh indicates the final number of detected people.
In step S802, clustering is performed in order to group the measurement targets as human candidate regions. Clustering of human candidate areas is a process necessary for performing human detection, and is performed to distinguish a person from other areas.

Here, the clustering method of candidate areas in step S802 will be described in detail using the flowchart shown in FIG.
First, in step S901, three-dimensional data having a height threshold LL or more is extracted from the distance information of the three-dimensional data.
Next, in step S902, the three-dimensional data in the height range equal to or higher than the threshold extracted in step S901 is mapped to plane coordinates with the floor plane as a reference. That is, those whose height from the floor is less than the height threshold LL are excluded from the measurement target.

  Further, in step S903, all the pixels are classified into binary values depending on whether or not there is three-dimensional data having distance information higher than the height threshold LL at the coordinates corresponding to each pixel. For example, if the distance information of a certain pixel is the same distance information as the height threshold LL, the pixel is set to “0”, and if the distance information is higher than the height threshold LL, it is set to “1”. Set. In this way, the pixels classified into the binary values are collected to generate a rough human candidate region. The generation of the rough candidate area in step S903 is referred to as first clustering here. Further, the total number of human candidate regions generated at this time is Nmax. For example, when there are three person candidate regions, Nmax is “3”. When the candidate human area is detected, an index is assigned to the candidate human area Idx in order from 1.

In step S904, one candidate candidate area Idx processed in step S903 is extracted.
In step S905, the maximum height Lmax that is the maximum height is detected in the three-dimensional data of the candidate area extracted in step S904.
In step S906, Lmin lower by a predetermined distance D is calculated from the maximum height Lmax with respect to the human candidate region. In addition, what is necessary is just to set the distance D so that the upper body part of a person can be extracted, for example. That is, the length of the distance D may be adjusted in proportion to the maximum height Lmax.

In step S907, the three-dimensional data of the human candidate region is extracted in the height range from Lmax to Lmin.
In step S908, the data extracted in step S907 is mapped onto the floor surface.
In step S909, the second clustering is performed in which all pixels are classified into binary values depending on whether or not there is three-dimensional data having distance information higher than Lmin, and the classified pixels are collected to generate a human candidate region again.

In step S910, the human candidate region Idx obtained by these processes is stored.
In step S911, the index of the person candidate area Idx is incremented.
In step S912, it is determined whether or not the index of the human candidate area Idx is larger than the total number Nmax of human candidate areas. If Idx is equal to or smaller than Nmax, the process returns to step S904, and the processes from step S904 to step S910 are repeated. . If Idx is greater than Nmax, it means that the processing from step S904 to step S909 has been performed for all the human candidate regions clustered in step S903, and thus the processing of clustering the human candidate regions is terminated.

Here, the clustering process of the human candidate area will be described in detail with reference to FIGS.
FIG. 10 shows the result of extracting the height threshold value LL or more in the height direction with respect to the three-dimensional data area. As an example, the height threshold LL is set so that the height above the knee of the person can be extracted.

FIG. 11 shows a diagram in which the extracted person candidate regions are mapped to plane coordinates with the floor plane as a reference. FIG. 4 is a diagram in which the three-dimensional space coordinates of FIG. 3 are mapped onto XY plane coordinates, and each pixel represented by the XY plane coordinates has distance information in the height direction at the pixel position. That is, the hatched portion is a pixel region having a distance difference from the height threshold LL obtained from the three-dimensional data, and is a portion to be measured. Further, a rectangular area indicated by a broken line is a human candidate area, and an area of a pixel that includes the measurement target and is larger than the measurement target is taken. This is because the shape of the measurement object varies depending on the sampling frame depending on the operation of the measurement object. If only the shape of one sampling frame (for example, the shaded portion in FIG. 11) is used as a reference, it is the same in the other sampling frames. This is because it is difficult to determine that the person is a person.
Next, as shown in FIG. 12, one person candidate area is extracted, the maximum height Lmax is searched in the height direction of the person candidate area, and Lmin which is lower by a distance D in the height direction from Lmax is calculated. Furthermore, Lmin is calculated for each candidate region.
FIG. 13 shows a diagram in which three-dimensional data is extracted in the range of Lmax and Lmin for each person shown in FIG. The candidate area is set again using Lmin as the plane reference (the plane in the height direction Z = 0). By setting the distance D in this way, even if the height of the person is different, data can be extracted at a height that is lower than the head of the person by the relative distance D, and the influence of the difference in height can be eliminated.

FIG. 14 is a diagram in which this human candidate region is mapped onto a plane based on the height of Lmin.
Only the first clustering process shown in FIG. 11 does not unify the shape of the hatched portion for each measurement object by detecting distance information such as legs and arms, but by performing the second clustering process shown in FIG. The distance D can be set from the head to the chest for each person, and the influence of detecting distance information such as legs and arms can be reduced to obtain a unified shape of the measurement object.

Here, returning to step S803 in FIG. 8, the candidate human region Idx after the clustering process of the candidate human region is performed in step S802.
In step S804, it is determined whether the area of the candidate area is larger than a preset minimum area Smin. The value of the area Smin is set, for example, by calculating an average value from each person candidate area based on an area obtained by mapping the person candidate area. If the area of the candidate area is smaller than the area Smin, the process proceeds to step S809. On the other hand, when the area of the candidate area is larger than the area Smin, the process proceeds to step S805.

  In step S805, human feature detection is performed on the human candidate area on the assumption that the extracted human candidate area is likely to be a person, and the human candidate area is determined to be a person. As a determination method, for example, there is a pixel area having an average maximum height Lmax at the center of the human candidate area, and a peripheral pixel area having a height higher than Lmax is centered on the pixel area having the maximum height Lmax. Assume that it is determined that the distance information is low and symmetrical. In this case, it can be determined that the measurement target included in the human candidate region corresponding to the pixel having Lmax corresponding to the head and the peripheral pixel corresponding to the shoulder is a person.

In step S806, it is determined whether or not the human feature is detected in step S805. If the human feature cannot be detected, the process proceeds to step S809. On the other hand, if the human feature is detected, the process proceeds to step S807, and the number of people counter Nh is incremented.
In step S808, since it is determined that the human candidate area Idx is a person by the human feature detection, the human candidate area Idx is determined as the person Idx.

In step S809, it is determined that the extracted person candidate area is not a person, and the person candidate area Idx is deleted.
In step S810, the person candidate area Idx is incremented.
In step S811, it is determined whether the human candidate area Idx is larger than the human candidate area number Nmax. If the human candidate area Idx is equal to or less than Nmax, the process returns to step S803, and the processes from step S803 to step S811 are repeated. On the other hand, if the candidate person area Idx is larger than Nmax, the process proceeds to step S812 and person data is stored. The person detection process is completed by the above steps.

Next, the processing operation of the person tracking (step S211) shown in FIG. 2 will be described in detail with reference to the flowchart of FIG.
First, in step S1501, the persons Idx1 and Idx2 detected in steps S202 and S207 of FIG. 2 are initialized, and initial values of the persons Idx1 and Idx2 are set to “1”.
Next, in step S1502, the person Idx1 of the person data 1 is extracted.

In step S1503, a variable V that stores the correlation value between the person Idx1 and the person Idx2, a variable Vmax that stores the maximum correlation value, and a variable ManIdx that stores the index of the person in the person data 2 are prepared and initialized to “0”. Turn into.
In step S1504, the person index is extracted one by one from the person Idx2 of the person data 2.

In step S1505, the correlation value between the person Idx1 and the person Idx2 is stored in V. For the correlation value, there are a method of taking a correlation value with the shape of a candidate region separated when clustering of candidate candidates, a method using three-dimensional data extracted with the maximum heights Lmax and Lmin in the region, and the like. can give. For example, there is a normalized correlation method as a method for obtaining a correlation value from the shape between human candidate regions. In the matching by the normalized correlation, the person candidate area of the person Idx1 acquired in the next sampling frame is shifted by one pixel within the detection area, and the person candidate area of one person index of the person Idx2 acquired in the next sampling frame and the mutual distance information The correlation value is calculated, and the maximum correlation value is obtained assuming that the human candidate region exists at the pixel position where the maximum correlation value is obtained.
Since not only the correlation value of the distance information but also the brightness information stored in the brightness memory of FIG. 1 can be used, the brightness data can be output. Therefore, the above-described density data corresponding to the human candidate area is used. The correlation value may be calculated by a normalized correlation method.
In step S1506, it is determined whether or not the correlation value V is greater than Vmax. If V is greater than Vmax, the process proceeds to step S1507. If V is less than Vmax, the process proceeds to step S1508.
In step S1507, Vmax is updated by replacing with the correlation value V stored in step S1505, and ManIdx is updated by replacing with the person Idx2 of the person data 2.
In step S1508, the person Idx2 is incremented by one.

  In step S1509, it is determined whether the person Idx2 is larger than the detected number Nh2. If the person Idx2 is larger than the detected number Nh2, the process for the number Nh2 detected in step S207 of FIG. 2 is completed for one person Idx1, and the process proceeds to step S1510. On the other hand, if the person Idx2 is less than or equal to the detected number Nh2, the process returns to step S1504 to repeat the processes from step S1504 to step S1509 in order to process the remaining persons.

  In step S1510, it is determined whether Vmax is larger than the minimum correlation value. The minimum correlation value is set to an optimum value in advance. If Vmax is larger than the minimum correlation value, there is a person Idx2 that is determined to be the same person as the person Idx1, and therefore the correlation table is updated in step S1511. On the other hand, if Vmax is less than or equal to the minimum correlation value, the process proceeds to step S1513. The correlation table records what correlation values the person of the person data 1 shows with the person of the person data 2.

  An example of the correlation table will be described in detail with reference to FIG. In the correlation table, the person index of the person data 1, the person index in the person data 2 having the maximum correlation value with the person of the index, and the maximum correlation value are stored in association with each other. No. The person of index 3 of person data 1 in FIG. 3 has a low correlation value with a plurality of persons existing in person data 2 and there is no corresponding person, so the person index of person data 2 and person data 2 The maximum correlation value is “0”. That is, the detected number Nh1 of the person data 1 is 3, and the detected number Nh2 of the person data 2 is 2, indicating that there is no one person from the detection area.

In step S1512, the person Idx1, which is the index of the person data 1, is incremented by one.
In step S1513, it is determined whether or not the person Idx1 is larger than the detected number Nh1, and if the person Idx1 is larger than the detected number Nh1, the process for Nh1 detected in the process of step S202 in FIG. The process proceeds to S1514. On the other hand, when the person Idx1 is equal to or less than the detected number Nh1, since there is a person Idx1 that has not been processed yet, the process returns to step S1502, and the processes from step S1502 to step S1513 are repeated for the unprocessed person Idx1.

In step S1514, the tracking data described above is updated.
In step S1515, it is determined whether or not the number of persons Idx1 is larger than the number of persons Idx2. If the number of persons Idx1 is larger than the number of persons Idx2, the process proceeds to step S1516, the tracking stop of the person Idx1 having a low correlation value is determined from the correlation table, and the tracking flag of tracking data as shown in FIG. 6 is set to zero. Then, the person tracking process ends. On the other hand, if the number of persons Idx1 is equal to or less than the number of persons Idx2, the process proceeds to step S1517.

  In step S1517, it is determined whether the number of persons Idx1 is smaller than the number of persons Idx2. If the number of persons Idx1 is smaller than the number of persons Idx2, the process advances to step S1518 to start a new person tracking and add new data to the tracking data shown in FIG. On the other hand, when the number of people Idx1 is larger than the number of people Idx2, that is, when the number of people Idx1 and the number of people Idx2 are equal from the relationship with step S1515, the process is completed for the number of people detected in the person data 2 In order to show that, the person tracking process is terminated.

According to the first embodiment described above, by clustering measurement objects using distance information, it is possible to perform person recognition and person tracking with higher accuracy than a method using luminance information, and with high accuracy. The number of people can be counted.
(Second Embodiment)
The second embodiment of the present invention uses the following determination for the human feature detection (step S805) in the human detection process of FIG. 8 to determine whether the measurement target included in the human candidate region is a person or a dog Also, it is determined whether it is a small animal such as a cat, and even when a person is carrying a baggage, the person and baggage are separated with high accuracy, and the approximate height of the person, the height of the baggage, etc. It is characterized by measuring.

  The measurement target determination process will be described in detail with reference to the flowchart of FIG. 17 and FIGS. 18 to 20. As shown in FIGS. 18 to 20, the number counter 100 according to the present embodiment has a laser light emitting unit 101 and a laser light receiving unit 103 according to the number counter 100 installed on a ceiling 1801, and a person from the ceiling 1801 toward the floor 1802. Measure candidate regions. Further, the output of the person counter 100 is displayed on the monitor unit 1804 and the counter display unit 1805. As a display method, for example, a method is used in which the monitor unit 1804 outputs and displays distance information and luminance information, and the counter display unit 1805 displays the detected number of people from the number counting unit 118. The distance information and the luminance information may be collectively output from the number counting unit 118, or may be extracted from the distance memory 107 and the luminance memory 111 and output to the outside.

First, in step S1701, it calculates a vertical distance H _n from the distance information for each pixel in the detection region.
Calculation of the vertical distance _{H n} from the laser emitting unit 101 and the laser receiving portion 103 and the distance _{A n} to the floor 1802, and the angle theta _n from perpendicular drawn from the laser emitting unit 101 and the laser receiving portion 103 to the floor 1802 It is calculated | required by (Formula 1).
H _n = A _n · cos θ _n (Formula 1)
(N = 1, 2, 3,..., N−1, n: pixels corresponding to the floor around the measurement target)
Next, in step S1702, in order to reduce variations in the distance for each pixel, the mean value H of the vertical distance H _n peripheral pixels few collectively calculated by (Equation 2).
H = (ΣH _n ) / n (Formula 2)
In step S1703, the initial value of the m is the index of the shortest distance _{J mn} to "1". The shortest distance J _mn refers to the shortest distance among the distances from the laser light emitting unit 101 and the laser light receiving unit 103 to the measurement target in the detection region. In addition, when m is “1”, the pixel set has the shortest distance, and when m is “2”, the pixel set having the shortest distance is excluded except for the pixel having the shortest distance.

In step S1704, it is determined whether there is a pixel whose shortest distance has not been detected in addition to the pixel having the distance to the floor 1802. As a result of this determination, if there is an undetected pixel, the process proceeds to step S1706, and if there is no undetected pixel, the measurement target determination process ends.
In step S1705, a pixel having distance information of the shortest distance _Jmn among undetected pixels is detected, and its surrounding pixels are also detected. The method of calculating the peripheral pixels, as well as the method of calculating the perpendicular distance H _n may be calculated using (Equation 1).

In step S1706, calculates the shortest distance _{J m} averaged as with the vertical distance _{H n} using Equation (2). If the shortest distance J _m is known and the measurement object is a person, the approximate height L of the person can also be obtained from (Equation 3).
L = H−J (Formula 3)
In step S1707, (also referred to as a pixel area) area of the pixel region represented by peripheral pixels including the shortest distance J _m it is determined whether it is within a threshold value. If the pixel area is within the threshold value, it is likely that the head is a human or small animal head, and the process advances to step S1709. On the other hand, if the pixel area is larger than the threshold value, the process advances to step S1708.

In step S1708, it is determined that the region represented by the pixel area is not a human head but a luggage. Thereafter, m is incremented and the process returns to step S1704, and the processes from step S1704 to step S1707 are repeated.
In step S1709, a pixel having distance information with the next shortest distance after J _m and its surrounding pixels are calculated, and the averaged distance J _{m + 1} is calculated using (Equation 2).
In step S1710, the distance J _m pixel area is _{+ 1} is within the threshold value, and the distance J determines the distance difference is whether within the threshold of _m and the distance J _{m + 1.} If the two conditions are satisfied, the process proceeds to step S1711, the distance J pixel area that is _m and the head recognition moiety, the pixel area is the distance J _{m + 1} to the head near the shoulder recognition moiety. On the other hand, if at least one condition is not satisfied, the process advances to step S1708 to determine that the pixel areas J _m and J _{m + 1} are luggage.

In step S <b> 1712, it is determined whether the pixel area of the head periphery shoulder recognition portion exists symmetrically around the pixel area recognized by the head. If the pixel area exists symmetrically, the process proceeds to step S1713. On the other hand, if the pixel area does not exist symmetrically, the process proceeds to step S1714.
In step S1713, since the head periphery shoulder recognition part is considered to correspond to the shoulder of a person, it is determined that the pixel area having the distance J _m and the distance J _{m + 1} is a person. Thereafter, m is incremented, and the process returns to step S1704 to perform the above-described processing.
In step S1714, the shoulder recognition moiety near the head is present on one side of the head recognition moiety, further if the pixel area is greater than the threshold, the distance a pixel area that is J _m and the distance J _{m + 1} is a small animal rather than a human Is determined. Thereafter, m is incremented, and the process returns to step S1704 to perform the above-described processing.

Here, an example of feature detection when the measurement target is a person will be described with reference to FIG.
18A shows a state in which a person 1803 is present directly below the laser light emitting unit 101 and the laser light receiving unit 103 installed on the ceiling 1801, and FIG. 18B shows FIG. 18A viewed from the side. It is a figure showing the distance information of a height direction. A detection area by the laser light emitting unit 101 and the laser light receiving unit 103 is represented as a monitor unit 1804. As shown in FIG. 18 (b), with respect to the detection region, centered on the pixel area is the shortest distance J _m (head recognition moiety), a pixel area next distance comprises a short pixel (head peripheral shoulder (Recognition part) exists symmetrically, it can be seen that the pixel area with the shortest distance J corresponds to the head, and the pixel area with the next shortest distance corresponds to the human shoulder. Therefore, it can be determined that the measurement target is the person 1803 by the human feature detection process shown in FIG.

Next, the case where the measurement target is a small animal such as a dog or a cat will be described with reference to FIG.
(A) is a state in which the small animal 1901 exists directly under the distance camera. As shown in (b), since the head periphery shoulder recognition part exists only on one side with respect to the head recognition part and is longer than the case where the head periphery shoulder recognition part is a person, the small animal 1901 Is determined.
Furthermore, the shortest distance J is longer than the person, that is, the estimated height L is low, and the shoulder perimeter recognition part is such that the center of the shoulder and the center of the head are biased as a dog or a cat, Can classify small animals. For example, it is possible to distinguish between a dog and a cat by setting in advance a threshold value such that a dog is set to L within 1 m and a cat is set to L within 0.5 m.

Next, separation determination between a person and a truck such as a cart will be described with reference to FIG.
(A) has shown the state in which the person 2001 is carrying the luggage | load 2002 by pushing a trolley | bogie as a measuring object. As separation determination procedure, first, detects a pixel of the shortest distance J _1, then detects the short J ₂ pixels of distance. The processing from step S1707 to step S1713 shown in FIG. 17, the portion of the distance _{J 1} and the distance _{J 2} is determined to be human 2001.
Furthermore, among the pixels other than the pixel area that is recognized as a person 2001, it detects a pixel having a minimum distance J _3. Pixel group including a pixel is the distance J ₃ is, whether the pixel area is J ₃ in step S1707 of FIG. 17 is determined to be greater than a threshold range, it is determined that there is no portion corresponding to the shoulder on the periphery by the step S1711 Therefore, it is determined that the baggage 2002. Note that even when a small animal has its head lowered, it may be determined as a baggage. In that case, the human feature detection process may be continuously determined for a certain period and updated as needed.

  According to the second embodiment described above, it is possible to determine whether the measurement target is a person or a small animal such as a dog or a cat, and the person is carrying a load. However, it is possible to determine the separation of a person and a baggage with high accuracy and to measure the approximate height of the person and the height of the baggage.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying constituent elements without departing from the scope of the invention in each stage of implementation. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in each embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 100 ... Person counter, 101 ... Laser light emission part, 102 ... Pulse control part, 103 ... Laser light-receiving part, 104 ... Phase detection part, 105, 109 ... A / D conversion part 106 ... Distance measuring unit, 107 ... Distance memory, 108 ... Amplitude detecting unit, 110 ... Luminance measuring unit, 111 ... Luminance memory, 112 ... Mirror, 113 ... X Axis control unit, 114 ... Y-axis control unit, 115 ... clustering unit, 116 ... measurement target detection unit, 117 ... measurement target tracking unit, 118 ... number of people counting unit, 1801 ... Ceiling, 1802 ... floor, 1803, 2001 ... person, 1804 ... monitor unit, 1805 ... counter display unit, 1901 ... small animal, 2002 ... luggage.

Claims (5)

For each pixel in the detection area obtained by the camera, a measurement unit that measures the distance from the camera to the measurement target;
First distance information associating a pixel position with the distance when the pixel is mapped to a plane on which the measurement object is grounded, and a second distance in the height direction perpendicular to the plane at the pixel position A storage unit that stores, for each pixel, three-dimensional data including distance information;
From the three-dimensional data stored in the storage unit, only the three-dimensional data including the second distance information within the threshold is extracted, and the extracted data is clustered to represent the pixel region including the measurement target. A clustering unit that obtains human candidate regions for each sampling frame;
A tracking unit that determines the same person candidate area based on a correlation value between a plurality of person candidate areas obtained for each sampling frame by the clustering unit, and tracks a movement trajectory of the same person candidate area;
A counting unit that counts the number of passing people as the number of people passing when the candidate region passes through a pixel region set in advance larger than the candidate human region based on the tracking result of the tracking unit. Characteristic people counter.   The clustering unit extracts three-dimensional data having the second distance information that is equal to or greater than a first threshold and uses the three-dimensional data as first extracted data. The clustering unit performs first clustering on the first extracted data for each candidate area. The maximum value of the second distance information in the candidate area is detected as the maximum height, and the height range from the maximum height to the second threshold is set as the upper limit with the maximum height as the upper limit. The number counter according to claim 1, wherein second extraction data is obtained by extraction from the extraction data, and second clustering is performed on the second extraction data. Further comprising a detection unit for detecting the measurement object,
In the detection unit, a first pixel area indicating an area of a pixel region including a pixel having the second distance information of the maximum height and a peripheral pixel of the pixel is within a third threshold range, and the maximum height A second pixel area indicating an area of a pixel region including a pixel having the second highest distance information and a peripheral pixel of the pixel is within a fourth threshold range, and the second pixel area and the first pixel When the distance difference of the second distance information of the area is within a fifth threshold range, and the second pixel area exists symmetrically about the first pixel area, the first pixel area and the second pixel area The person counter according to claim 2, wherein the person counter is detected as a person.   The detection unit includes the first pixel area and the second pixel area when the second pixel area exists only on one side with the first pixel area as a center and the maximum height is lower than a sixth threshold. The number counter according to claim 3, wherein the counter is detected as a small animal.   The detecting unit may be configured such that when the first pixel area is larger than the third threshold range, when the second pixel area is larger than the fourth threshold range, or when the second pixel area and the first pixel area are 4. When at least one of the distance differences of two distance information is larger than the fifth threshold value, at least one of the first pixel area and the second pixel area is detected as a package. Or the number counter of Claim 4.

Images