JP5756709B2 - Height estimation device, height estimation method, and height estimation program - Google Patents

Description

  The present invention relates to a height estimation device, a height estimation method, and a height estimation program.

  Conventionally, the position of the person in the real space is estimated from the position of the person's foot on the image, the height of the person is estimated from the estimated position in the real space, and the estimated height is used to A technique for identifying a concealed portion of the person and estimating and tracking the exact position of the person in the real space is known (see, for example, Patent Document 1).

JP 2010-237872 A

  However, in the conventional technology as described above, when the foot position of the person on the image is concealed, the estimation accuracy of the height is lowered.

  The present invention has been made in view of the above circumstances, and includes a height estimation device, a height estimation method, and a height estimation program for accurately estimating the height of a person even if the foot position of the person is concealed on an image. The purpose is to provide.

  In order to solve the above-described problems and achieve the object, a height estimation apparatus according to one aspect of the present invention includes an imaging unit that captures an image including a target person in time series, and each of the images captured in time series. A head detecting unit that detects the head of the target person from the above, and a foot that estimates the foot position of the target person for each assumed height from each of the images captured in time series using the detected head A position estimation unit, a real space position estimation unit that estimates the position of the target person in real space in time series for each assumed height, using the estimated foot position, and is estimated for each assumed height. And a height estimation unit that creates a human-like movement trajectory using a time-series position in the real space and estimates the assumed height from which the most human-like movement trajectory was created to the height of the target person. To do.

  The height estimation method according to another aspect of the present invention includes an imaging step in which an imaging unit images an image including a target person in time series, and a head detection unit from each of the images captured in time series. A head detection step for detecting the head of the target person, and a foot position estimation unit assuming the foot position of the target person from each of the images captured in time series using the detected head. A step of estimating a foot position every time, and a real space position estimating unit that estimates the position of the target person in real space in time series for each assumed height using the estimated foot position. An estimation step and a height estimation unit creates a human-like movement trajectory using the estimated time-series position in the real space for each of the assumed heights, and uses the assumed height from which the most human-like movement trajectory was created as the target. Person's height Characterized in that it comprises a body height estimating step of estimating, the.

  The height estimation program according to another aspect of the present invention includes an imaging step of capturing an image including a target person in time series, and a head for detecting the head of the target person from each of the images captured in time series. A step of estimating a foot position of the target person for each assumed height from each of the images captured in time series using the detected head, and the estimated foot position A real space position estimation step for estimating the position of the target person in real space for each assumed height in a time series, and the estimated time series position in the real space for each assumed height. A human-like movement trajectory, and a computer that executes a height estimation step of estimating the assumed height from which the most human-like movement trajectory was created to the height of the target person. .

  According to the present invention, there is an effect that the height of a person can be accurately estimated even if the foot position of the person is concealed on the image.

FIG. 1 is a block diagram illustrating an example of a configuration of the height estimation apparatus according to the first embodiment. FIG. 2 is an explanatory diagram illustrating an example of a foot position estimation method performed by the foot position estimation unit according to the first embodiment. FIG. 3 is an explanatory diagram illustrating an example of a position in the real space of the target person for each assumed height stored in the storage unit according to the first embodiment. FIG. 4 is a diagram illustrating an example of a movement trajectory created by the height estimation unit according to the first embodiment. FIG. 5 is a diagram illustrating an example of the relationship between the assumed height and the position of the target person in real space estimated from the assumed height. FIG. 6 is a diagram illustrating an example of the relationship between the assumed height and the lateral movement of the target person estimated from the assumed height. FIG. 7 is a diagram illustrating an example of the relationship between the assumed height and the movement distance of the target person estimated from the assumed height in real space. FIG. 8 is a diagram illustrating an example of the relationship between the assumed height and the moving distance of the target person in real space estimated from the assumed height. FIG. 9 is a diagram illustrating an example of the relationship between the assumed height and the vertical movement of the target person estimated from the assumed height. FIG. 10 is a flowchart illustrating an example of a flow of a height estimation process performed by the height estimation apparatus according to the first embodiment. FIG. 11 is a flowchart illustrating an example of a flow of processes performed by the height estimation unit of the height estimation apparatus according to the first embodiment. FIG. 12 is a block diagram illustrating an example of a configuration of the height estimation apparatus according to the second embodiment. FIG. 13 is a flowchart illustrating an example of a flow of a tracking process performed by the height estimation apparatus according to the second embodiment. FIG. 14 is a flowchart illustrating an example of a flow of a squatting determination process performed by the height estimation apparatus according to the first modification. FIG. 15 is a flowchart illustrating an example of a flow of a squatting determination process performed by the height estimation apparatus according to the second modification.

  Hereinafter, embodiments of a height estimation device, a height estimation method, and a height estimation program according to the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
First, the structure of the height estimation apparatus of 1st Embodiment is demonstrated.

  FIG. 1 is a block diagram illustrating an example of the configuration of the height estimation apparatus 100 according to the first embodiment. As shown in FIG. 1, the height estimation device 100 includes an input unit 110, a storage unit 120, an imaging unit 130, a control unit 140, and a display unit 150.

  The input unit 110 inputs various data, and can be realized by an existing input device such as a keyboard or a mouse. The input unit 110 inputs, for example, a plurality of assumed heights, imaging parameters of the imaging unit 130 described later, reference data, and the like.

  In this embodiment, the imaging parameters are the installation height of the imaging unit 130, the depression angle, the angle of view, and the vanishing point on the image, but are not limited thereto. In the present embodiment, the reference data includes the height of the reference object, the distance from the imaging unit 130 to the reference object, and the position of the reference object on the image, but is not limited thereto. The reference data may be internal / external parameters of the imaging unit 130.

  The storage unit 120 stores various data input from the input unit 110 and data processed by the control unit 140 described later. The storage unit 120 is, for example, at least one of various existing storage devices that can store magnetically, optically, or electrically such as a hard disk drive (HDD), a solid state drive (SSD), and a random access memory (RAM). It can be realized by either.

  The imaging unit 130 captures an image including the target person in time series, and can be realized by, for example, a digital camera.

  The control unit 140 controls each unit of the height estimation apparatus 100 and can be realized by, for example, a CPU (Central Processing Unit). The control unit 140 includes a head detection unit 141, a foot position estimation unit 143, a real space position estimation unit 145, and a height estimation unit 147.

  The head detection unit 141 detects the head of the target person from each of the images captured in time series by the imaging unit 130. For example, the head detection unit 141 detects the head of the target person from the image using a filter. The filter may be any filter as long as it can detect the position considered to be the head, such as “the upper end of the moving body is the upper end of the head”.

  The foot position estimation unit 143 uses the head detected by the head detection unit 141 to estimate the foot position of the target person for each assumed height from each of the images captured in time series by the imaging unit 130. Specifically, the foot position estimation unit 143 uses a plurality of assumed heights and imaging parameters input by the input unit 110 and stored in the storage unit 120, and the head detected by the head detection unit 141, The foot position of the target person is estimated for each assumed height from each of the images captured in time series by the imaging unit 130.

FIG. 2 is an explanatory diagram illustrating an example of a foot position estimation method performed by the foot position estimation unit 143 according to the first embodiment. In the example shown in FIG. 2, the installation height H _cam , the depression angle α, the vertical angle of view β _v (not shown), and the vanishing point P _v (X _v , Y _v ) correspond to the imaging parameters and are known. The head coordinates P _h (X _h , Y _h ) on the target person 160 image correspond to the head detected by the head detection unit 141 and are known. Further, the height H _man of the target person 160 corresponds to the assumed height and is known. In this embodiment, it is assumed that the assumed height is set in increments of 5 cm in a range of 155 cm to 185 cm. That is, the assumed height is assumed to be 155 cm, 160 cm, 165 cm, 170 cm, 175 cm, 180 cm, and 185 cm.

Here, assuming that the foot position coordinates on the image of the target person 160 are P _f (X _f , Y _f ), the foot position estimation unit 143 calculates the formula (1) for each height H _man (assumed height) of the target person 160. ) is used to estimate the foot position _{Y f} of the target person 160.

  Here, A is represented by Equation (2), and B is represented by Equation (3).

Here, the first vector V1 heading from the imaging unit 130 toward the head of the target person 160, the first vector, the X-axis and Y-axis direction components are the same, and the Z-axis direction component is the optical axis of the imaging unit 130. The angle ρ _h formed by the same second vector V2 is expressed by Equation (4). When the target person 160 is present in front of the imaging unit 130, as shown in FIG. 2, the first vector V1 from the imaging unit 130 toward the head of the target person 160 and the optical axis of the imaging unit 130 ( angle between the second vector V2) is [rho _h.

  Note that imgHeight is the height of the image (the number of pixels in the vertical direction) and is known.

  The real space position estimation unit 145 uses the foot position estimated by the foot position estimation unit 143 to estimate the position of the target person in real space for each assumed height in time series. Specifically, the real space position estimation unit 145 uses the imaging parameters and reference data input by the input unit 110 and stored in the storage unit 120, and the foot position estimated by the foot position estimation unit 143, The position of a person in real space is estimated in time series for each assumed height.

  The real space position estimation unit 145 can estimate the position of the target person in the real space for each assumed height in time series using, for example, the method described in Japanese Patent Application Laid-Open No. 2008-286638. However, the technique described in Japanese Patent Application Laid-Open No. 2008-286638 requires that the camera be installed horizontally (angle of depression = 0), and therefore it is necessary to remove the influence of the depression angle.

  Specifically, in the method described in Japanese Patent Application Laid-Open No. 2008-286638, the perspective is used to calculate the person's apparent size (the number of pixels corresponding to the person's height on the image) from the person. Find the distance between the camera and the camera. However, if the depression angle is not 0, the number of pixels corresponding to the height of the person on the image is affected not only by the distance between the person and the camera but also by the depression angle.

  For this reason, when using the method described in Japanese Patent Application Laid-Open No. 2008-286638 in this embodiment, the real space position estimation unit 145 corresponds to the height of the person on the image that has changed due to the influence of the depression angle. The number of pixels to be corrected is corrected by equation (5). Then, the real space position estimation unit 145 obtains the size of the person from which the influence of the depression angle is removed by applying a perspective method to the number of pixels corresponding to the height of the person on the corrected image, Estimate the position of a person.

S ′ = S (cos α + sin α tan (α + ρ _h )) (5)

  Note that S ′ [pixels] is the number of pixels corresponding to the height of the person on the image after correction, and S [pixels] is the number of pixels corresponding to the height of the person on the image before correction. .

  Then, the real space position estimation unit 145 stores the estimated position of the target person of each assumed height in real space in the storage unit 120 for each time series (frame). The real space position estimation unit 145 stores the position of the target person in real space for each assumed height in the storage unit 120 for the latest F frames. That is, the real space position estimation unit 145 deletes the position of the target person in the oldest frame for each assumed height in real space from the storage unit 120, and then deletes the position of the target person in the latest frame in real space for each assumed height. Is stored in the storage unit 120. In this embodiment, F is set to 10. However, the present invention is not limited to this, and can be set as appropriate.

  FIG. 3 is an explanatory diagram illustrating an example of a position in the real space of the target person for each assumed height stored in the storage unit 120 according to the first embodiment. In the example illustrated in FIG. 3, the real space position estimation unit 145 associates the frame number, date and time, height (a plurality of assumed heights), and position (a position in the real space corresponding to the assumed height) in the storage unit 120. I remember it.

  The height estimation unit 147 creates a human-like movement trajectory using the estimated time-series position in the real space for each assumed height, and estimates the assumed height from which the most human-like movement trajectory was created to the height of the target person. .

  Specifically, the height estimation unit 147 determines, for each assumed height, whether or not the target person can move to a position next to the position for each estimated time-series position in the real space. In the present embodiment, the height estimation unit 147 predetermines a time series interval, that is, a range in which the target person can move between frames, and calculates the position for each estimated time series position in the real space. Whether or not the target person can move is determined based on whether or not the distance to the next position is within a predetermined movable range. Here, assuming that the average walking speed of humans is V and the interval between frames is Δt, the movable range R is R = V × Δt. Then, the height estimation unit 147 creates a movement trajectory connecting the position and the next position when the movement is possible, and estimates the assumed height on which the predetermined number or more of movement trajectories are created as the height of the target person. The height estimation unit 147 deletes information such as the position used for estimating the height of the target person from the storage unit 120.

  FIG. 4 is a diagram illustrating an example of a movement trajectory created by the height estimation unit 147 of the first embodiment. In the example shown in FIG. 4, the movement trajectory when the assumed height is 155 cm, 165 cm, 175 cm, and 185 cm is shown. In the example shown in FIG. 4, the height estimation unit 147 estimates the height of the target person to 175 cm because the most movement trajectory is created when the assumed height is 175 cm.

  Here, the relationship between the assumed height and the position and movement distance in the real space estimated from the assumed height will be described.

  FIG. 5 is a diagram illustrating an example of the relationship between the assumed height and the position of the target person in real space estimated from the assumed height. Although the target persons 160 to 162 are the same person, different reference numerals are given for the sake of explanation.

  When the assumed height is larger than the actual height of the target person 160, the position of the target person 160 in the real space is estimated by the real space position estimation unit 145 closer to the front side than the actual position. That is, in the example shown in FIG. 5, if the assumed height is larger than the actual height of the target person 160, the real space position estimation unit 145 estimates the target person 160 as the target person 161.

  Similarly, when the assumed height is smaller than the actual height of the target person 160, the position of the target person 160 in the real space is estimated behind the actual position by the real space position estimation unit 145. That is, in the example illustrated in FIG. 5, when the assumed height is smaller than the actual height of the target person 160, the real space position estimation unit 145 estimates the target person 160 as the target person 162.

  FIG. 6 is a diagram illustrating an example of the relationship between the assumed height and the lateral movement of the target person estimated from the assumed height.

  As shown in FIG. 6, when the assumed height is larger than the actual height of the target person 160, that is, when the target person 160 is estimated as the target person 161, the direction in which the target person 161 crosses the imaging unit 130, that is, When moving in the horizontal direction, the horizontal interval between positions of the target person 161 for each frame becomes smaller than the horizontal interval between positions of the target person 160 for each frame. For this reason, if the distance between the adjacent positions of the target person 160 falls within the movable range, the distance between the adjacent positions of the target person 161 also falls within the movable range. Can be connected by a movement trajectory.

  On the other hand, when the assumed height is smaller than the actual height of the target person 160, that is, when the target person 160 is estimated as the target person 162, the target person 162 moves in the direction crossing the imaging unit 130, that is, in the horizontal direction. The horizontal interval between the positions of the target person 162 for each frame is larger than the horizontal interval between the positions of the target person 160 for each frame. Therefore, even if the distance between adjacent positions of the target person 160 is within the movable range, the distance between adjacent positions of the target person 162 is not necessarily within the movable range. In this case, the height estimation is performed. The unit 147 cannot connect adjacent positions of the target person 162 with a movement locus.

  That is, as the assumed height becomes smaller than the actual height of the target person 160, the movement trajectory created by the height estimation unit 147 does not become a human-like movement trajectory.

  7 and 8 are diagrams showing an example of the relationship between the assumed height and the movement distance of the target person in real space estimated from the assumed height.

  As shown in FIG. 7, when the assumed height matches the actual height of the target person 160, the head of the target person 160 has moved by p [pixels] in the vertical direction on the image (the depth direction in the real space). And In this case, the position of the target person 160 in the real space is estimated by the real space position estimation unit 145 as a position on the back side of x1 [m].

  On the other hand, as shown in FIG. 8, when the assumed height is larger than the actual height of the target person 160, that is, when the target person 160 is estimated as the target person 161, the head of the target person 161 is displayed on the image. It is assumed that p [pixels] are moved in the vertical direction (the depth direction in the real space). In this case, the position of the target person 161 in the real space is estimated by the real space position estimation unit 145 as a position on the back side by x2 (x2> x1) [m].

  FIG. 9 is a diagram illustrating an example of the relationship between the assumed height and the vertical movement of the target person estimated from the assumed height.

  As shown in FIG. 9, when the assumed height is smaller than the actual height of the target person 160, that is, when the target person 160 is estimated as the target person 162, the target person 162 moves forward / backward with respect to the imaging unit 130. When moving in the moving direction, that is, in the vertical direction, the vertical interval between positions of the target person 162 for each frame becomes smaller than the vertical interval between positions of the target person 160 for each frame. For this reason, if the distance between the adjacent positions of the target person 160 falls within the movable range, the distance between the adjacent positions of the target person 162 also falls within the movable range. Can be connected by a movement trajectory.

  On the other hand, when the assumed height is larger than the actual height of the target person 160, that is, when the target person 160 is estimated as the target person 161, the direction in which the target person 161 moves forward / backward with respect to the imaging unit 130, that is, When moving in the vertical direction, the vertical interval between positions of the target person 161 for each frame is larger than the vertical interval between positions of the target person 160 for each frame. Therefore, even if the distance between adjacent positions of the target person 160 is within the movable range, the distance between adjacent positions of the target person 161 is not necessarily within the movable range. In this case, the height estimation is performed. The unit 147 cannot connect adjacent positions of the target person 161 with a movement locus.

  That is, as the assumed height becomes larger than the actual height of the target person 160, the movement locus created by the height estimation unit 147 does not become a human-like movement locus.

  From the above, as the assumed height becomes closer to the actual height of the target person 160, the movement locus created by the height estimation unit 147 becomes a human-like movement locus.

  In addition, when there are a plurality of assumed heights for which a predetermined number or more of movement trajectories have been created, the height estimation unit 147 uses the plurality of assumed heights and the number of movement trajectories for each of the plurality of assumed heights. Estimate height. As described above, as the assumed height is closer to the actual height of the target person, a human-like movement trajectory is created. Therefore, when there are a plurality of assumed heights in which a predetermined number or more of movement trajectories are created, these assumed heights are adjacent. It is highly possible that For this reason, in this embodiment, for example, when there are two assumed heights in which a predetermined number or more of movement trajectories are created, the height estimation unit 147 estimates the height of the target person using Equation (6).

Here, H is the estimated height of the target person, H _a and H _b are adjacent assumed heights, N _a is the number of trajectories of H _a , and N _b is the movement of H _b The number of trajectories.

  In the present embodiment, the height estimation unit 147 determines the most human-like movement trajectory based on the number of movement trajectories created, but is not limited thereto, and the number of movement trajectories created continuously, The most human-like movement locus may be determined based on the shape of the created movement locus.

  The display unit 150 displays various types of information, and can be realized by, for example, a display. The display unit 150 displays the height estimated by the height estimation unit 147. The display unit 150 may display a trajectory created together with the height estimated by the height estimation unit 147. Further, the display unit 150 may display the trajectory created by the height estimation unit 147 for each assumed height.

  Next, the operation of the height estimation device of the first embodiment will be described.

  FIG. 10 is a flowchart illustrating an example of a flow of a height estimation process performed by the height estimation apparatus 100 according to the first embodiment.

  First, the input unit 110 inputs a plurality of assumed heights, imaging parameters, and reference data (step S100).

  Subsequently, the imaging unit 130 captures an image including the target person in time series (step S102).

  Subsequently, the head detection unit 141 detects the head of the target person from each of the images captured in time series by the imaging unit 130 (step S104). When the head of the target person cannot be detected from each of the captured images (No in step S104), head detection is performed again (step S104).

  When the head of the target person can be detected from each of the captured images (Yes in step S104), the foot position estimation unit 143 uses the head detected by the head detection unit 141 to detect the time. The foot position of the target person is estimated for each assumed height from each of the images captured in the series (step S106).

  Subsequently, the real space position estimation unit 145 estimates the position of the target person in the real space in time series for each assumed height using the foot position estimated by the foot position estimation unit 143 (step S108). Then, the real space position estimation unit 145 stores the position in the real space in the storage unit 120 for each assumed height in time series (step S110).

  Subsequently, the height estimation unit 147 creates a human-like movement trajectory for each assumed height using a time-series position in the real space, and confirms whether or not a movement trajectory exceeding the reference has been created (step S112). . When a movement trajectory exceeding the reference is created (Yes in step S112), the movement trajectory is regarded as the most human-like movement trajectory, and the assumed height from which the movement trajectory is created is estimated as the height of the target person (step S114). On the other hand, when the movement locus exceeding the reference is not created (No in step S112), the process returns to step S104.

  FIG. 11 is a flowchart illustrating an example of a flow of processes performed by the height estimation unit 147 of the height estimation apparatus 100 according to the first embodiment.

If all the assumed heights are confirmed, the height estimation unit 147 ends the process (Yes in step S120). Height estimating unit 147, if not sure all assumed height (No at step S120), obtains the unconfirmed assumption height, and assumes height _{H a} (step S122).

Subsequently, when the height estimation unit 147 has not processed all adjacent frames in the latest F frame for the assumed height H _a (No in step S124), the unestimated adjacent frame is acquired and the frames F _i , F _j (step S126). Note that the assumption height _{H a,} if it is processing all of the adjacent frames in the last F frames (Yes in step S124), the flow returns to step S120.

Subsequently, height estimating section 147 calculates the time Δt of the frame _F i, _{F j,} the target person is the frame _F i, to calculate the movable range R between _{F j} (step S128).

Subsequently, the height estimation unit 147 confirms whether the distance between the positions in the real space estimated by the frames F _i and F _j is equal to or less than the range R (step S130).

When the distance between the positions in the real space estimated in the frames F _i and F _j is equal to or smaller than the range R (Yes in step S130), the height estimation unit 147 determines the distance in the real space estimated in the frames F _i and F _j. Are linked, and a movement trajectory connecting the two positions is created (step S132). If the distance between the positions in the real space estimated by the frames F _i and F _j is not equal to or less than the range R (No in step S130), the process returns to step S124.

Subsequently, height estimation unit 147, when linked N (0 <N <F) or more frames, i.e., when you create a more movement trajectory N (in step S134 Yes), the target person assumptions height _{H a} height And output together with the created movement trajectory (step S136). Then, the process returns to step S124. Note that if N (0 <N <F) frames or more are not connected (No in step S134), the process returns to step S124.

  As described above, according to the first embodiment, as long as a person's head is imaged, the height of the person can be estimated by estimating the person's foot, so the position of the person's foot is hidden on the image. However, the height of the person can be estimated. In particular, since the feet are likely to cause erroneous detection such as shadows and specular reflections, the height of the person can be accurately estimated by estimating the height without detecting the feet.

(Second Embodiment)
In the second embodiment, an example in which the target person is tracked using the estimated height will be described. In the following, differences from the first embodiment will be mainly described, and components having the same functions as those in the first embodiment will be given the same names and symbols as those in the first embodiment, and description thereof will be given. Is omitted.

  FIG. 12 is a block diagram illustrating an example of a configuration of the height estimation apparatus 200 according to the second embodiment. In the height estimation apparatus 200 of the second embodiment, the foot position estimation unit 243 and the real space position estimation unit 245 are different from the first embodiment and further include a tracking unit 249.

  The foot position estimation unit 243 estimates the height of the target person whose height is estimated by the height estimation unit 147 from the image captured by the imaging unit 130 after the height estimation unit 147 estimates the height of the target person. Estimate using.

  The real space position estimation unit 245 uses the foot position estimated by the foot position estimation unit 243 to estimate the position of the target person in the estimated height in real space.

  The tracking unit 249 tracks the position of the target person of the estimated height estimated by the real space position estimation unit 245 in the real space.

  FIG. 13 is a flowchart illustrating an example of a flow of a tracking process performed by the height estimation apparatus 200 according to the second embodiment.

  First, the imaging unit 130 captures an image including the target person (step S200).

  Subsequently, the head detection unit 141 detects the head of the target person from the image captured by the imaging unit 130 (step S202). When the head of the target person cannot be detected from the captured image (No in step S202), the process returns to step S200.

  When the head of the target person can be detected from the captured image (Yes in step S202), the foot position estimation unit 243 estimates the head detected by the head detection unit 141 and the height estimation unit 147. Using the height, the foot position of the target person is estimated from the image captured by the imaging unit 130 (step S204).

  Subsequently, the real space position estimation unit 245 uses the foot position estimated by the foot position estimation unit 243 to estimate the position of the estimated person in the real space (step S206).

  Subsequently, when the tracking unit 249 has created new tracking information (Yes in Step S208), the tracking unit 249 updates the tracking information (Step S210). Thereby, the tracking unit 249 tracks the position of the target person of the estimated height in the real space. Then, the process returns to step S200. On the other hand, also when new tracking information is not created (No in step S208), the process returns to step S200.

  Here, the update of the tracking information will be described.

First, when the real space position estimation unit 245 estimates the position of the estimated person in real space, the tracking unit 249 sets an initial value D _t0 as the distance threshold D _t . The initial value D _t0 is preferably a sufficiently large value such as 100 [m].

Subsequently, the tracking unit 249 acquires, from the tracking information stored in the storage unit 120, the position P _{i in} the previous real space of the target person of estimated height and the acquisition time T _i thereof. Then, the tracking unit 249 uses the position P _i and the latest position P in the real space of the target person of the estimated height estimated by the real space position estimation unit 245 in step S206 to determine the distance D _i between both positions. Calculate (D _i = P−P _i ). Furthermore, the tracking unit 249 uses the average walking speed V of the human, the time T _i , and the acquisition time T of the position P of the target person in the latest real space, and the time T-time T _i of the target person with the estimated height. A range R that can be moved between is calculated (R = V × (T−T _i )).

Subsequently, when the distance D _i is not more than the range R and not more than the distance threshold value D _t , the tracking unit 249 stores the position P and time T in the storage unit 120 as new tracking information, and updates the tracking information. At this time, the tracking unit 249 associates the position P _i and the position P, and creates a movement trajectory that connects both positions. The tracking unit 249 sets the distance D _i as the distance threshold D _t and uses it for the next update of the tracking information.

  However, if the tracking information is updated by the above-described method, an error occurs in the estimated position in the real space due to noise due to light or shadow, and the tracking may fail. For this reason, you may make it update tracking information using the moving average of a moving speed. Thereby, the error which generate | occur | produces suddenly by noise can be suppressed.

In this case, first, when the real space position estimation unit 245 estimates the position of the target person of the estimated height in the real space, the tracking unit 249 sets the initial value V _t0 as the variance threshold V _t . The initial value V _t0 is preferably a sufficiently large value such as 100 [m / s].

Subsequently, the tracking unit 249 determines, based on the tracking information stored in the storage unit 120, positions P _{i_1 to} P _{i_n in} the real space for the latest n frames of the target person of estimated height and their acquisition times T _{i_1 to} T _{i. -N} is acquired. It is assumed that the time series is in the order of i_n, i_n-1, ... i_1.

Subsequently, the tracking unit 249 calculates movement speeds V _{i — 1} to V _{i — n} between adjacent positions. For example, V _{i — 1} is represented by V _i — ₁ = D (P, P _i — ₁ ) / (T−T _i — ₁ ), and V _i — _n is V _i — _n = D (P _{i — n−1} , P _{i — n} ) / (T _{i — n−1).} −T _{i — n} ). Note that D (x, y) is a function for obtaining the distance between the position x and the position y.

Subsequently, the tracking unit 249 calculates the moving average M _i and the variance V _i from the moving speeds V _{i — 1} to V _{i — n} .

Subsequently, the tracking unit 249, if the upper limit value T _h or less and the dispersion V _i of the moving average M _i is likely human moving speed is equal to or less than the variance threshold value V _t, the storage unit the position P and the time T as a new trace information 120, and the tracking information is updated. At this time, the tracking unit 249 associates the position P _{i — 1} with the position P, and creates a movement trajectory that connects both positions. The tracking unit 249 sets the variance V _i in the dispersion threshold V _t, is used when updating the next tracking information.

Here, in order to link the positions, the variance V _i is compared with the variance threshold value V _t because a person cannot stop suddenly. This is because it can be judged.

  As described above, according to the second embodiment, since a person can be tracked without considering a step that is likely to cause a false detection such as a shadow or a specular reflection, the person can be accurately tracked.

(Modification)
In addition, this invention is not limited to the said embodiment, A various deformation | transformation is possible.

(Modification 1)
For example, if the target person crouches while tracking the target person, the height of the target person's head changes, and the correct position of the target person in real space cannot be estimated. For this reason, in the second embodiment, the squatting determination may be performed when tracking the target person.

  When the target person crouches, a phenomenon occurs in which the height of the target person at the same position in the real space gradually decreases. Therefore, the tracking unit 249 may recognize this phenomenon and determine whether the target person has crouched or another person with a short height has appeared.

  FIG. 14 is a flowchart illustrating an example of a flow of a squatting determination process performed by the height estimation apparatus 200 according to the first modification.

  First, the imaging unit 130 captures an image including the target person (step S300).

Subsequently, the tracking unit 249 tracks the position of the target person in the real space (step S302). When the tracking unit 249 detects a new person (Yes in step S304), the real space position estimation unit 245 determines the position P _c (X _c , Y _c ) of the new person in the real space and the height H _c. Is estimated (step S306).

Subsequently, the tracking unit 249 refers to the height H _i and the position P _i (X _i , Y _i ) of the target person from the tracking information of the target person (step S308), and the position P _c and the position P _i are constant. It is confirmed whether or not it exists within the range (step S310).

Then, when the position P _c and the position P _i are within a certain range (Yes in step S310), the tracking unit 249 acquires the tracking number i and the acquisition time T _{c of the} position P _c as information for squatting determination. stores the height _{H c} in the storage unit 120 (step S312), the process ends.

On the other hand, it does not exist within a predetermined a position _{P c} and the position _{P i} in the range (No at step S310), information for determining narrowing squatting is stored in the storage unit 120 (Yes at step S314), determination narrowing squatting If the information for use is a predetermined number of the same tracking number (Yes at step S316), the tracking unit 249, with the time course of time T _c, determines whether the height H _c is lower in the range of ΔH (Step S318). The tracking unit 249 determines if the height _{H c} is lower in the range of [Delta] H (in step S320 Yes), there included squatting (step S322).

If no new person is detected in step S304 (No in step S304), or if information for squatting determination is not stored in the storage unit 120 in step S314 (No in step S314), in step S316. If information for determining narrowing squatting is not a predetermined number in the same tracking number (no at step S316), and if the height _{H c} is not lower in the range of ΔH in step S320 (no at step S320), the tracking unit 249 determines that there is no squatting (step S324).

(Modification 2)
The squatting determination method of Modification 1 is based on the premise that there is no obstacle between the imaging unit 130 and the foot of the target person, and the person position in real space can be estimated from the foot position on the image. In the second modification, an example in which the squatting determination of the target person is performed without using such a premise will be described.

  FIG. 15 is a flowchart illustrating an example of a flow of a squatting determination process performed by the height estimation apparatus 200 according to the second modification.

  First, the imaging unit 130 captures an image including the target person (step S400).

Subsequently, the tracking unit 249 tracks the position of the target person in the real space (step S402). When the tracking unit 249 detects a new person (Yes in step S404) and tracking information in which the target person disappears within a predetermined time is stored in the storage unit 120 (Yes in step S406), the tracking unit 249 Refers to the height H _a and movement trajectory Q _a {P _a1 ,..., P _aN } of the person (target person) disappeared from the tracking information (step S408), and the new person's height H _b and movement trajectory Q Reference is made to _b {P _b1 ,..., P _bM } (step S410).

Subsequently, when the height H _a is larger than the height H _b (Yes in Step S412), the tracking unit 249 determines continuity from the movement locus Q _a and the movement locus Q _b (Step S414), and the continuity is determined. If there is (Yes in Step S416), it is determined that there is squatting (Step S418).

Here, for the determination of continuity, the determination method based on the movement range and movement speed described above may be used. For example, when determining using the movement range, the tracking unit 249 indicates that the position P _a1 immediately before disappearing in the movement locus Q _a and the position P _bM detected first in the movement locus Q _b are human. What is necessary is just to determine whether it enters in a movement range. Further, for example, when the determination is made using the moving speed, the tracking unit 249 determines the position Q immediately after being detected as K positions Q _a ′ {P _a1 ,..., P _aK } (K <N) immediately before disappearing. _b ′ {P _{M−L + 1} ,..., P _bM } (L <M) may be used to determine continuity.

On the other hand, when no new person is detected in step S404 (No in step S404), tracking information in which the target person disappears within a predetermined time is not stored in the storage unit 120 in step S406 (No in step S406). ), if the height _{H a} is smaller than the height _{H b} at step S412 (no at step S412), and if there is no continuity in step S416 (no at step S416), the tracking unit 249, without narrowing squatting and determines (Step S420).

(Modification 3)
In the above embodiment, when there are a plurality of assumed heights where a predetermined number or more of movement trajectories are created, the height estimation unit 147 uses the plurality of assumed heights and the number of movement trajectories of each of the plurality of assumed heights. Although the height of the target person is estimated, the height of the target person may be estimated using the variance of the moving speed of each of the plurality of assumed heights.

In this case, the height estimation unit 147 obtains the movement locus Q _a {P _a1 (X _a1 , Y _a1 ),..., P _ai ,..., P _aN } calculated from the assumed height H _a . Here, T _ai is the time when the position P _ai (i = 1,..., N) is estimated, and the latest data is input when i = 1. Then, the height estimation unit 147 obtains _a variance value V _a of the movement speed from the obtained movement locus Q _a (see Expression (7)). Here, the variables of Equation (7) are as shown in Equation (9) and Equation (11).

Similarly, the height estimation unit 147 obtains a movement locus Q _b {P _b1 (X _b1 , Y _b1 ),..., P _bi ,..., P _bM } calculated from the assumed height H _b . Here, it is assumed that the time when the position P _bi (i = 1,..., M) is estimated is T _bi, and the latest data is input when i = 1. The height estimating section 147 obtains the variance value _{V b} of the moving speed from the movement locus _{Q b} determined (see Equation (8)). Here, the variables of Equation (8) are as shown in Equation (10) and Equation (12).

  Then, the height estimation unit 147 estimates the height of the target person using Equation (13).

(Modification 4)
In the second embodiment, the target person is tracked using the estimated height. However, if the target person's feet can be reliably imaged during tracking, the target person is tracked by replacing the height estimated from the step. You may do it.

(Modification 5)
For example, when the height estimation device of each embodiment and each modification is used for height estimation and tracking of a robber, in addition to the estimated height and movement trajectory, other characteristics (human phase and clothes) are output. Also good. Further, the height estimation device of each of the above embodiments and modifications is preferably used in places such as retail stores and crowds where a person's feet are often not visible on a shelf or the like, and is applied to marketing, human flow analysis, etc. Can be expected.

(Hardware configuration)
The height estimation device according to each of the above embodiments and modifications includes a control device such as a CPU (Central Processing Unit), a storage device such as a ROM and a RAM, an external storage device such as an HDD and an SSD, and a display device such as a display. And an input device such as a mouse and a keyboard and a communication device such as a communication I / F, and has a hardware configuration using a normal computer.

  The height estimation program executed by the height estimation device of each of the above embodiments and modifications is a file in an installable format or an executable format, and is a CD-ROM, CD-R, memory card, DVD, flexible disk (FD). And the like stored in a computer-readable storage medium.

  Further, the height estimation program executed by the height estimation device of each of the above embodiments and modifications may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Good. Further, the height estimation program executed by the height estimation device of each of the above embodiments and modifications may be provided or distributed via a network such as the Internet. Further, a height estimation program executed by the height estimation device of each of the above embodiments and modifications may be provided by being incorporated in advance in a ROM or the like.

  The height estimation program executed by the height estimation device according to each of the embodiments and the modifications has a module configuration for realizing the above-described units on a computer. As actual hardware, for example, the CPU reads out a program from the HDD to the RAM and executes it, so that the above-described units are realized on the computer.

100, 200 Height estimation device 110 Input unit 120 Storage unit 130 Imaging unit 140 Control unit 141 Head detection unit 143, 243 Foot position estimation unit 145, 245 Real space position estimation unit 147 Height estimation unit 249 Tracking unit 150 Display unit

Claims (8)

An imaging unit that captures images including the target person in time series;
A head detection unit that detects the head of the target person from each of the images captured in time series;
Using the detected head, a foot position estimation unit that estimates the foot position of the target person for each assumed height from each of the images captured in time series,
Using the estimated foot position, a real space position estimation unit that estimates the position of the target person in real space in time series for each assumed height;
A height estimation unit that creates a human-like movement trajectory using the estimated time-series position in the real space for each assumed height, and estimates the assumed height from which the most human-like movement trajectory was created to the height of the target person When,
A height estimation apparatus comprising:   The height estimation unit determines whether or not the target person can move to a position next to the position for each of the assumed heights for each estimated time-series position in the real space. 2. The movement trajectory connecting the position and the next position is created in some cases, and an assumed height on which a predetermined number or more of movement trajectories are created is estimated as the height of the target person. Height estimation device.   When there are a plurality of assumed heights where a predetermined number or more of movement trajectories are created, the height estimation unit uses the plurality of assumed heights and the number of movement trajectories of each of the plurality of assumed heights to calculate the height of the target person. The height estimation device according to claim 2, wherein the height is estimated.   When there are a plurality of assumed heights in which a predetermined number or more of movement trajectories are created, the height estimation unit distributes the movement speed of the target person for each of the plurality of assumed heights from the movement trajectories of the plurality of assumed heights. The height estimation according to claim 2, wherein the height of the target person is estimated by calculating and using a variance of the plurality of assumed heights and a movement speed of the target person for each of the plurality of assumed heights. apparatus. The foot position estimation unit estimates the foot position of the target person from an image captured after the height of the target person is estimated using an estimated height that is the height estimated by the height estimation unit,
The real space position estimating unit estimates the position of the target person in real space of the estimated height using the estimated foot position,
The height estimation apparatus according to claim 1, further comprising a tracking unit that tracks a position of the estimated height of the target person in real space.   6. The height estimation apparatus according to claim 5, wherein the tracking unit determines whether or not the target person of the estimated height has crouched during tracking. An imaging step in which the imaging unit captures an image including the target person in time series; and
A head detection step in which a head detection unit detects the head of the target person from each of the images captured in time series;
A foot position estimation unit that estimates the foot position of the target person for each assumed height from each of the images captured in time series using the detected head; and
A real space position estimation unit that estimates the position of the target person in real space in time series for each assumed height using the estimated foot position;
The height estimation unit creates a human-like movement trajectory using the estimated time-series position in the real space for each assumed height, and the assumed height from which the most human-like movement trajectory was created is used as the height of the target person. A height estimation step to estimate;
A height estimation method characterized by comprising: An imaging step of imaging an image including the target person in time series;
A head detection step of detecting the head of the target person from each of the images captured in time series;
Using the detected head, a foot position estimation step for estimating the foot position of the target person for each assumed height from each of the images captured in time series,
Using the estimated foot position, a real space position estimating step for estimating the position of the target person in real space in time series for each assumed height;
For each assumed height, a human-like movement trajectory is created using the estimated time-series position in the real space, and a height estimation step is performed to estimate the assumed height from which the most human-like movement trajectory was created to the height of the target person. When,
Height estimation program to make the computer execute.

Images