JP4628910B2 - Length measuring device and height measuring device - Google Patents

Description

  The present invention relates to a length measuring device that measures the length of an object from a captured image, and more particularly to an apparatus that measures the entire length from an image of a part of the object.

  Conventionally, for example, Patent Document 1 discloses a technique for measuring the length of an object from a captured image. In the technique disclosed in Patent Document 1, a human is used as an object for measuring the total length. In a state where the human is standing on the floor surface used as the reference plane of the three-dimensional space, a two-dimensional image is obtained by photographing with a camera so that the full length of the human is reflected. However, calibration is performed before photographing, and it is determined which position in the two-dimensional image corresponds to each position in the three-dimensional space. A portion where a person is photographed is extracted from the two-dimensional image, and two-dimensional coordinates (Ub, Vb) and (Ut, Vt) of the foot and the top of the extracted image are determined. Since the photographed person is standing on the floor, it is determined that the position of the human foot is zero in the height direction (Z direction) in the three-dimensional space. Using this and the above-described calibration, the three-dimensional coordinates (X1, Y1, 0) of the human foot are determined from the two-dimensional coordinates of the position of the human foot. However, X1 is a coordinate in the horizontal direction (X direction) in the three-dimensional space, and Y1 is a coordinate in the depth direction (Y direction). The coordinates in the X and Y directions of the human foot and the top of the head are regarded as the same, and the coordinates of the human head in the Z direction are determined using X1, Y1, Ut, and Vt. The coordinates in the Z direction represent the total length of the human being photographed.

JP-A-2005-3377

  However, with this technique, it is necessary to photograph the entire length of a person. On the other hand, in order to capture the detailed characteristics of a human, it may be necessary to capture and record an upper body image of the human. The above technology cannot simultaneously meet these demands.

  An object of the present invention is to provide a length measuring device capable of measuring the entire length of an object from a two-dimensional image obtained by capturing a part of the object.

A length measuring device according to an aspect of the present invention includes a photographing unit. This photographing means photographs a subject to be photographed in a three-dimensional space. The three-dimensional space is defined by a first three-dimensional coordinate axis, a second three-dimensional coordinate axis orthogonal to the first three-dimensional coordinate axis, and a third three-dimensional coordinate axis orthogonal to the first and second three-dimensional coordinate axes. Yes. The to-be-photographed object has a third three-dimensional coordinate axis direction from a predetermined position where a position on the first or second three-dimensional coordinate axis of a plane orthogonal to the third three-dimensional coordinate axis in the three-dimensional space is known in advance. Has a length. For example, there is a floor as a plane through which the first and second three-dimensional coordinate axes pass, and a person standing at a predetermined position on the floor is the object to be photographed. Photographing means for photographing a part, for example, the upper body of a person from an upper end of the object to be photographed object. The imaging means captures an image from the upper end of the object to be imaged that moves substantially along the first or second three-dimensional coordinate axis through the predetermined position to a part below. A detecting means detects that the object to be photographed has passed through the predetermined position. After the detection means detects the passage of the person, a plurality of two-dimensional images are obtained from the photographing means at different times on the first or second three-dimensional coordinate axis of the object to be photographed. A position estimation means estimates the position. The plurality of two-dimensional images can be obtained at predetermined time intervals, or the time intervals at which the two-dimensional images are obtained can be different. For each of these respective two-dimensional images, to confirm the two-dimensional image of the upper end of the object to be photographed object being photographed, and the two-dimensional coordinates of the upper end, is captured on the two-dimensional image that is estimated by the position estimation means The position determining means determines the position on the third three-dimensional coordinate axis of the upper end of the object to be photographed from the position of the photographed object on the first or second three-dimensional coordinate axis. Prior to the position determination by the position determination means, the correspondence of each position in the three-dimensional space to the position in the two-dimensional image, that is, calibration is executed. The third position of the three-dimensional coordinate of the upper end of the plurality of the object to be photographed object is determined by the position determining means, and statistical processing means statistically processed, determine the length to the upper end from the lower end of the object to be photographed object To do.

The height measuring device according to another aspect of the present invention has a photographing means. This photographing means photographs a person in a three-dimensional space. The three-dimensional space is defined by a first three-dimensional coordinate axis, a second three-dimensional coordinate axis orthogonal to the first three-dimensional coordinate axis, and a third three-dimensional coordinate axis orthogonal to the first and second three-dimensional coordinate axes. Yes. The person is tall in the direction of the third three-dimensional coordinate axis from a predetermined position in which the position on the first or second three-dimensional coordinate axis of the surface orthogonal to the third three-dimensional coordinate axis is known in advance in the three-dimensional space. have. For example, there is a floor surface through which the first and second three-dimensional coordinate axes pass. The photographing means photographs from the top of the person moving through the predetermined position substantially along the first or second three-dimensional coordinate axis to a part below. The detecting means detects that the person has passed through the predetermined position. After this detecting means detects the passage of the person, the position on the first or second three-dimensional coordinate axis of the person when a plurality of two-dimensional images are obtained from the photographing means at different times, respectively. Position estimation means estimates. The plurality of two-dimensional images may be obtained at predetermined time intervals, or the time intervals at which the two-dimensional images are obtained may be different. For each of these two-dimensional images, a two-dimensional image of the head of the person being photographed is determined, and the two-dimensional image of the head of the head and the two-dimensional image estimated by the position estimating means are photographed. The position determining means determines the position of the top of the person on the third three-dimensional coordinate axis from the position of the person on the first or second three-dimensional coordinate axis. Prior to the position determination by the position determination means, the correspondence of each position in the three-dimensional space to the position in the two-dimensional image, that is, calibration is executed. The statistical processing means statistically processes the positions on the third three-dimensional coordinate axes of the heads of the plurality of persons determined by the position determining means to determine the height of the person.

  When a person moves, it may be considered that the passing speed is substantially constant. For example, this is a case where a person passes through a relatively short area such as a security gate. In such a case, if a person almost passes along the first or second coordinate axis, if the position of the installation position of the detection means on the first or second three-dimensional coordinate axis is known, the detection means is After the person passes, the position on the first or second three-dimensional coordinate axis of the person photographed in the plurality of two-dimensional images obtained from the photographing unit can be estimated for each two-dimensional image by the position estimating unit. . By using this estimated position and the two-dimensional coordinates of the person's head on each two-dimensional image, the height can be measured as described above. As a result, the height can be measured even if the person to be photographed does not stop. However, since the person is walking, the position of the top of the head varies depending on the degree of opening of the front and rear of the foot. Accordingly, the plurality of heights obtained are statistically processed by the statistical processing means to obtain the true value.

  As described above, according to the present invention, the total length of an object can be measured from an image obtained by capturing a part of the object.

  The length measuring apparatus according to the first embodiment of the present invention is installed in a security gate 2 such as an airport, for example. As shown in FIG. 1, this length measuring device has photographing means such as a video camera 4. The video camera 4 is arranged in the vicinity of the security gate 2 so that a predetermined position can be photographed on the entrance side of the security gate 2. This installation allows the subject to be photographed, for example, when the passerby 6 of the security gate 2 passes a predetermined position, the tip of the passerby 6, for example, a part below the head, for example, the upper body can be photographed. Has been done.

  As shown in FIG. 2, the area photographed by the video camera 4 is defined by a first three-dimensional coordinate such as an Xw axis, a second three-dimensional coordinate axis such as a Yw axis, and a third three-dimensional coordinate axis such as a Zw axis. It is a three-dimensional space. The Xw axis and the Yw axis are orthogonal to each other, and the Zw axis is orthogonal to the Xw axis and the Yw axis. The intersection 0w of these Xw axis, Yw axis, and Zw axis is the origin of the three-dimensional space. This three-dimensional space has a reference surface, for example, a floor 8 on which a security gate 2 is installed. In this embodiment, the floor 8 is on a plane through which the Xw axis and the Yw axis pass. And the passerby 6 moves on this floor 8 along the Yw-axis direction, for example. The positions on the Xw and Yw axes at the predetermined positions are known, and the position on the Yw axis is, for example, Y1.

  A video signal from the video camera 4 is input to a processing means such as a personal computer 12 via the video capture board 10 and displayed on a display screen of a display device 14 such as an LCD or CRT as shown in FIG.

  This display screen corresponds to the image plane XY of the video camera 4 shown in FIG. 2, and has a first two-dimensional coordinate axis, for example, a U-axis, and a first two axes with the upper left corner of the display screen as the origin O. It is defined by a second two-dimensional coordinate axis V-axis orthogonal to the dimensional coordinate axis. The upper body of the passerby 6 is projected on this two-dimensional imaging screen.

  The video camera 4 is calibrated by a program used by the personal computer 12, and in the three-dimensional space in FIG. 2 and the display screen of the display device 14, each position in the three-dimensional space corresponds to which position on the display screen. It is decided whether to do it. Since this calibration method is known, a detailed description thereof will be omitted.

  The personal computer 2 measures the height of the passerby 6 based on the video signal from the video camera 4. Therefore, first, as shown in FIG. 4, person detection is performed from the video signal from the video camera 4 (step S2). This person detection is performed by, for example, calculating a difference between one frame of the video signal of the video camera 4 in which the upper body of the passerby 6 is reflected and one frame of a video signal in which the same place is photographed in the absence of the passerby. As a result, the upper body photographing portion 6a of the passerby 6 is extracted. Note that various known methods other than those described above can be used as the person detection method.

  Next, the personal computer 12 determines, for example, a rectangular frame 18 and draws it on the display screen of the display device 14 so as to touch and surround the extracted upper body photographing portion 6a. The personal computer 12 determines the coordinates of the upper left corner of the frame 18 as (UL, VL) and the coordinates of the upper right corner as (UR, VR), and further determines the midpoint coordinate (UC) from these two coordinates. , VC). The midpoint coordinates represent the two-dimensional coordinates of the captured image of the top of the passerby 6. Thus, since the two-dimensional coordinate of the top of the passerby 6 is acquired, the personal computer 12 functions as a two-dimensional coordinate detection means.

  Although the personal computer 12 determines the two-dimensional coordinates of the top of the head, it can be specified by operating a pointing device such as a mouse (not shown) by a security gate operator to specify the top of the head. It is also possible for the personal computer 12 to calculate the two-dimensional coordinates of the top of the head from the position.

  By using the two-dimensional coordinates of the top of the passerby 6 obtained in this way and the Y coordinate Y1 (which is known in advance) among the three-dimensional coordinates of the position where the passer-by 6 stands, The height of the person is measured (step S4).

  Since the calibration is performed as described above, each position in the three-dimensional space being photographed can be converted into a corresponding position on the display screen of the display device 14. However, each position on the display screen cannot be directly converted to a corresponding position in the three-dimensional space.

  Therefore, in the process of step S4, a three-dimensional space passing through the top of the passerby in the center c of the video camera 4 and the image plane XY (corresponding to the display screen of the display device 14) of the video camera 4 shown in FIG. The vector A at is obtained, and the Zw coordinate Z1 of the point a through which the vector passes the top of the passerby is calculated. Here, since the Yw coordinate in the three-dimensional coordinate is known in advance as Y1, this is used.

Here, the relationship between the three-dimensional space coordinates Xw, Yw, Zw shown in FIG. 2 and the camera coordinates (x, y, z) of the video camera 4 is expressed by the following equation. The camera coordinates are coordinates having an x, y, and z axis parallel to the Xw, Yw, and Zw axes with the center c of the video camera as the origin. In the following equation, R is a 3 × 3 rotation matrix, and T is a translation vector [Tx, Ty, Tz] ^T.

  The coordinates (UC, VC) on the display screen are converted into coordinates Xd, Yd on the image plane XY of the video camera 4 by the following expression. However, the center Pc of the image plane XY is the origin, and this origin Pc corresponds to the origin c of the camera coordinates of the video camera 4. Further, uc and vc in the following expression represent the coordinates of the origin Pc of the image plane XY in U and V coordinates.

The coordinates (Xd, Yd) are converted into coordinates (Xu, Yu) in which the lens distortion in the video camera 4 is corrected. Since this conversion is publicly known, detailed description thereof is omitted. In FIG. 2, a ray vector Vr = [xv, yv, zv] ^T passing through the coordinates Xu, Yu on the XY plane from the camera coordinate origin c is expressed by the following equation. Here, f is a focal length in camera coordinates.

  When the light vector Vr is represented by a straight line in a three-dimensional space, the following expression is obtained. Where α is a real number.

Here, since Yw is found to be Y1, α can be determined by the following equation.

  By substituting the value of α obtained in Equation 5 into the equation related to Zw in Equation 4, Zw coordinates Z1 corresponding to UC and Vc on the display screen of the display device can be obtained by Equation 6.

  By using Equation 6, the total length of the passerby 6 can be measured from the two-dimensional image of the upper body. Thus, the personal computer 12 functions as a position determining unit.

  When the height is measured in this way, the height measured as shown in FIG. 4 and the upper body image of the passerby 6 are paired and stored in the storage means of the personal computer 12, such as a hard disk or memory. (Step S6).

  A length measuring apparatus according to a second embodiment of the present invention is shown in FIGS. In the height measuring device of the first embodiment, the height at the moment when the passerby 6 passes a predetermined position is measured. However, when a person walks, the position of the top of the head varies depending on the degree of opening in the front-rear direction of the foot, and therefore an error may occur depending on the timing of passage. The length measuring device of the second embodiment is an improvement of this point.

  A distance D through which the passerby 6 passes the security gate 2 indicated by a one-dot chain line in FIG. 5 is, for example, about 50 cm. When the passerby passes this short distance D, the passerby 6 is considered to go straight at almost constant speed. Therefore, a passer-by detector 20 is installed at the entrance side of the security gate 2, and a passer-by detector 22 is installed at the exit side of the security gate 2. By measuring the time until the passerby is detected by the traffic detector 22 and dividing the distance D by this time, the passing speed of the passerby 6 is determined. Note that various known detectors such as a photoelectric detector and an ultrasonic detector can be used as the entrance and exit detectors 20 and 22.

  If the passing speed is found and the passerby 6 moves substantially along the Yw coordinate axis as in the first embodiment, the video camera 4a can be used if the Yw coordinate of the installation position of the passer-in detector 20 is known. The Yw coordinate of the passer-by when the passer-by is shown in each frame of the video signal is calculated. For example, a frame is generated every 1/30 seconds, the passing time of the passerby 6 at the distance D is T, the position on the Yw coordinate axis of the passer-by detector 20 is Y1, and the passer-by detector 20 passes If the video camera 4a generates one frame when the passer 6 is detected, the Yw coordinate of the passer 6 when the passer 6 appears in the n (n is a positive integer of 1 or more) frame is: Y1 + (D / T) * (n-1) * 1/30. Thereafter, the position of the passer 6 on the Yw coordinate axis can be estimated for each frame obtained until the passer 6 is detected by the passer-out detector 22 in the same manner.

  Accordingly, as in the first embodiment, if the position of the passerby 6 on the Yw coordinate axis is estimated for each frame by the personal computer 12 using the video camera 4a, the capture board 10, the personal computer 12, and the display device 14, After that, as in the first embodiment, α can be calculated for each frame, whereby the position on the Zw coordinate axis of the top of the head is determined for each frame, and the height of the passerby is measured for each frame. can do. As shown in FIG. 5, the video camera 4a is installed on the exit side of the security gate 2 so that the upper body of the passerby 6 can be photographed between the entrance and the exit of the security gate 2. Thus, the personal computer 12 functions as a position estimation unit. Note that it is not always necessary to estimate the position of the passerby for each frame, and the position of the passerby 6 may be estimated for each of a plurality of frames obtained at appropriate intervals.

  However, since the passerby 6 opens his / her legs back and forth while passing, the top of the head moves up and down while passing. For this reason, the height measured during traffic also changes up and down as shown in FIG. Therefore, statistical processing is performed by the personal computer 12 to determine the height of the stationary upright state. For example, the average value and intermediate value of each measured height (the value obtained by adding the maximum value and the minimum value among a plurality of target data and dividing by 2; hereinafter, the term of the intermediate value is used in the above definition) Alternatively, the median can be a true value.

  Alternatively, among the measured heights, the height regarded as a mid-stance state can be estimated by statistical processing. In other words, when a person walks, the state where the center of gravity is placed on one leg and the leg is directly under the upper body is called the middle stance phase. Closest to the height of the person at the time. Therefore, the height when the head is in the mid-stance state is extracted from the measured height of the passerby 6 whose head is passing up and down. As a result, a value closest to the true height is obtained.

  The measured value of the person's height is not considered to be larger than the measured height when standing upright in the case of normal walking where the person does not walk while stretching out. Therefore, as a means for estimating the height in the mid-stance state of stance, the height in the mid-stance state, that is, the substantially stationary upright state, can be obtained by using the one that extracts the maximum value among the measured heights. Alternatively, an average value, an intermediate value, or a median value of each measured height can be regarded as a height in a mid-stance state. In addition, the maximum value of each measurement height may be a value far from the true middle stance state due to measurement error. That is, the outlier value may be included in each measurement height. If this is the case, arrange the measured heights in order of size, and remove the predetermined percentage of the measured heights from the largest value of the measured heights, and determine the maximum value from the remaining measured heights. It can be considered as the height of the state and extracted.

  Alternatively, as a means for estimating the height when in the middle stance state, in addition to extracting the maximum value, the height of the person is determined from the measured height that is equal to or greater than the value obtained by adding the standard deviation to the arithmetic average value of each measured height You can also use what you want. For example, if each measurement height is arranged in the order of size and the maximum value is extracted from the remaining measurement heights after removing a predetermined proportion of measurement heights from the largest value among them, the measurement height is not an outlier May also be removed. Therefore, the measurement height having a value equal to or greater than the average value of each measurement height is regarded as data near the middle of the stance, and the height is determined from those data. Prevent measurement height removal and improve measurement accuracy. In addition, when estimating a height from the measured value more than the value which added the standard deviation to the average value, the maximum value and the intermediate value of the measured values more than the value which added the standard deviation to the average value can be extracted.

  Furthermore, the arithmetic average value or median value of the measured values equal to or greater than the value obtained by adding the standard deviation to the arithmetic average value of each measured height can be set as the height of the person. Various methods can be considered as to which of the measured values equal to or greater than the value obtained by adding the standard deviation to the average value as the measured value in the true middle stance state. However, a measured value that is equal to or greater than the average value plus the standard deviation may include outliers. For example, when the maximum value and the intermediate value are true heights as described above, if the measured value is equal to or greater than the value obtained by adding the standard deviation to the average value, and the outlier is included, the maximum value or the intermediate value is Largely affected by outliers. On the other hand, when using the arithmetic average value or median value of the measured values equal to or greater than the value obtained by adding the standard deviation to the arithmetic average value of each measured height, the value obtained by adding the standard deviation to the arithmetic average value of each measured height Even if outliers are included in the measured values, the influence of the outliers can be reduced, and values close to the true height can be stably obtained.Therefore, the standard deviation is added to the arithmetic average value of each measured height. The arithmetic mean or median of the measured values that are equal to or greater than the added value is used.

  Thus, the personal computer 12 functions as a statistical processing means.

  In both the above-described embodiments, the capture board 10, the personal computer 12, and the display device 14 are used. However, the present invention is not limited to this. The present invention can also be implemented using a digital signal processing device such as a DSP. In the above embodiment, the position Y1 of the passerby 6 on the Yw coordinate axis is known. However, even when the position on the Xw coordinate axis is known, α can be calculated. , Can measure the height of passers-by. In the first embodiment, the present invention is implemented to measure the height of a walking human. However, the present invention is not limited to this. For example, an upper body image of a person standing still in front of a bank counter is recorded. The height can be measured and recorded while the height can be accurately measured since there is no vertical movement during walking. In both the above embodiments, the present invention is implemented to measure the height of a person, but the present invention is not limited to this. For example, when measuring the length of a product manufactured in a factory or the like, In this case, since it is not necessary to install a video camera so that the entire length can be photographed, there is no restriction on the installation location of the video camera. In both of the above embodiments, a video camera is used, but a digital still camera can also be used. In the second embodiment, the pass-in and pass-by detectors 20 and 22 are provided. For example, when the pass-through speed of the passer-by 6 is constant and the value is known, the pass-through detection is performed. Only the vessel 20 may be provided.

It is a block diagram of the length measuring device of a 1st embodiment of the present invention. It is explanatory drawing of the measurement principle of the length measuring apparatus of FIG. It is a figure which shows the image displayed on the display apparatus of the length measuring apparatus of FIG. It is a flowchart of the length measuring apparatus of FIG. It is a schematic block diagram of the length measuring apparatus of the 2nd Embodiment of this invention. It is a figure which shows the relationship between the measured value and measurement position by the length measuring apparatus of FIG.

Explanation of symbols

4 Video cameras (photographing means)
12 Personal computer (position determination means)

Claims (2)

A third three-dimensional coordinate axis in a space defined by the first three-dimensional coordinate axis, the second three-dimensional coordinate axis orthogonal to the first three-dimensional coordinate axis, and the third three-dimensional coordinate axis orthogonal to the first and second three-dimensional coordinate axes; A position perpendicular to the three-dimensional coordinate axis on the first or second three-dimensional coordinate axis has a length in a direction from the predetermined position to the third three-dimensional coordinate axis, and passes through the predetermined position; The second two-dimensional coordinate axis orthogonal to the first two-dimensional coordinate axis and the first two-dimensional coordinate axis from the upper end of the object to be photographed moving substantially along the first or second three-dimensional coordinate axis to a part below the object. Photographing means for photographing in two dimensions defined by
Detecting means for detecting that the object to be photographed has passed through the predetermined position;
After the detection means detects the passage of the object to be photographed, the plurality of two-dimensional images respectively obtained at different times from the photographing means are used on the first or second three-dimensional coordinate axis of the object to be photographed. Position estimation means for estimating the position of
The two-dimensional coordinates of the upper end of the object to be photographed for each of the two-dimensional images are determined, and the two-dimensional coordinates and the first or second tertiary of the object to be photographed estimated by the position estimating unit. Position determining means for respectively determining a position on the third three-dimensional coordinate axis of the upper end of the object to be photographed from a position on the original coordinate axis;
The third position of the three-dimensional coordinate of the upper end of the plurality of the object to be photographed object determined by the position determining means and statistical processing, the statistical processing means for determining the length from the lower end of the shooting object to the upper end And
A length measuring device. In a space defined by the first three-dimensional coordinate axis, the second three-dimensional coordinate axis orthogonal to the first three-dimensional coordinate axis, and the third three-dimensional coordinate axis orthogonal to the first and second three-dimensional coordinate axes; The head of a person who moves substantially along the first or second three-dimensional coordinate axis through a predetermined position where the position on the first or second three-dimensional coordinate axis of the plane orthogonal to the three-dimensional coordinate axis is known in advance. An imaging means for imaging in a two-dimensional manner defined by a first two-dimensional coordinate axis and a second two-dimensional coordinate axis orthogonal to the first two-dimensional coordinate axis from the top to a part below;
Detecting means for detecting that the person has passed through the predetermined position;
After the detecting means detects the passage of the person, the position of the person on the first or second three-dimensional coordinate axis is estimated for each of a plurality of two-dimensional images respectively obtained at different times from the photographing means. Position estimation means for
The two-dimensional coordinates of the head of the person photographed for each of the two-dimensional images are determined, and the two-dimensional coordinates and the first or second three-dimensional coordinate axis of the person estimated by the position estimating means Position determining means for determining the position on the third three-dimensional coordinate axis of the person's head from the position of
Statistical processing means for statistically processing the positions on the third three-dimensional coordinate axes of the heads of the plurality of persons determined by the position determination means to determine the height of the person;
Equipped with a height measuring device.

Images