JP3996864B2 - Surveillance device using stereo images - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a monitoring apparatus that monitors a monitoring object that moves in a three-dimensional monitoring space using a stereo image.
[0002]
[Prior art]
Regarding the measurement of the position information of the monitoring object, there is described an apparatus that uses two imaging devices and measures the position information of the monitoring object moving in a three-dimensional monitoring space from the difference image of the stereo image by trigonometry. Has been. A method using such a stereo image can acquire three-dimensional information by differential processing, and is generally used (see Patent Document 1).
[Patent Document 1]
Japanese Patent Laid-Open No. 2002-175513 (paragraphs 0034 to 0040, 0064, FIG. 4, FIG. 8)
[0003]
[Problems to be solved by the invention]
However, the method using a stereo image requires a high processing capability in the calculation part, and there is a problem that the apparatus scale becomes large and the processing time becomes long. On the other hand, if the number of scanning lines is reduced to speed up the processing, there is a problem that the measurement accuracy is lowered. In addition, there is a problem that the resolution of the distance decreases as the object to be monitored becomes farther away, or that the object to be monitored enters between the scanning lines and cannot be detected in the farther place.
[0004]
The present invention provides a monitoring device having a simple configuration capable of obtaining position information of a monitoring object moving in a three-dimensional monitoring space using a stereo image at high speed and with high accuracy. In particular, the monitoring object is located far away. Even in such a case, an object is to provide a monitoring device capable of measuring with the same accuracy as in the case of being nearby.
[0005]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the monitoring device using a stereo image according to claim 1 is a monitoring device 1 using a stereo image for monitoring a monitoring object 12 moving in a three-dimensional monitoring space 10. The first imaging device 16 that is installed toward the monitoring space 10 and captures the monitoring object 12 that changes over time is separated from the first imaging device 16 by a predetermined interval w, and A second imaging device 18 that is installed toward the monitoring space 10 and images the monitoring object 12 that changes over time, and an image that is captured by the first imaging device 16 at a first time T0. A first difference image forming unit 24 that sequentially forms a first difference image that is a difference image from an image captured at a second time T1 delayed by a predetermined time ΔT from the first time T0; The first imaging device 18 allows the first imaging device Second difference image forming means 25 for sequentially forming a second difference image, which is a difference image between the image taken at time T0 and the image taken at the second time T1, and the first difference image And the second difference image, the respective areas occupied by the monitoring object 12 on the first difference image and the second difference image are defined as a first dynamic region and a second dynamic image, respectively. The dynamic area extracting means 27 that extracts the area and the scanning lines that scan the same portion of the monitoring object 12 from the scanning lines of the first imaging device 16 and the second imaging device 18 are mutually connected. A scanning line extracting unit 32 that extracts a predetermined number n of scanning lines from the scanning lines corresponding to each other, and the predetermined number in the first imaging device 16 and the second imaging device 18 as corresponding scanning lines. n scan lines, respectively, for the first dynamic region and the first Based on the first dynamic region and the second dynamic region formed by using the scanning line concentrating means 33 for concentrating and scanning the two dynamic regions, and the concentrated and scanned scanning lines, Distance information calculating means 26 for calculating the distance of the monitoring object 12 from a predetermined reference line is provided.
[0006]
Here, the first imaging device 16 and the second imaging device 18 according to claim 1 typically use a sensor such as a CCD sensor or a CMOS sensor, and use a triangulation method to measure the distance. A detection device can be configured.
[0007]
With this configuration, by concentrating the scanning lines on the operation area of the monitoring object 12, position information of the monitoring object 12 moving in the three-dimensional monitoring space 10 is obtained with high speed and high accuracy using a stereo image. It is possible to provide a monitoring device 1 having a simple configuration that can be measured, in particular, the monitoring device 1 that can measure with the same accuracy as when the monitoring object 12 is located in the distance.
[0008]
According to a second aspect of the present invention, the monitoring apparatus using the stereo image according to the first aspect is based on the distance information storage means 39 for storing the distance and its change with time, and the change with time of the distance. A determination unit 28 is provided for determining a movement state of the monitoring object.
[0009]
If comprised in this way, the moving direction of the monitoring target object 12 can be detected efficiently from the time-dependent change of the distance of the monitoring target object 12.
[0010]
The monitoring apparatus using the stereo image according to claim 1 or 2 according to the invention of claim 3, wherein the dynamic region extraction means 27 constitutes the first difference image or the second difference image. For each scanning line that has been scanned in a concentrated manner from each pixel, the absolute value of the pixel value is larger than the first threshold, and the region where the signal of the difference image occurs is sandwiched between the region where there is no signal An annular pixel region formed by extracting one or more pairs of pixel regions and connecting each pair of pixel regions extracted by adjacent scanning lines to each other, or surrounding the annular pixel region The planar pixel areas to be extracted are extracted as the first dynamic area or the second dynamic area, respectively.
[0011]
With this configuration, in many cases, the contour of the monitoring object 12 becomes a boundary where a difference image signal is generated, so that the dynamic region of the monitoring object 12 can be efficiently grasped. Here, when the person 12 moves his / her hand away from the torso or opens his / her leg, the scanning lines for scanning these portions cross the person 12 a plurality of times, This is because the area corresponding to the contour becomes a boundary where a signal of the difference image is generated. In addition, there are two types of dynamic areas: a pixel area of the contour portion of the person 12 and a pixel region surrounded by the contour portion. Here, if there is a portion with little movement in the outline of the person, the absolute value of the brightness difference is small, so that the extracted dynamic region is a part lacking a ring shape, Even if it is missing, if a portion with a large absolute value of the brightness difference is extracted to some extent, later correlation processing and calculation of distance can be performed, so a part of the ring may be missing. Similarly, a part of the planar shape may be missing. That is, the annular pixel region includes a part lacking. Further, the planar pixel region includes a part lacking.
[0012]
The monitoring apparatus using the stereo image according to claim 1 or 2 according to the invention described in claim 4, wherein the dynamic region extraction means 27 constitutes the first difference image or the second difference image. For each of the scanning lines that are scanned in a concentrated manner, the pixel value having an absolute value greater than the first threshold value is extracted for each of the concentrated scanning lines, and the outermost pair of pixel areas is extracted. An annular pixel region formed by connecting the extracted pairs of pixel regions to each other, or a planar pixel region surrounding the annular pixel region, respectively, the first dynamic region or the Extract as a second dynamic region.
[0013]
With this configuration, the outermost pair of pixel regions is extracted as a dynamic region for each scanning line, so that the monitoring object 12 can be reliably placed inside the dynamic region. In addition, there are two types of dynamic areas: a pixel area of the contour portion of the person 12 and a pixel region surrounded by the contour portion. A part of the annular or planar shape may be missing. That is, the annular pixel region includes a part lacking. Further, the planar pixel region includes a part lacking.
[0014]
In the monitoring apparatus using the stereo image according to any one of claims 1 to 4 according to the invention described in claim 5, the scanning line concentration unit 33 may be configured to detect the first dynamic region or the second dynamic region. Among the constituent pixels, the uppermost pixel is between the uppermost scanning line and the next upper scanning line, and the lowermost pixel is between the lowermost scanning line and the next lower scanning line. The predetermined number n of scanning lines are concentrated and scanned in the first dynamic area or the second dynamic area.
[0015]
With this configuration, when the uppermost scanning line scans the lower side of the uppermost pixel, or when the next lower scanning line scans the upper side of the lowermost pixel, The range in which a predetermined number n of scanning lines are concentrated is widened, and when the uppermost scanning line scans the upper side of the uppermost pixel, or the lower side of the lowermost pixel is the upper next When the scanning line becomes the scanning line, the scanning line interval is changed again so that the range where the predetermined number n of scanning lines are concentrated is narrowed. A predetermined number n of scanning lines are permanently concentrated on the monitoring object 12 by adjusting the lowermost pixel between the lowermost scanning line and the next lower scanning line. Thus, the operation of the monitoring object 12 can be grasped with a certain accuracy.
[0016]
In the monitoring apparatus using the stereo image according to any one of claims 1 to 5 according to the invention described in claim 6, the scanning line concentration unit 33 may be configured to detect the first dynamic region or the second dynamic region. When the extraction cannot be performed, the state of the scanning line in the immediately preceding frame is maintained, and when the first dynamic region or the second dynamic region cannot be extracted more than a predetermined number of times, A predetermined number n of scanning lines are extended to the entire monitoring space 10.
[0017]
With this configuration, when the dynamic area cannot be extracted, the monitoring target 12 is regarded as stationary, and the state of the scanning line in the immediately preceding frame is maintained, and monitoring can be continued. Further, when the dynamic region cannot be extracted a predetermined number of times or more, the scanning range of the scanning line can be expanded and the position of the monitoring object 12 can be reconfirmed.
[0018]
In the monitoring apparatus using the stereo image according to any one of claims 1 to 6 according to the invention described in claim 7, the distance information calculation unit 26 includes the first dynamic region, the second dynamic region, The correlation output value related to the parallax between the first imaging device 16 and the second imaging device 18 is calculated from the correlation between the first imaging device 16 and the second imaging device 18, and a predetermined value of the monitoring object 12 is determined from the correlation output value using trigonometry. The distance from the reference line 43 is calculated.
[0019]
If comprised in this way, the distance information of the monitoring target object 12 can be efficiently grasped | ascertained with a simple apparatus structure.
[0020]
The monitoring apparatus using the stereo image according to claim 7 according to the invention according to claim 8, wherein the distance information calculation means 26 includes a correlation output value indicating a maximum correlation peak value among the correlation output values, and A correlation output value indicating the second correlation peak value is extracted, and the maximum correlation peak value is determined only when the difference between the maximum correlation peak value and the second correlation peak value is greater than a second threshold. The distance is calculated from the correlation output value shown.
[0021]
With this configuration, since the correlation peak value is usually a single true correlation output value and becomes a large value, noise can be prevented from being mistakenly recognized as the contour of the monitoring object 12.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol or a similar code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted.
[0023]
FIG. 1 is a schematic partially broken perspective view of a monitoring device 1 according to an embodiment of the present invention. The monitoring device 1 monitors a person 12 as a monitoring object that moves or enters a three-dimensional monitoring space 10. The three-dimensional monitoring space refers to an area that can be defined by, for example, orthogonal coordinates in the horizontal direction (X axis), the depth direction (Y axis), and the vertical direction (Z axis). A person 12 exists on the floor surface 3 which is the XY plane, and there are wall surfaces on the XZ plane and the YZ plane.
[0024]
The monitoring device 1 is installed toward the monitoring space 10 and is separated from the first imaging device 16 for imaging a person 12 whose shape changes over time by a predetermined interval w. A second imaging device 18 that images the person 12 that is installed toward the space 10 and whose shape changes with time is provided. The first imaging device 16 and the second imaging device 18 are, for example, wall surfaces (XZ plane). ) 4 to image the person 12 in the monitoring space 10. The first and second imaging devices 16 and 18 use CCD sensors as sensors. In addition, an infrared light emitting element 17 is disposed around the objective lens, connected to the IF unit by a cable, controlled to automatically light in the dark state, and can be monitored day and night. A CMOS can also be used as a sensor.
[0025]
The first imaging device 16 and the second imaging device 18 are connected to the control device 2 via a communication cable, and transmit a video signal to an IF unit 19 that is an interface of the control device 2. The control device 2 includes an IF unit 19, a calculation unit 30 connected to the IF unit 19, a storage unit 31 connected to the calculation unit 30, and a scanning line control unit 29 connected to the IF unit 19 and the calculation unit 30. The input unit 37 connected to the calculation unit 30 and the transmission unit 38 connected to the IF unit 19 and the calculation unit 30 are provided.
[0026]
An input device 35 for inputting information for operating the monitoring device 1 is connected to the control device 2 via an input unit 37, and an output device 36 and an imaging device for outputting calculation results and determination results from the monitoring device 1. An image recording / reproducing device 34 that records and reproduces video signals from 16 and 18 is connected via a transmission unit 38. The input device 35 is, for example, a touch panel, a keyboard, a mouse, and the like, and the output device 36 is a display, a printer, an alarm device, or the like. In FIG. 1, the input device 35 and the output device 36 are illustrated as externally attached to the control device 2, but may be built in. Further, the control device 2 may include a light emission control unit (not shown) that controls turning on and off of the infrared light emitting element 17.
[0027]
The storage unit 31 stores an imaging signal storage unit (CCD1: T0) 20 that stores an imaging signal captured at the first time T0 by the first imaging device 16, and stores an imaging signal captured at the second time T1. An imaging signal storage unit (CCD1: T1) 21, an imaging signal storage unit (CCD2: T0) 22 for storing an imaging signal captured at the first time T0 by the second imaging device 18, and an imaging at the second time T1 An imaging signal storage unit (CCD2: T1) 23 that stores imaging signals and a position information storage unit 39 that stores positional information of the monitoring target 12 are provided. The position information includes distance information and position information in the X and Z directions, and the position information storage unit 39 has a function as a distance information storage unit. It also stores other data that can be acquired during the calculation.
[0028]
In the arithmetic unit 30, a first difference image forming unit (CCD 1: difference) 24 that forms a first difference image by the first imaging device 16 and a second difference image by the second imaging device 18 are formed. A second difference image forming unit (CCD2: difference) 25, a position calculating unit 26 for calculating position information of a monitoring object such as a person 12, and a first motion from the first difference image and the second difference image, respectively. A dynamic region extraction unit 27 that extracts a target region and a second dynamic region, and a determination unit 28 that determines a movement state of a person that is the monitoring target 12 from a temporal change in position information. The first difference image forming unit 24 and the second difference image forming unit 25 have functions as a first difference image forming unit and a second difference image forming unit, respectively, and the position calculation unit 26 is a distance information calculation unit. The dynamic region extraction unit 27 has a function as a dynamic region extraction unit, and the determination unit 28 has a function as a determination unit.
[0029]
The scanning line control unit 29 includes a scanning line selection unit 32 having a function as a scanning line selection unit for selecting a scanning line to be synchronized and a scanning line concentration unit 33 for concentrating a predetermined number n of scanning lines in a dynamic region. In addition, control is performed to synchronize the scanning lines corresponding to each other in the first imaging device 16 and the second imaging device 18.
[0030]
The operation of the monitoring device 1 will be described with reference to a partially broken perspective view of FIG. In the monitoring space 10, for example, an inner edge portion 43 defined along the wall surface 4 on the floor surface 3 is set in order to set a range for monitoring the person 12 approaching or moving away from the wall surface 4. An outer edge portion 42 that is defined far away from the inner edge portion 43 by a predetermined distance D (corresponding to a predetermined reference line) is set. The first and second imaging devices 16 and 18 are installed on the inner edge 43.
[0031]
Here, the first imaging device 16 has the first number of scanning lines, the second imaging device 18 has the second number of scanning lines, and both the imaging devices 16 and 18 have the scanning line direction. Commonly installed in parallel. In the present embodiment, the first imaging device 16 and the second imaging device 18 are described as separate bodies, but may be configured as a single unit. The monitoring space 10 is a region having a predetermined distance D sandwiched between the outer edge portion 42 and the inner edge portion 43, and the first and second imaging devices 16 and 18 are configured to detect the person 12 moving in the monitoring space 10. Take an image. In the present embodiment, the first imaging device 16 and the second imaging device 18 are installed on the wall surface 4 and oriented in the horizontal direction. However, the walls and ceiling are inclined toward a three-dimensional monitoring space. May be installed.
[0032]
In addition, the number of scanning lines of the first imaging device 16 and the second imaging device 18 is typically the same, and the number of scanning lines of each imaging device is, for example, 500, and each scanning line. Has a pixel of 500 pixels. In addition, it is desirable that the first imaging device 16 and the second imaging device 18 are installed horizontally in the scanning line direction.
[0033]
The control device 2 receives imaging signals from the first imaging device 16 and the second imaging device 18, and extracts images formed on the scanning lines (horizontal direction) corresponding to each other in synchronization. The scanning lines corresponding to each other means scanning the same part of the monitoring object 12. As will be described later, correlation calculation and distance calculation are performed from the pixel data on the scanning lines corresponding to each other. Thus, the pixel data of recently captured images of the first imaging device 16 and the second imaging device 18 are compared and contrasted, and the correspondence relationship between the scanning lines is grasped in advance.
[0034]
The scanning line selection unit 32 has a function as scanning line extraction means, and extracts a predetermined number n of scanning line pairs from scanning lines corresponding to each other as scanning lines to be synchronized. The predetermined number n is 2 or more, but can be freely selected according to the distance and position of the monitoring object 12. The predetermined number n is, for example, a level that protects the privacy of the person 12 or a level that does not slow down the processing time as the processing amount increases, and may be about 10 to 20. It can be selected at regular intervals, randomly selected, or appropriately changed according to the purpose of use.
[0035]
About 500 scanning lines (excluding those that do not correspond since the scanning lines corresponding to each other may be displaced in the vertical direction) included in each imaging apparatus may be extracted, and the number of scanning lines that divide the imaging region at a predetermined interval may be extracted. Scan lines may be extracted. For example, when detecting a person 12, a scanning line that captures an image with an interval of about 100 mm is extracted, and when detecting a small animal, a scanning line that captures an image with an interval of about 50 mm is extracted, and all other close objects are detected. In this case, it is possible to individually set scanning lines for picking up images with an interval of about 10 mm. For example, in order to detect that any part of the person 12 has entered the monitoring space 10, when at least about 500 scanning lines of each imaging device are used, about 20 scanning lines are provided at regular intervals. Extract it. In the present embodiment, 20 scanning lines are extracted as a predetermined number n.
[0036]
As another case, it is assumed that the number of scanning lines of the first imaging device 16 is 500 and the number of scanning lines of the second imaging device 18 is 600. In this case, if the entire imaged area is the same, the image scanned with the five scanning lines of the first imaging device 16 and the image scanned with the six scanning lines of the second imaging device 18 are the same. Make correspondence. That is, one of the five scanning lines of the first imaging device 16 and one of the six scanning lines of the second imaging device 18 are scanning lines corresponding to each other. Therefore, the number of scanning lines corresponding to each other is about 100 (except for the case where the scanning lines corresponding to each other are displaced in the vertical direction, so that they do not correspond). Even in this case, 20 scanning lines can be extracted from about 100 corresponding scanning lines as a predetermined number n. When the time required for one scan in the first imaging device 16 and the time required for one scan in the second imaging device 18 are different, the scanning start time of each scanning line is synchronized. Let Furthermore, the scanning lines to be synchronized may be extracted in time series.
[0037]
The monitoring device 1 receives the image information of the person 12 captured by the first imaging device 16 at the time T0 through the IF unit 19, and the imaging signal storage unit at the first time T0 of the storage unit 31 via the arithmetic unit 30. Store to 20. In parallel, the image information of the person 12 imaged at the time T0 by the second imaging device 18 is received through the IF unit 19, and the imaging signal storage unit 22 at the first time T0 of the storage unit 31 is received via the arithmetic unit 30. Remember me. The image information is typically array data of pixel values (lightness) of pixels constituting the image.
[0038]
Subsequently, the image information of the person 12 at time T1 captured by the first imaging device 16 after a predetermined time ΔT from the time T0 is received through the IF unit 19, and the second time in the storage unit 31 is obtained via the arithmetic unit 30. The image is stored in the imaging signal storage unit 21 of T1. In parallel, the image information of the person 12 at time T1 imaged by the second imaging device 18 after a predetermined time ΔT from the time T0 is received through the IF unit 19, and the second information in the storage unit 31 is received via the arithmetic unit 30. Is stored in the imaging signal storage unit 23 at time T1.
[0039]
The first difference image forming unit 24 and the second difference image forming unit 25 in the calculation unit 30 form a difference image from image information of each image formed on the scanning line included in each imaging signal. It functions as a first difference image forming unit and a second difference image forming unit.
[0040]
The first difference image forming unit 24 sequentially calculates a first difference image that is a difference image between the image of the person 12 at time T0 and the image of the person 12 at time T1 captured by the first imaging device 16. The first difference image is temporarily stored. In parallel, the second difference image forming unit 25 outputs a second difference image that is a difference image between the image of the person 12 at time T0 and the image of the person 12 at time T1 captured by the second imaging device 18. A sequential calculation process is performed to temporarily store the second difference image. These difference images are image information indicating the contour (edge) of the person 12 from which the moved part of the person 12 is extracted.
[0041]
The two images for forming the difference image are respectively acquired at the imaging time T0 and the imaging time T1 delayed by a predetermined time ΔT from the imaging time T0. The delay time (predetermined time) ΔT to be shifted is The amount of movement does not become too large, and the time can be regarded as substantially the same position, for example, about 0.1 seconds. Or it is set as 1-10 periods (1/30 second-1/3 second) of a television period. When such a difference image is calculated and acquired, the background of the person 12 is removed, and only an image of the person 12 with movement can be extracted.
[0042]
Since the images formed on the scanning lines of the imaging devices 16 and 18 do not move in the background, their pixel values change abruptly at the boundary of the moving person 12. In the present embodiment, the dynamic region extraction unit 27 in the calculation unit 30 reads out each difference image from the first difference image forming unit 24 and the second difference image forming unit 25, and the monitoring target 12 has each difference. It has a dynamic area extraction function for extracting areas occupied on the image as a first dynamic area and a second dynamic area, respectively. Among the pixels in which the absolute value of each pixel value (lightness difference) constituting the difference image is larger than the first threshold value, the boundary pixel where the signal of the difference image is generated is regarded as the boundary of the person 12, and each dynamic region Extract as In the present embodiment, a case where an annular pixel region is extracted as a dynamic region will be described.
[0043]
Usually, the absolute value of the pixel value (brightness difference) increases at the contour of the monitoring object 12. The difference image includes a brightened pixel and a darkened pixel. For example, if the pixel value when the image becomes bright is +, the pixel value when the image becomes dark is-. The first threshold value is provided to remove noise.
[0044]
FIG. 2 shows an example of an image photographed by the imaging device in the monitoring device 1 of the present embodiment. 2A and 2B show examples of images taken by the first imaging device at times T0 and T1, respectively. Between time T0 and time T1, if both the arms are separated from the torso and the person 12 with both legs open moves to the left as it is, the shape changes from a dotted line in FIG. 2B to a solid line. A portion between the solid line and the dotted line is formed as the first difference image, and the first difference image substantially coincides with the one representing the outline of the person.
[0045]
2A and 2B, the scanning lines are m1 to m13 from the top. On the scanning line m4, two areas p1 and p2 are areas where the absolute value of the brightness difference is large. Since the position of the face changes as the person 12 moves, a difference image signal is generated between p1 and p2, and the pixel value is not zero. On the left side of p1 and the right side of p2, the pixel value of the difference image is 0 because the background is unchanged. On the scanning line m7, six regions p3 to p8 are regions where the absolute value of the brightness difference is large. Since the position of the clothes changes with the movement of the person 12, a difference image signal is generated between p3 and p4, between p5 and p6, and between p7 and p8, and the pixel value is not zero. On the other hand, on the left side of p3, between p4 and p5, between p6 and p7, and on the right side of p8, the pixel value of the difference image is 0 because the background is unchanged. In the scanning line m10, there are four areas from p9 to p12 where the absolute value of the brightness difference is large. A difference image signal is generated between p9 and p10 and between p11 and p12, and the pixel value is not zero. The pixel value of the difference image is 0 on the left side of p8, between p9 and p10, and on the right side of p8.
[0046]
Therefore, it can be said that the pixel areas where the absolute value of the brightness difference is large are areas where a difference image signal is generated, that is, a pair of areas sandwiching an area occupied by a person. That is, p1 and p2, p3 and p4,..., P11 and p12 are pixel regions that form a pair on the boundary with a region where there is no signal and sandwich the region where the signal of the difference image is generated. It is a pixel area which makes a pair across the area occupied by. An annular pixel region formed by collecting such pixel regions and connecting each pair of pixel regions extracted by adjacent scanning lines to each other is extracted as a first dynamic region. That is, the extracted first dynamic area substantially coincides with a pixel area where the absolute value of the brightness difference of the difference image is large, and is extracted as an area substantially occupied by the person 12 on the difference image. The first dynamic area extracted in this way represents the outline of a person. Here, if there is a portion with little motion in the outline of the person, the absolute value of the brightness difference is small, so that the extracted dynamic region lacks a part of the annular shape. However, even if a portion is missing, if a portion with a large absolute value of the brightness difference is extracted to some extent, later correlation processing and distance calculation are possible.
[0047]
In addition, a difference image is formed between the scanning line m4 and the scanning line m11, but a difference image is not formed between the scanning line m3 and the scanning line m12. Therefore, the upper end of the first dynamic area is between m3 and m4. It can be seen that the lower end of one dynamic region is between m11 and m12. Also, when moving to the right, the same difference image and points constituting the dynamic region can be obtained. Also, when the person in front moves forward, the outline of the person enlarges, and when the person moves backward, the outline of the person shrinks, so that a similar difference image is formed and the first motion is formed. Target area can be extracted. Similarly, the second dynamic region can be extracted from the second difference image.
[0048]
When a plurality of pairs of pixel regions are generated as in the scanning lines m7 and m10, a pair of pixel regions on the outermost side on the scanning line, p3, is used as a point constituting the first dynamic region. And p8, and only p9 and p12 may be selected. A pair of pixel regions selected in this way are extracted for each scanning line, collected, and an annular pixel region formed by connecting each pair of pixel regions extracted by adjacent scanning lines is defined as a first pixel region. Even if it is extracted as one dynamic region, the region occupied by the person 12 on the difference image is included inside the first dynamic region. If the second dynamic region is extracted in the same manner, when the correlation process is performed, the correlation process is performed except for the contours such as the inner side of the limbs. In many cases, substantially the same result as when a plurality of pairs are selected and correlation processing is performed can be obtained.
[0049]
If the person who is the monitoring target 12 is stationary, the dynamic area is not extracted. Therefore, when no dynamic region is extracted, the person is considered to be stationary.
[0050]
As described above, the calculation unit 30 does not perform the dynamic region extraction processing for all the scanning lines, but performs the processing for a predetermined number (20 in this embodiment) of scanning lines corresponding to each other. An area where the person 12 has moved can be extracted.
[0051]
By the way, in order to monitor the movement of the person 2, it is sufficient to concentrate a predetermined number of scanning lines on the dynamic region extracted for the person 12, without spreading it over the entire area of the captured image.
[0052]
FIG. 3 shows a state of scanning lines for scanning the person 12 when a predetermined number n of scanning lines are concentrated on the person 12. 3A shows a state where the person 12 is at a short distance from the monitoring apparatus 1, and FIG. 3B shows a state where the person 12 is at a long distance from the monitoring apparatus 1. If the person 12 is near the inner edge 43, the dynamic area becomes larger, if the person 12 is near the outer edge 42, the dynamic area becomes smaller, and as the person 12 approaches the inner edge 43 from the outer edge 42, the dynamic area becomes smaller. Monotonically increasing.
[0053]
The scanning line concentration unit 33 has a function as a scanning line concentration unit. The scanning line concentration unit 33 concentrates a predetermined number n of scanning lines corresponding to each other on the dynamic region and extracts them at substantially equal intervals. As the person 12 approaches the inner edge 43 from the outer edge 42, the interval between the scanning lines to be extracted is gradually increased, and as the person 12 moves away from the inner edge 43 to the outer edge 42, the interval between the scanning lines to be extracted is gradually reduced. Go. In this way, for example, the third scanning line m3 from the top always scans near the shoulder, and the third scanning line m18 from the bottom always scans near the knee. Can be monitored with accuracy. Further, the remote monitoring object 12 is not scanned between the scanning lines, and it cannot be detected.
[0054]
The uppermost scanning line and the lowermost scanning line are selected so that the entire person 12 can be accommodated with a margin. For example, when the number of scanning lines of the first imaging device 16 and the second imaging device 18 is 500 and the number of scanning lines corresponding to each other is approximately 500, the image of the person 12 is captured in the outer edge portion 42. Of about 160 scanning lines from the top to about 340, 20 lines are selected at intervals of about 9, and the image of the person 12 approaches the direction of the inner edge 43 side. At the position where the image is spread over the entire imaging screen, 20 lines are selected at intervals of about 25 lines, and the intervals between the scanning lines corresponding to each other are gradually changed. From the position where the image of the person 12 spreads across the imaging screen [further], on the inner edge 43 side, 20 states are maintained at intervals of about 25.
[0055]
Further, for example, when the number of scanning lines of the first imaging device 16 is 500, the number of scanning lines of the second imaging device 18 is 600, and the number of scanning lines corresponding to each other is about 100, the outer edge portion If the image of the person 12 is about 1/3 of the imaging screen at 42, 20 lines of about 40 scanning lines from about 30 to about 70 from the top are selected at intervals of about 2, and the inner edge 43 At the position where the image of the person 12 approaches in the direction of the side and the image of the person 12 spreads out, 20 lines are selected at intervals of about 5 lines, and the intervals between the scanning lines corresponding to each other are gradually changed. On the inner edge 43 side from the position where the image of the person 12 extends over the entire shooting range, 20 states are maintained at intervals of about 5 lines.
[0056]
Further, for example, among the pixels constituting the first dynamic region or the second dynamic region, the lowest pixel is between the highest scanning line and the next higher scanning line. When a predetermined number n of scanning lines are concentrated on the first dynamic area or the second dynamic area so as to be between the lowest scanning line and the next lower scanning line, the highest scanning line is obtained. When the uppermost scanning line scans the lower side of the pixel, or when the lowermost scanning line scans the upper side of the lowermost pixel, a predetermined number n of scanning lines are concentrated. In order to widen the range and when the next upper scanning line scans the upper side of the highest pixel, or the lower upper scanning line scans the lower side of the lowermost pixel. In this case, the interval between the scanning lines is changed again so that the range of the predetermined number n of scanning lines is reduced. By adjusting the lowermost pixel between the upper scanning line and the next upper scanning line so as to be between the lowermost scanning line and the next lower scanning line, a predetermined number n of scanning lines can be obtained. It is permanently concentrated on the monitoring object 12, and the operation of the monitoring object 12 can be grasped with a certain accuracy. For example, the adjustment is performed so that the uppermost scanning line is ½ of the current scanning line interval from the uppermost pixel and the lowermost scanning line is set to the current scanning line interval from the lowermost pixel. The n-2 scanning lines are extracted so that the scanning line interval is substantially uniform between them so as to be 1/2 lower.
[0057]
If the dynamic region is not extracted, the person 12 is considered to be stationary, so the state of the scanning line in the immediately preceding frame is maintained. Further, when a dynamic region is not extracted a predetermined number of times or more, in order to reconfirm the state of the monitoring object 12, a predetermined number n of scanning lines are extended to the entire monitoring space. If the dynamic area is confirmed, a predetermined number n of scanning lines are concentrated on the newly extracted dynamic area.
[0058]
The position calculation unit 26 has a function as distance information calculation means. By performing correlation processing based on the first difference image and the second difference image, the distance between the inner edge portion 43 (predetermined reference line) and the person 12 and the position in the X direction or the Z direction are calculated. The position information of each part of the person 12 acquired in step 1 is temporarily stored in the position information storage unit 39.
[0059]
The position calculation unit 26 has a function of calculating a correlation output value between the first dynamic region extracted from the first difference image and the second dynamic region extracted from the second difference image. is doing. The position calculator 26 calculates a predetermined reference line (inner edge) 43, that is, a distance from the installation position of the imaging device to the person 12 based on the calculated correlation output value.
[0060]
For example, the position calculation unit 26 uses, as a dynamic area, a pixel area where the absolute value of each pixel value is large from the first difference image, that is, a pixel area whose contrast has changed drastically due to the movement of the person 12. To extract. Further, the image information is extracted from the second difference image using a pixel region having a large absolute value of the pixel value as a dynamic region. When the image information of each extracted dynamic region is compared, it can be seen that the two dynamic regions are images in which the positions of the corresponding pixel regions are slightly different from the parallax relationship between the imaging devices 16 and 18. This is because the two imaging devices 16 and 18 are installed at a constant interval w and the same monitoring object 12 is monitored.
[0061]
Subsequently, the position calculation unit 26 performs correlation processing between the first dynamic region and the second dynamic region, shifts one image in units of one pixel, and the position where the correlation peak is maximized (correlation peak position). ) Is calculated, the average correlation output value between the two dynamic regions or the correlation output value at the approximate center of the extracted region is calculated and acquired.
[0062]
Here, the correlation output value is a relative imaging position difference generated by parallax between the first imaging device 16 and the second imaging device 18, and typically, the number of pixels (pixels) is obtained by correlation processing. ) Is the output value. Based on the parallax between the scanning line of the first imaging device 16 and the scanning line of the second imaging device 18 based on the correlation output value, the position calculation unit 26 uses the trigonometry to obtain the imaging device, that is, the predetermined reference line 43. The distance to the person 12 is calculated.
[0063]
Correlation processing means that one of the first difference image and the second difference image, or one of the first dynamic region and the second dynamic region, and the two images substantially coincide. This is a process of calculating by shifting the pixel unit until it is done, and indicating the shifted amount by, for example, the number of pixels. Judgment of whether two difference images match or not is when one difference image is fixed and the other difference image is shifted, or one dynamic region is fixed and the other dynamic region is shifted. Execute based on the strength of the entire signal that overlaps. The coordinates at which the signal peaked are the coincidence points and the correlation peak positions.
[0064]
In addition, the correlation process binarizes the first difference image and the second difference image with an appropriate threshold value, and extracts an edge portion of the monitoring target object 12 so that the moving monitoring target object 12 exists. May be extracted, and correlation processing may be performed only on the extracted region. By doing so, the correlation processing uses binarized image data, so the processing becomes simple.
[0065]
If it is a true correlation output value, the correlation peak value usually jumps and becomes large. However, the acquired correlation output value data may not be the case. Therefore, the maximum correlation peak value and the second largest correlation peak value are compared. If the correlation peak value level difference is equal to or greater than the predetermined threshold R, the correlation output value that maximizes the correlation peak value is acquired. If the correlation peak value is equal to or less than the threshold R, the correlation output value that maximizes the correlation peak value is acquired. Do not do. That is, if the correlation peak value is not large and does not increase, it is regarded as unreliable data. Even when there is a maximum peak value, if the maximum peak value is equal to or less than the threshold value R ′, it is regarded as unreliable data, and the correlation output value with the maximum correlation peak value is not acquired. .
[0066]
Of course, when the first imaging device 16 and the second imaging device 18 are the same model and almost all of the scanning lines corresponding to each other are used, in this embodiment, the extracted scanning lines are used as dynamic regions. Since the images are concentrated on the whole and extracted at equal intervals, an image of a portion corresponding to the head of the person 12 to the toes can be obtained. Then, with respect to the dynamic region extracted from the difference image obtained from each corresponding scanning line, the distance for each part of the person 12 on each scanning line can be obtained by correlation processing. Alternatively, the distance about the person 12 can be obtained by correlation processing for the dynamic region extracted from the difference image obtained from the entire predetermined number n of scanning lines. The average distance may be calculated by collecting the distances of the respective parts.
[0067]
The position information of the person who is the monitoring object 12 in the X direction and the Z direction is the X direction of the first difference image or the second difference image, or the first dynamic region or the second dynamic region. Derived from position and distance information in the Z direction. The position information storage unit 39 has a function as distance information storage means, and temporarily stores distance information, position information in the X direction and Z direction, and changes with time thereof.
[0068]
The determination unit 28 has a function as a determination unit, determines the movement state of the person who is the monitoring object 12 from the time-dependent change of the distance information, and determines the movement state to the output device 36 via the transmission unit 38. Is output. Further, the moving state may be displayed on the image recording / reproducing apparatus 34 via the transmission unit 38. The movement state of the person is information such as forward / backward / left / right movement and stillness. It is determined that the distance is approaching when the distance is short, and is moving away when the distance is long. When combined with time information, the moving speed can also be calculated. Alternatively, the size of the dynamic region may be calculated, and it may be determined that the size of the dynamic region is approaching when it is large, and that the region is moving away when it is small.
[0069]
FIG. 4 illustrates a flowchart of the processing flow of the monitoring apparatus. Here, the left image is an image captured by the first imaging device 16, and the right image is described as an image captured by the second imaging device 18. The process starts from the start step S10. First, the control device 2 captures the left image and the right image with the first imaging device 16 and the second imaging device 18, respectively (step S11). To obtain a stereo image.
[0070]
The control device 2 compares the images recently captured by the first imaging device 16 and the second imaging device 18 in the scanning line selection unit 32 of the scanning line control unit 29, and scans the first imaging device 16. Data of a pair of scanning lines corresponding to each other that scan the same portion of the monitoring object 12 from the scanning line of the second imaging device 18 and the scanning line of the second imaging device 18 is held in advance. The scanning line selection unit 32 determines a predetermined corresponding one of the monitoring objects 12 from the scanning line of the first imaging device 16 and the scanning line of the second imaging device 18 based on the data of the pair of scanning lines corresponding to each other. N scanning lines are extracted (step S12).
[0071]
The control device 2 acquires a left image of the T0 frame obtained by imaging the person 12 in the monitoring space 10 at the first time T0 (step S13). Next, the left image of the T1 frame captured at the second time T1 = T0 + ΔT when the predetermined time ΔT has elapsed is acquired (step S14).
[0072]
Next, the first difference image forming unit 24 extracts a first difference image that is a difference image (left) between the left image of the T0 frame and the left image of the T1 frame (step S15). Next, the dynamic area extraction unit 27 associates the first difference image with either the T0 frame or the T1 frame, and extracts the contour of the person 12 with high contrast from the first difference image. After performing the dynamic region extraction process (step S16), the process proceeds to the subsequent correlation process of step S21.
[0073]
In addition, the control device 2 acquires a right image of the T0 frame obtained by capturing the person 12 in the monitoring space 10 at the first time T0 (step S17). Next, the right image of the T1 frame imaged at the second time T1 = T0 + ΔT when ΔT of a predetermined time has elapsed is acquired (step S18).
[0074]
Next, the second difference image forming unit 25 extracts a second difference image that is a difference image (right) between the right image of the T0 frame and the right image of the T1 frame (step S19). Next, the dynamic region extraction unit 27 associates the second difference image with either the T0 frame or the T1 frame, and extracts the contour of the person 12 with high contrast from the second difference image. After performing the second dynamic region extraction process (step S20), the process returns to step S12.
[0075]
At this time, the scanning line concentration unit 33 of the scanning line control unit 29 concentrates the predetermined number n of scanning lines on the operation area (first and second dynamic areas), The lower scanning line, the scanning line interval, and the like are calculated, and the scanning line selection unit 32 selects a predetermined number n of scanning lines corresponding to each other based on the calculation result. If the predetermined number n of scanning lines are already concentrated in the dynamic area, the process proceeds to the correlation processing in the subsequent step S21.
[0076]
In the present embodiment, the first imaging device and the second imaging device perform imaging at each time (T0, T1 = T0 + ΔT) simultaneously and in parallel. In this case, ΔT may be appropriately determined according to the setting conditions of the counting device, but it is preferable to set the time from when a certain frame N is captured until the next frame N + 1 is captured.
[0077]
Next, the position calculation unit 26 uses the first dynamic region that is the outline of the person 12 with high contrast in the left image of the T0 frame, and uses the first dynamic region that is the outline of the person 12 with high contrast in the right image of the T0 frame. The corresponding point of the dynamic region is searched for and correlation processing is executed (step S21). Distance information corresponding to the distance from the inner edge 43 as a predetermined reference line to the person 12 is calculated from the corresponding point information obtained by this correlation processing (step S22). For example, the distance to the person 12 is calculated based on the number of pixels that match the left image and the right image.
[0078]
Subsequently, the determination unit 28 compares the spatio-temporal image created in the immediately preceding frame with the spatio-temporal image of the current frame, and the person 12 or part thereof present in the spatio-temporal image of the current frame is the monitoring space 10. Is determined based on distance information and position information in the X direction (step S23). When the distance is shortened, it is determined that the person 12 or the part thereof is approaching the wall surface 4, that is, the predetermined reference line 43, and the determination result is output to the output device 36 via the transmission unit 38. 12 is present in the monitoring space 10, the video signal of the person 12 imaged by the first imaging device 16 or the second imaging device 18 is sequentially recorded in the image recording / reproducing device 34 via the transmission unit 38. A measure such as continuing (recording) is performed (step S24). When the distance becomes long, it is determined that the person 12 or a part thereof is away from the wall surface 4. Further, the movement in the X direction or the Z direction is determined from the change in the position information in the X direction or the Z direction. Note that the determination may be made based on the movement state of the pixel address indicating the outline of the person 12 or its portion. For example, when the contour address of the person 12 widens, it is determined that the person 12 is approaching the wall surface 4, and when the person 12 becomes narrow, it is determined that the person 12 is moving away from the wall surface 4. Thus, the process ends (step S25) or returns to the start.
[0079]
FIG. 5 shows a schematic block diagram for explaining the principle of the distance measuring method in the present embodiment. Here, a method of calculating the distance between the imaging device and the person 12 using the trigonometric method will be described. FIG. 5 illustrates a method for measuring the distance to the left end of the person 12 when the person 12 is observed from the front and the person 12 is imaged from the horizontal direction by the first imaging device 16 and the second imaging device 18. To do.
[0080]
Here, w is the distance (base line length) between the optical axes of the first imaging device 16 and the second imaging device 18, and f is the lens focus when the light receiving lens of each of the imaging devices 16 and 18 is a single lens. The distance, d, is the parallax on the image plane of the first imaging device 16 (relative to the second imaging device 18). The focal length f here is the focal length f of the combination lens when a commonly used combination lens is used. Thereby, the distance a between the left end (left shoulder) of the target person 12 and the monitoring device 1 can be calculated by the following equation. a = w × f / d (1)
[0081]
In this way, the sensor control unit 60 that has acquired the stereo image from the first and second imaging devices 16 and 18 (installed at the inner edge 43 that is a predetermined reference line) has an internal difference image forming unit. 62. A difference image (first difference image) of the first image pickup device 16 and a difference image (second difference image) of the second image pickup device 18 are formed by 62, and further, the first and second difference images are used as the first difference image. The first and second dynamic regions are extracted, the two dynamic regions are subjected to correlation processing by the correlation output calculation unit 64, and the correlation processing signal is output to the control device 2, whereby the distance from the imaging device to the person 12 Is calculated.
[0082]
Further, the invasion distance value z of the person 12 is calculated by subtracting the distance from the reference distance h (measured from the outer edge portion 42) between the background measured in advance and the imaging device to the person 12, and the intrusion distance value z is determined as the position. Temporarily stored in the calculation unit 26. Further, it is also possible to configure so that video signals are directly transmitted from the two imaging devices to the control device 2 without using the sensor control unit 60.
[0083]
FIG. 6 shows another schematic block diagram for explaining the principle of the distance measuring method in the present embodiment. As shown in FIG. 6, the distance a ′ between the right end (right shoulder) of the target person 12 and the imaging device is also the parallax d ′ on the image plane of the second imaging device 18 (first imaging device). 16) (where a is changed to a ′ and d is changed to d ′). The sensor control unit 60 forms a difference image by the difference image forming unit 62, extracts dynamic regions from the two difference images, and performs correlation processing on the two dynamic regions by the correlation output calculation unit 64, and outputs a correlation processing signal. Is output to the control device 2 to calculate the distance from the imaging device to the person 12. When the person 12 is facing the front with a symmetrical posture, a ′ is equal to a and d ′ is equal to d.
[0084]
Returning to FIG. 1, the operation of the monitoring apparatus 1 will be further described. The calculation unit 30 can monitor the person 12 based on the penetration distance value z temporarily stored in the position calculation unit 26.
[0085]
In addition, the calculation unit 30 stores the intrusion distance value z between the left end and the right end of the person 12 temporarily stored in the position information storage unit 39 in time series, and based on this, the position information (including distance information) of the person 12 is stored. ), Size, moving direction, posture, etc. are determined. That is, the determination unit 28 compares the reference distance h to the outer edge portion 42 with the intrusion distance value z of the person 12 at a certain time, thereby determining the presence of the person 12, its position information, size, moving direction, and posture. And so on.
[0086]
Further, the position calculation unit 26 determines the final existence of the person 12, the position information, the moving direction, the posture, and the like corresponding to each of a plurality of scanning lines extracted from the scanning lines of the imaging device. To calculate the position information of the person 12. The moving direction of the person 12 includes not only a horizontal change in the position of the person 12 but also a vertical change in which the person 12 stands or sits down, for example.
[0087]
It should be noted that the monitoring device of the present invention is not limited to the above-described embodiment, and various changes can be made without departing from the scope of the present invention.
[0088]
For example, in the present embodiment, the distance from the predetermined reference line 43 to the monitoring object 12 is calculated by performing correlation processing on the first dynamic region and the second dynamic region. 26 obtains pixels corresponding to each of the first dynamic area and the second dynamic area, calculates the positional deviation of the pixels in correspondence, and based on the principle of triangulation from the positional deviation, that is, parallax. Thus, the distance to the monitoring object 12 may be calculated.
[0089]
In this way, the position calculation unit 26 does not need to perform correlation processing at all, and can calculate parallax with a very simple process. This method is also effective when the distance to the person 2 becomes short and the parallax increases.
[0090]
In the present embodiment, the absolute value of the pixel value in the dynamic region is larger than the first threshold, and the pair of pixels at the boundary with the region where there is no signal sandwiching the region where the difference image signal is generated An area is extracted and extracted as an annular pixel area formed by connecting one or more pairs of pixel areas extracted in adjacent scanning lines to each other. It may be extracted as a surrounding planar pixel region. If it does in this way, the boundary of clothes etc. will be extracted. Further, even if a part of the annular pixel region or the planar pixel region is missing, it can be said that the present invention is applicable because correlation processing and distance calculation are possible.
[0091]
In the present embodiment, the correlation output value is calculated from the correlation between the first dynamic region and the second dynamic region, but from the correlation between the first difference image and the second difference image. It may be calculated.
[0092]
【The invention's effect】
As described above, according to the present invention, the monitoring device having a simple configuration that can obtain the position information of the monitoring target moving in the three-dimensional monitoring space using the stereo image at high speed and with high accuracy, particularly the monitoring target. Even when an object is far away, it is possible to provide a monitoring device that can measure with the same accuracy as when it is nearby.
[Brief description of the drawings]
FIG. 1 is a schematic partially broken perspective view of a monitoring device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of an image photographed by the imaging device in the embodiment of the present invention.
FIG. 3 is a diagram showing a state of a scanning line that scans a monitoring object when the monitoring object is near and far from the embodiment of the present invention.
FIG. 4 is a flowchart illustrating the processing flow of the monitoring apparatus according to the embodiment of the present invention.
FIG. 5 is a diagram for explaining the principle of a distance measuring method used in the embodiment of the present invention.
FIG. 6 is another diagram for explaining the principle of the distance measuring method used in the embodiment of the present invention.
[Explanation of symbols]
1 Monitoring device
2 Control device
3 Floor (XY plane)
4 Wall surface (XZ surface)
10 Monitoring space
12 Monitoring object (person)
16 First imaging device
17 Infrared light emitting device
18 Second imaging device
19 I / F section
20-23 Imaging signal storage unit
24 1st difference image formation part
25 Second difference image forming unit
26 Position calculator
27 Dynamic region extraction unit
28 judgment part
29 Scanning line controller
30 Calculation unit
31 Memory unit
32 Scanning line selector
33 Scanning line concentration part
34 Image recording and playback device
35 Input device
36 Output device
37 Input section
38 Transmitter
39 Location information storage unit
40 Direction of movement
42 Outer edge
43 Inner edge
44 Wall surface (YZ surface)
60 Sensor control unit
62 Difference image forming unit
64 Correlation output calculator

Claims (8)

A monitoring device using a stereo image for monitoring a monitoring object moving in a three-dimensional monitoring space;
A first imaging device that is installed toward the monitoring space and images the monitoring object that changes over time;
A second imaging device that is installed at a predetermined distance from the first imaging device and is directed toward the monitoring space and that images the monitoring object that changes over time;
A first difference image that is a difference image between an image captured at a first time by the first imaging device and an image captured at a second time delayed by a predetermined time from the first time. First difference image forming means for sequentially forming;
Second difference image forming means for sequentially forming a second difference image that is a difference image between the image picked up at the first time by the second image pickup device and the image picked up at the second time. When;
From the first difference image and the second difference image, the respective areas occupied by the monitoring object on the first difference image and the second difference image are defined as first dynamic areas. Dynamic area extracting means for extracting as a second dynamic area;
From the scanning lines of the first imaging device and the scanning lines of the second imaging device, scanning lines that scan the same portion of the monitoring target are scanning lines that correspond to each other, and a predetermined number of scanning lines from the scanning lines that correspond to each other. Scanning line extracting means for extracting a pair of scanning lines;
Scanning line concentration means for concentrating and scanning the predetermined number of scanning lines in the first imaging device and the second imaging device in the first dynamic region and the second dynamic region, respectively;
Distance information calculation for calculating the distance from the predetermined reference line of the monitoring object based on the first dynamic area and the second dynamic area formed using the concentrated scanning lines. Comprising means;
A monitoring device using stereo images. Distance information storage means for storing the distance and its change over time;
Determining means for determining a movement state of the monitoring object based on a change with time of the distance;
The monitoring apparatus using the stereo image according to claim 1. The dynamic region extraction unit is configured such that an absolute value of a pixel value is a first threshold value for each of the concentrated scanning lines from each pixel constituting the first difference image or the second difference image. One or more pairs of pixel regions that are extracted by adjacent scanning lines by extracting one or more pairs of pixel regions at the boundary with a region that has a larger difference image signal and sandwiches the region where there is no signal Are extracted as a first dynamic region or a second dynamic region, respectively. ;
The monitoring apparatus using the stereo image according to claim 1 or 2. The dynamic region extraction unit is configured such that an absolute value of a pixel value is a first threshold value for each of the concentrated scanning lines from each pixel constituting the first difference image or the second difference image. An annular pixel region formed by extracting a pair of pixel regions that are larger and outermost and connecting each pair of pixel regions extracted by adjacent scanning lines to each other, or the annular pixel region Extracting planar pixel areas around the pixel area as the first dynamic area or the second dynamic area, respectively;
The monitoring apparatus using the stereo image according to claim 1 or 2. The scanning line concentration means is configured such that, among the pixels constituting the first dynamic region or the second dynamic region, the highest pixel is between the highest scanning line and the next higher scanning line. The predetermined number of scanning lines are concentrated in the first dynamic area or the second dynamic area so that the lowest pixel is between the lowest scanning line and the next lower scanning line. Scan;
The monitoring apparatus using the stereo image according to claim 1. The scanning line concentration means maintains the state of the scanning line in the immediately preceding frame when the first dynamic area or the second dynamic area cannot be extracted, and further, the scanning line concentration means is more than a predetermined number of times, When the first dynamic area or the second dynamic area cannot be extracted, the predetermined number of scanning lines are extended to the entire monitoring space;
The monitoring apparatus using the stereo image according to claim 1. The distance information calculation means calculates a correlation output value related to a parallax between the first imaging device and the second imaging device from a correlation between the first dynamic region and the second dynamic region. Calculating and calculating a distance from the predetermined reference line of the monitored object using trigonometry from the correlation output value;
The monitoring apparatus using the stereo image according to claim 1. The distance information calculation means extracts a correlation output value indicating a maximum correlation peak value and a correlation output value indicating a second correlation peak value from the correlation output values, and extracts the maximum correlation peak value and the second correlation peak value. A distance from the predetermined reference line is calculated from the correlation output value indicating the maximum correlation peak value only when the difference from the correlation peak value is greater than the second threshold value;
The monitoring apparatus using the stereo image according to claim 7.

Images