JP2003052034A - Monitoring system using stereoscopic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a simple monitoring system capable of accurately monitoring a target region, and also performing high speed processing. SOLUTION: This monitoring system 1 is provided with a first image pickup element 11 having the first number of scanning lines 13 for imaging a region to be monitored, a second image pickup element 12 arranged in parallel with the first image pickup element 11 in the scanning direction having the second number of scanning lines 14 for imaging the region to be monitored, a scanning line extracting means 22 for extracting the third number of mutually corresponding scanning lines which is smaller than either the first number or the second number from the first image pickup element 11 and the second image pickup element 12, a distance information acquiring means 23 for acquiring distance information to an object 2 based on a picture formed on the scanning line extracted by the scanning line extracting means 22, and a detection processing means 27 for monitoring the object 2 by using the distance acquired by the distance information acquiring means 23.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surveillance system, and more particularly to a surveillance system for monitoring changes in height or posture of an object or a person in a target area by using a CCD camera, a CMOS sensor or the like.

[0002]

2. Description of the Related Art Conventionally, as a monitoring system for knowing an abnormality in a hospital room, a toilet, etc., two cameras have conventionally taken images of a monitoring target area to monitor the state of objects and persons in the target area. There was a device to do.

[0003]

However, according to the conventional apparatus as described above, the target area is monitored by performing processing such as detection of corresponding points of stereo images picked up by the two cameras. Therefore, not only the amount of calculation increases and complicated processing is required, but also a complicated device is required.

Therefore, an object of the present invention is to provide a monitoring system that not only can accurately monitor the target area but also can perform high-speed processing and is simple.

[0005]

In order to achieve the above object, a monitoring system 1 according to the first aspect of the present invention, as shown in FIG. 1, FIG. 2 and FIG. First imaging device 1 having one scanning line 13
1; a second number of scanning lines 1 that are provided in parallel with the first image sensor 11 in the scanning line direction and image the monitoring target region.
A second image pickup device 12 having 4; a first image pickup device 11
From the two image pickup devices, the image pickup device and the second image pickup device 12, a third image pickup device which is smaller in number than the first image pickup device and the second image pickup device.
Scanning line extracting means 22 for extracting the scanning lines 15 and 16 corresponding to each other; and the distance information to the object 2 based on the image formed on the scanning lines extracted by the scanning line extracting means 22. Distance information acquisition means 23 to acquire;
The detection processing unit 27 that monitors the object 2 using the distance information acquired by the distance information acquisition unit 23.

With this configuration, the first image pickup device 1
1, the second image sensor 12, the scanning line extracting means 22,
Since the distance information acquisition unit 23 and the detection processing unit 27 are provided, two of the first image pickup device 11 and the second image pickup device 12 are provided.
An image formed on the extracted scanning lines by extracting the scanning lines 15 and 16 corresponding to each other with a third number smaller than either the first number or the second number from one image sensor. Based on the above, by acquiring the distance information to the target object 2 and monitoring the target object 2 using the acquired distance information, not only accurate monitoring of the target area can be performed but also high-speed processing can be performed. It is possible to provide a monitoring system that is possible and simple.

The monitoring system 1 also comprises means 27 for calculating the position information of the object 2 from the distance information acquired by the distance information acquisition means 23, and the detection processing means 27 comprises:
Based on the distance information to the object 2 and the position information,
It may be configured to monitor the object 2.

Further, as described in claim 2, in the surveillance system 1 described in claim 1, the distance information acquisition means 23 includes:
Difference image forming means for forming a difference image from the image information of the image formed on each of the extracted scanning lines 15 and 16 of the images captured at different times for each of the two image pickup devices 11 and 12. 24; and area extracting means 2 for extracting the maximum area among the areas in which the absolute value of each pixel value forming the difference image is divided by pixels larger than a predetermined threshold value.
5 and 5 are preferably provided.

Further, as described in claim 3, in the monitoring system 1 described in claim 2, the distance information acquisition means 23 is
The correlation output calculating means 26 is provided for calculating the correlation output value of the maximum areas extracted by the area extracting means 25 from the difference image corresponding to each image picked up by the two image pickup elements 11 and 12; Object 2 based on output value
It is characterized by acquiring distance information up to.

Further, as described in claim 4, in the monitoring system 1 described in claim 2, the distance information acquisition means 23 is
In the maximum area extracted by the area extracting means 25 from the difference image corresponding to each image captured by the two image pickup devices 11 and 12, the correspondence of the pixels larger than the threshold value is obtained, and the correspondence is obtained. It is characterized in that the distance information to the object 2 is acquired based on the displacement of the pixel.

According to a fifth aspect, in the surveillance system 1 according to any one of the first to fourth aspects, the images formed on the scanning lines 15 and 16 corresponding to each other are synchronized. It is characterized by being extracted.

Further, as described in claim 6, in the monitoring system 1 according to any one of claims 1 to 5, the scanning line extracting means 22 extracts at least two scanning lines. Is characterized by.

Further, as described in claim 7, in the surveillance system 1 according to any one of claims 1 to 6, the scanning line for scanning the object 2 in the first image pickup device 11 is further added. 13 and means 28 for calculating a correlation value with respect to the scanning line 14 for scanning the object 2 in the second image pickup device 12; the first so that the calculated correlation value is maximized. It is characterized in that the positional relationship between the image sensor 11 and the second image sensor 12 is determined.

Further, as described in claim 8, in the surveillance system 1 according to any one of claims 1 to 6, a scanning line for scanning the object 2 in the first image pickup device 11 is further added. 13 and means 28 for calculating a correlation value with respect to the scanning line 14 for scanning the object 2 in the second image pickup device 12; the first image pickup so that the calculated correlation value is maximized. Corresponding scanning line selection means 29 that determines the positional correspondence between the scanning line 13 of the element 11 and the scanning line 14 of the second image sensor 12 may be provided.

Further, as described in claim 9, in the surveillance system 1 according to any one of claims 1 to 8, the image pickup elements 11 and 12 are CCD cameras or CMs.
It is an OS sensor.

In order to achieve the above-mentioned object, a monitoring system 1 according to a tenth aspect of the present invention, for example, as shown in FIG.
2 and the image pickup elements 11 and 12 each composed of a plurality of one-dimensional light receiving element arrays 17 and 18 corresponding to each other between the two image pickup elements 11 and 12, and the one-dimensional light receiving element array 1
Distance information acquisition means 23 for acquiring distance information to the object 2 based on the images formed on 7, 18; and monitoring the object 2 using the distance information acquired by the distance information acquisition means 23. Detection processing means 27 for

With this configuration, the two image pickup devices 1
Image pickup elements 11 and 12 each composed of a plurality of one-dimensional light-receiving element arrays 17 and 18 corresponding to each other between 1 and 12;
Since the distance information acquisition unit 23 and the detection processing unit 27 are provided, the distance information to the object 2 is acquired based on the images formed on the one-dimensional light receiving element arrays 17 and 18, and the acquired distance information is acquired. By monitoring the target object 2 by using, it is possible to provide not only accurate monitoring of the target region, but also a high-speed processing and simple monitoring system.

In order to achieve the above object, a monitoring method according to an eleventh aspect of the present invention is a monitoring method according to an eleventh aspect, in which two image pickup elements having a first number of scanning lines and a second image pickup element having a second number of scanning lines are provided.
And an image pickup step of picking up an image of a monitoring target area by two image pickup elements; and a third number smaller than either the first number or the second number from the two image pickup elements corresponding to each other. A scanning line extracting step of extracting a scanning line; a distance information acquiring step of acquiring distance information to the object based on an image formed on the scanning line extracted by the scanning line extracting step; And a detection processing step of monitoring the object using the distance information acquired in the information acquisition step.

According to this structure, the image pickup step, the scanning line extraction step, the distance information acquisition step, and the detection processing step are included. Therefore, the first image pickup device and the second image pickup device can be selected from the two image pickup devices. A third number of scanning lines corresponding to each other, which is smaller than any one of the above, is extracted, and distance information to the object is acquired based on the image formed on the scanning lines extracted by the scanning line extracting step. By monitoring the object using the acquired distance information, it is possible to provide not only an accurate monitoring of the target area but also a monitoring method capable of high-speed processing.

Further, as described in claim 12, claim 1
In the monitoring method according to item 1, among the two image pickup devices,
The first scan line for scanning the object in the first image sensor and the scan line scan for the target in the second image sensor corresponding to the scan line are on the same plane. The positional relationship between the image pickup device and the second image pickup device may be determined.

Further, as described in claim 13, claim 1
In the monitoring method according to claim 1 or 12, among the two image pickup elements, a scanning line that scans the object in the first image pickup element and a scanning line that scans the object in the second image pickup element. On the other hand, a step of calculating a correlation value is provided; the positional relationship between the first image pickup device and the second image pickup device may be determined so that the calculated correlation value is maximized.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. In each drawing, the same or corresponding members are designated by the same reference numerals or similar reference numerals, and redundant description will be omitted.

FIG. 1 is a schematic perspective view of a monitoring system 1 according to an embodiment of the present invention. In the figure, the object 2 is present on the floor surface 3. An orthogonal coordinate system XYZ is set so that the XY axes are placed on the floor surface 3. A wall 4 is formed perpendicular to the floor surface 3, that is, on the YZ plane. Object 2
Is a person in the present embodiment.

On the other hand, on the wall 4 in the drawing, a first number having a first number of scanning lines 13 (see FIG. 3) for imaging the monitored region is provided.
The image pickup device 11 and the second image pickup device 12 having the second number of scanning lines 14 (see FIG. 3) are installed.
The first image sensor 11 and the second image sensor 12 are installed in parallel in the scanning line direction. The first image sensor 11 and the second image sensor 12 are typically CCD cameras, but CMOS sensors may be used. In addition, in the present embodiment, the first image sensor 11 and the second image sensor 12 are
Although described as a separate body, they may be configured as one body. Further, the first number and the second number are typically the same number, and more typically about 500. Further, in the present embodiment, the first image sensor 11 and the second image sensor 12 are used.
Is installed on the wall, but if there is a ceiling, it may be installed on the wall, and the installation location may be appropriately determined according to the purpose and specifications of the monitoring system.

An example of the configuration of the monitoring system 1 will be described with reference to FIG. The monitoring system 1 is configured to include a first CCD camera 11 as a first image sensor, a second CCD camera 12 as a second image sensor, and an arithmetic unit 20. The first CCD camera 11 and the second CCD camera 12 are connected to the arithmetic unit 20.
The arithmetic unit 20 includes a first CCD camera 11 and a second CC
The image captured by the D camera 12 is configured to be acquired. In addition, the captured images may be configured to be acquired in time series. Arithmetic unit 20
Is typically a personal computer or a microcomputer.

The arithmetic unit 20 has a control unit 21 and controls the entire monitoring system 1. An interface 33 is connected to the control unit 21, and the first C
The CD camera 11 and the second CCD camera 12 are connected to and controlled by the control unit 21 via the interface 33.

A storage unit 31 is connected to the control unit 21, and a reference distance storage unit 32 for storing a reference distance to be compared with a measurement distance measured by a third number of scanning lines 15 and 16 described later. Is provided. Further, the storage unit 31 can store data such as calculated information. The reference distance may be a distance at a time point in the past of the monitoring time point, and is typically a distance in the background in a state where the person 2 does not exist in the monitoring target area, but is not limited thereto, and for example, the distance is periodically The distance may be one frame before when is detected.

The control unit 21 is also connected to an input device 35 for inputting information for operating the monitoring system 1 and an output device 36 for outputting a result processed by the monitoring system 1. The input device 35 is, for example, a touch panel, a keyboard or a mouse, and the output device 36 is, for example, a display, a printer or an alarm device. Although the input device 35 and the output device 36 are illustrated as being externally attached to the arithmetic device 20 in the present drawing, they may be incorporated therein.

In the control unit 21, the first CCD camera 1
Two CCD cameras 1 including a first CCD camera 12 and a second CCD camera 12.
1 to 12, a third number of corresponding scan lines 15, 16 that is smaller than either the first number or the second number.
A scanning line extracting unit 22 is provided as a scanning line extracting unit for extracting (see FIG. 3). At this time, the images formed on the scanning lines corresponding to each other are extracted in synchronization.
In addition, the scanning line extracting unit 22 is configured to include at least two scanning lines 1
5 and 16 are extracted. Further, the scanning line extraction unit 22 may extract the scanning lines 15 and 16 in time series. Further, in order to extract the scanning lines 15 and 16 from the first CCD camera 11 and the second CCD camera 12, if a synchronization signal from the outside is used, the corresponding scanning lines 1 and 16 are extracted.
A circuit for extracting and processing 5 and 16 can be simplified, and software processing is also facilitated.

As shown in the schematic view of FIG. 3, the images formed on the scanning lines 13 and 14 are formed on one or a plurality of scanning lines. Further, the third number is such that the privacy of the person 2 can be protected or the processing amount does not increase significantly, and may be about 2 to 20, but more preferably the first number and the second number are 500. When the number is about 20, the number is about 20 at equal intervals.

Further, as shown in the schematic diagram of FIG.
The CD cameras 11 and 12 include a plurality of one-dimensional light receiving element arrays 17 in which the two image pickup elements 11 and 12 are associated with each other.
Two CCD cameras 11 ', 12' composed of 18
May be set. In this case, the one-dimensional light receiving element arrays 17 and 18 correspond to the scanning lines 15 and 16, respectively. The number of the one-dimensional light receiving element arrays 17 and 18 is the same as the above-mentioned third number, that is, about 20. The one-dimensional light receiving element array is typically a line CCD or CM
It is an OS sensor. In this way, the two CCD cameras 11 'and 12' are the same as the CCD cameras 11 and 12 described above.
Since the above-mentioned scanning line extraction unit 22 is not necessary as compared with the case of using, it is possible to adopt a simpler configuration. Further, the size can be reduced as compared with the case where the monitoring system 1 is configured by using the same number of distance sensors as the third number.

Further, in the control unit 21, the scanning line extraction unit 22
A distance calculator 23 is provided as a distance information acquisition unit that acquires distance information to the person 2 based on the images formed on the scanning lines 15 and 16 extracted by. In the following description of the present embodiment, the distance information acquisition unit is a distance calculation unit that calculates a distance.

The distance calculation unit 23 includes the first CCD camera 1
A difference image from the image information of the images formed on the scanning lines 15 and 16 extracted by the scanning line extracting unit 22 of the images captured at different times for each of the first and second CCD cameras 12. And a difference image forming unit 24 as a difference image forming unit for forming The image information is typically a pixel value of a pixel forming an image. The two images for forming the difference image are acquired with a time shift, and the shift time is a time such that the movement amount of the person 2 does not become too large and can be regarded as substantially the same position, for example, 0. It may be about 1 second. Alternatively, the period is 1 to 10 (1/30 to 1/3) of the television period. When such a difference image is taken, the background is removed and an image of the person 2 who is moving can be extracted.

Further, the distance calculation unit 23 serves as a region extracting means for extracting the maximum region among the regions divided by pixels whose absolute values of the pixel values forming the difference image are larger than a predetermined threshold value. The area extraction unit 25 of FIG.

Since the backgrounds of the images formed on the scanning lines 15 and 16 do not move, the pixel values at the boundary of the moving person 2 change abruptly. Therefore,
The distance calculation unit 23 determines that the largest area among the areas in which the absolute value of each pixel value forming the difference image formed by the difference image forming unit 24 is divided by pixels larger than a predetermined threshold value. It can be regarded as the boundary of 2. Here, the reason why the maximum area is set is that the pixel value may change in areas other than the boundary area due to movement of patterns such as clothes.

Further, the distance calculation unit 23 correlates the maximum areas extracted by the area extraction unit 25 from the difference image corresponding to each image captured by the first CCD camera 11 and the second CCD camera 12. A correlation output calculation unit 26 is provided as a correlation output calculation unit that calculates an output value. The distance calculation unit 23 calculates the distance to the person 2 based on the correlation output value.

As shown in the schematic diagram of FIG.
3 is the image information of the maximum area extracted by the area extracting unit 25 from the difference image formed from the scanning lines 15, for example, and is extracted as in the state of (a). The end points of the curves shown in the figure are points where the change in contrast was drastic due to the movement. Similarly to the scan line 15, the scan line 1
The image information of the maximum area of the difference image formed from 6 is
It is assumed that it is extracted as in the state of (b). here,
The slightly different curves in (a) and (b) are:
Since the same object is viewed by the two different CCD cameras 11 and 12, a slightly different area image due to the parallax relationship,
This is because they were extracted respectively.

As described above, the distance calculation unit 23 (a)
(B) By performing correlation processing on each curve and measuring the position where the correlation peak becomes maximum, the average correlation output value between the two lines or the correlation at the center of the extracted region is obtained. Output value can be obtained.

Here, the correlation output value is a relative image forming position difference caused by the parallax between the first CCD camera 11 and the second CCD camera 12, and is typically pixel by the correlation processing. It is a value that is output as a number. Distance calculator 23
Is the scanning line 15 and the scanning line 16 according to this correlation output value.
The distance is calculated by the trigonometric method from the parallax with. A method of calculating the distance from the parallax will be described later with reference to FIG. Further, the correlation process is a process of shifting either one of the images obtained from the scanning line 15 and the scanning line 16 until the two images are substantially coincident with each other, and calculating the shifted amount, for example, the number of pixels. . The determination of matching is made based on the strength of the entire signal. The point where the signal peaks is the coincidence point, that is, the correlation peak position.

Correlation processing is performed by the first CCD camera 1
The difference image corresponding to each of the images captured by the first and second CCD cameras 12 is binarized with an appropriate value, and the edge portion thereof is extracted to extract a moving area portion,
The correlation process may be performed only on this extraction region.
In this case, the correlation process uses the binarized difference image, which facilitates the process.

In addition, the distance calculation unit 23 selects the maximum area extracted by the area extraction unit 25 from the difference image corresponding to each image captured by the first CCD camera 11 and the second CCD camera 12. It is also possible to find the correspondence of pixels larger than the respective thresholds and calculate the distance to the object based on the positional deviation of the corresponding pixels. In this case, the distance calculation unit 23 does not calculate the correlation output value of the maximum areas extracted by the area extraction unit 25 from the difference image as described above, and the distance output value is larger than the threshold value in each of the maximum areas. Corresponding pixels are obtained, and the positional deviation of the corresponding pixels is calculated. The distance calculator 23 calculates the distance by trigonometry from the positional shift of the corresponding pixels, that is, the parallax. A method of calculating the distance from the parallax will be described later with reference to FIG. Accordingly, the distance calculation unit 23 can number the pixels exceeding the threshold value from the same direction, and obtain the parallax from the positional shift between the pixels having the corresponding numbers.

In this way, the distance calculation unit 23 does not need to perform the correlation process, and can calculate the parallax with a very simple process. Further, the distance calculation unit 23 does not number the above-mentioned end points (see FIG. 5) when there are many pixels that exceed the threshold due to a change in contrast due to a pattern or the like in the extracted region. As a result, it is possible to reduce a phenomenon in which a pixel having a corresponding number is displaced from a pixel that should be originally associated due to the influence of parallax, and thus an error occurs in the value of the positional displacement. This is also effective when the parallax becomes large due to the short distance to the person 2.

Now, with reference to the schematic diagram of FIG. 6, a method of calculating the distance of the person 2 using the trigonometry will be described by taking as an example the case of measuring the distance of the person 2 from above in the vertical direction. Here, w is the distance between the optical axes of the first CCD camera 11 and the second CCD camera 12 (base line length), f is the focal length of the lens when the light receiving lens of the CCD camera is a single lens, and d Is the parallax on the image plane of the CCD camera. The focal length here is the focal length of the combined lens, when a commonly used combined lens is used. Thereby, the distance A of the target person 2 can be calculated by the following equation. A = w × f / d (1) In this way, the distance calculation unit 23 causes the scanning line extraction unit 22 to operate.
The distance to the person 2 can be calculated based on the images formed on the scanning lines 15 and 16 extracted by.

Returning to FIG. 2, the monitoring system 1 will be further described. The control unit 21 includes a detection processing unit 27 as a detection processing unit that monitors the person 2 based on the distance calculated by the distance calculation unit 23.

In addition, the detection processing section 27 calculates the distance calculated by the distance calculating section 23 of each of the scanning lines 15 and 16 corresponding to the third number and the scanning lines 15 and 16 of the respective third number. May be configured to be compared with a reference distance corresponding to. In this case, the comparison result by the detection processing unit 27 is typically acquired in time series, and the position information, movement, posture, and the like of the person 2 can be determined based on it. That is, by comparing the reference distance with the distance at a certain time point, the existence of the person 2, its position information, movement,
It is configured so that the posture and the like can be judged. Furthermore, the final determination of the presence, position information, movement, posture, etc. of the person 2 is made comprehensively from the determination results corresponding to the respective third scanning lines 15 and 16. The movement of the person 2 is not limited to the change of the position of the person 2
It also includes changes such as standing and sitting.

Here, an example of a method of calculating the height and position which are the position information of the person 2 will be described with reference to the schematic side view of FIG. It is assumed that the two CCD cameras 11 and 12 installed on the wall 4 having a height H from the floor 3 are monitoring a monitoring target area at a distance L (X-axis direction) from the wall 4. Two CCD cameras 11 and 1 of the person 2 existing in this monitoring target area
If the distance from 2 is measured as A, then person 2 to C
The height H1 to the CD cameras 11 and 12 and the distance L1 from the wall 4 can be calculated by the following equation. H1 = AH / (H ² + L ² ) ^1/2 (2) L1 = AL / (H ² + L ² ) ^1/2 (3) Direction orthogonal to X-axis and Z-axis (Y-axis) Direction (see Figure 1))
Position is calculated height H1, distance L1 and two CCDs
It can be calculated by the distance measurement direction of the cameras 11 and 12. Further, the position can be roughly grasped also from the position of the monitoring target area corresponding to each of the third number of scanning lines 15 and 16. In this way, the position information of the person 2 existing in the monitoring target area can be calculated. Further, for example, the floor surface 3
If the two CCD cameras 11 and 12 are arranged in the direction perpendicular to, the position where the cameras are attached becomes the position,
By setting the distance to be high, the position information can be easily acquired.

Further, the detection processing section 27 further compares the position information of the person 2 calculated immediately before with the same or different scanning lines 15 and 16, for example, if there is the previous position information, with the position information. Then, the moving distance and the average moving speed, which are the moving information of the person 2, are calculated. For example, the moving distance is the difference between the H1 and L1 data of the latest position information and the immediately preceding position information, and the average moving speed is the value obtained by dividing this data difference by the distance measurement interval time. Further, the position information and the movement information of the person 2 thus obtained are stored in the storage unit 3
It is preferable to store the reference distance in the reference distance storage unit 32 in 1 as the reference distance.

In this way, the detection processing unit 27 can determine the presence, posture, position and movement state of the person 2 from the calculated position information and movement information. For example,
If the person 2 does not move and the height becomes low, it can be determined that the person 2 is sitting down. If the person 2 exists at a low height and there is no movement, the person 2 becomes You can judge that you are in a state of falling. In this way, the monitoring system 1 can monitor the entry, the presence and the presence state of the person 2 in the monitoring target area.

Since the CCD camera can easily have a wide angle of view, by using a CCD camera having a wide angle of view, the lateral monitoring target area can be made sufficiently large.
When such a monitoring system 1 is installed in a closed space such as a toilet, the first CCD camera 11 and the second C camera
If the angle of view of the CD camera 12 is set to be the width of the monitoring target area in the closed space, there are advantages as described below.

An example in which the monitoring system 1 is installed in the toilet will be described with reference to the schematic diagram of FIG. 8A is a side view of the toilet, FIG. 8B is a front view of the toilet seen from the side of two CCD cameras, and FIG. 8C is a scanning line 15 or 16.
The image above. When the monitoring system 1 is installed to monitor a toilet which is a closed space as shown in the figure, the angle of view of the first CCD camera 11 and the second CCD camera 12 is set to be the width of the toilet.

Two CCD cameras 1 set in this way
1 and 12 are installed on the wall 4'with a slight inclination with respect to the horizontal plane. At this time, the scanning line extraction unit 22 extracts a third number of scanning lines 15 and 16 such that the image of the standing person 2 from the head to the floor surface 3 ′ is distributed at appropriate intervals. To do so. Also, the floor surface 3'when the person 2 is absent
It is advisable to set the distance to the wall 4'or the wall as a reference distance.

Thus, the two CCD cameras 11, 1
By setting 2, the distance to the person 2 is measured, and at the same time, which scanning line 15, 16 detects the person 2,
The presence of the person 2 and the height from the floor, that is, the posture thereof can be determined. In addition, by extracting the difference image and extracting the image of the person 2 on the captured image, the extracted person 2
The position in the toilet width direction (Y-axis direction) can also be obtained from the position of the image of the image.

Here, the installation of the two CCD cameras 11 and 12 will be described with reference to FIG. First, the monitoring system 1 includes the scanning line 13 of the first CCD camera 11 and the scanning line 1 of the second CCD camera 12 corresponding to the scanning line 13.
So that 4 and 4 are approximately on the same plane.
The positional relationship between the D camera 11 and the second CCD camera 12 is determined.

In addition, the monitoring system 1 includes an arithmetic unit 20.
In the control unit 21 of the person 2 in the first CCD camera 11
A scanning line 13 that scans the scanning line 14 and a scanning line 14 that scans the person 2 in the second CCD camera 12, a correlation calculation unit 28 that calculates a correlation value, and the calculated correlation value is maximized. As described above, a corresponding scanning line selection unit 29 as a corresponding scanning line selection unit that defines the positional correspondence between the scanning line 13 of the first CCD camera 11 and the scanning line 14 of the second CCD camera 12 is provided. To

Here, as shown in the figure, it is assumed that the person 2 is imaged on the scanning lines 13 and 14, respectively. In this case, since the two CCD cameras 11 and 12 are looking at the person 2 from a slightly different direction, almost unchanged images are formed on both scanning lines. Here, it is assumed that the correlation calculator 28 selects the fourth line of the scanning lines 13, for example. In this case, first, the correlation calculation unit 28 determines that the scan line 13
The above-described correlation processing between the line signal of the first line and the line signal of the first line of the scanning line 14 is performed. Correlation calculation unit 28
Performs the correlation processing with the line signal of the fourth line of the scanning line 13 in order from the first line of the scanning line 14 to the final line. In the correlation processing here, a correlation value that is the strength of the entire signal is checked. In addition, the correlation calculation unit 28 uses the scan line 13
The correlation processing may be performed until the correlation value between the line signal of the line and the line signal of the nth line of the scanning line 14 becomes maximum. When the number of scanning lines 14 (second number) is 500, n is a natural number of 1 to 500.

The corresponding scanning line selection unit 29 determines that the lines having the maximum correlation value correspond to each other, and the scanning line 13 of the first CCD camera 11 and the second CC
The positional correspondence with the scanning line 14 of the D camera 12 is determined. For example, the fourth line of the scanning line 13 and the seventh line of the scanning line 14
When it is determined that the scanning lines are the corresponding scanning lines, the positional correspondence relationship is as it is, the fifth line of the scanning line 13 and the eighth line of the scanning line 14, and the 100th line of the scanning line 13 and the scanning line. It is defined as the 103rd line of 14.

In the monitoring system 1, the scanning line 13 of the first CCD camera 11 and the scanning line 14 of the second CCD camera 12 are the same line, for example, the fourth line of the scanning line 13 and the fourth line of the scanning line 14. The positional relationship may be determined so that the correlation value calculated by the correlation calculation unit 28 with the eye becomes maximum. By doing so, the same line of the first image pickup device and the second image pickup device are installed so that scanning lines on substantially the same plane correspond to each other, and the positional relationship is determined so that the correlation value becomes maximum. The two CCD cameras 11 and 12 can be installed efficiently and highly accurately.

In addition, the monitoring system 1 determines the positional relationship between the two CCD cameras 11 and 12 or the positional correspondence relationship between the scanning lines 13 and 14 and then determines the baseline length w of the two CCD cameras 11 and 12. Even if is deviated from the set value, it is possible to correct the deviation in the scanning line direction by measuring the predetermined distance A1. In this case, assuming that the positions of the optical axis of the first CCD camera 11 and the optical axis of the second CCD camera 12 are displaced by S in the scanning line direction, the above equation (1) is A = w × By setting f / (d + S) ... (4), the deviation in the scanning line direction can be corrected.

In this way, the monitoring system 1 can not only accurately measure the distance to the person 2, but even if the CCD camera slightly moves due to a temperature change or a temporal change, for example, the pattern of the wall. Distance A1 determined in advance using
Accuracy can be maintained by measuring In addition, this correction may be performed at regular intervals.

In the above, the two CCD cameras 11 and 12
In the above description, the scanning lines are installed so that they correspond to each other, or the corresponding scanning lines are selected.
The CD cameras 11 and 12 may be adjusted so that the scanning lines correspond to each other, and fixed. In this way, the two CCD cameras 11 and 12 can be handled in the same manner as a distance sensor, and therefore the installation of the two CCD cameras 11 and 12 is very simple.

As described above, in the monitoring system 1, the size of the person 2 entering the area to be monitored and the state of the person 2 (at which position, standing, sitting,
A simple device can follow a series of movements such as whether the person 2 is lying down), whether the person 2 is moving, or whether the person 2 has left. In this case, since the difference from the reference distance is acquired, it can be used to judge the state even if the distance is not relatively accurate. In addition, the monitoring system 1
Since the difference image is used, for example, when the person 2 does not move, the area cannot be specified. However, since the state of the person 2 can be comprehensively determined from a series of movements, the person 2 remains stationary. The presence of person 2 who is moving or moving slowly can also be monitored. This is effective, for example, when monitoring a person 2 who is not moving, such as a person 2 sitting in a toilet or a person 2 slowly bathing in a bath.

In the closed space (toilet, bath, elevator, office), the area to be monitored is surrounded by walls and the like. Therefore, when the person 2 is absent, the distance to the floor 3 or the wall 4 is set. The state of the person 2 can be determined by setting it as a reference distance and following the change from that state.

According to the present embodiment as described above, it is possible to judge the state of the person 2 and accurately and very easily monitor whether or not the person 2 has fallen down or an illegal intruder exists. You can Moreover, since the entire frame is not acquired from the CCD camera, complicated and large-capacity image processing is not required, and a very inexpensive device is constructed with a relatively simple digital circuit or analog circuit, or a combination of these. can do. Further, the scanning lines to be extracted are so effective as to protect the privacy of the person 2, and therefore are very effective in monitoring the condition in a toilet, bath or the like.

[0064]

As described above, according to the present invention, a first image pickup device having a first number of scanning lines for picking up an image of a region to be monitored, and the first image pickup device are arranged in parallel in the scanning line direction. From the two image pickup elements provided, a second image pickup element having a second number of scanning lines for picking up the image of the monitoring target area, and the first image pickup element and the second image pickup element, the first image pickup element Scanning line extraction means for extracting scanning lines corresponding to a third number which is smaller than either the number of lines or the second number,
Based on the image formed on the scanning line extracted by the scanning line extracting means, using distance information acquiring means for acquiring distance information to the object, and distance information acquired by the distance information acquiring means Since it includes the detection processing means for monitoring the target object, it is possible to provide a monitoring system that not only can accurately monitor the target area but also can perform high-speed processing and is simple.

[Brief description of drawings]

FIG. 1 is a schematic perspective view showing an outline of a monitoring system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration example of a monitoring system used in the embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating a scanning line of a first CCD camera and a scanning line of a second CCD camera used in an embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating a plurality of one-dimensional light receiving element arrays used in the embodiment of the present invention.

FIG. 5 is a diagram illustrating a correlation process according to the embodiment of the present invention.

FIG. 6 is a schematic diagram illustrating a method of calculating the distance to a monitored object from the parallax of two CCD cameras according to the embodiment of the present invention.

FIG. 7 is a schematic side view illustrating a method of calculating position information of a monitoring target from the distance of the monitoring target in the embodiment of the present invention.

FIG. 8 is a schematic side view (a), a schematic front view (b), and an image on a scanning line (c) for explaining a case where the monitoring system according to the embodiment of the present invention is installed in a toilet. .

FIG. 9 is a schematic diagram illustrating installation of the monitoring system according to the embodiment of the present invention.

[Explanation of symbols]

1 monitoring system 2 people 3 floor 4 walls 11 First image sensor 12 Second image sensor 13 Scan line of first image sensor 14 Scan line of second image sensor 15 Scan lines extracted from the first image sensor 16 Scan lines extracted from the second image sensor 17 One-dimensional light-receiving element array of first image sensor 18 One-dimensional light-receiving element array of second image sensor 20 arithmetic unit 21 Control unit 22 Scan line extraction unit 23 Distance calculator 24 Difference image forming unit 25 Area Extraction Unit 26 Correlation Output Calculation Unit 27 Detection processing unit 28 Correlation calculator 29 Compatible scanning line selector 31 storage 32 Reference distance storage 33 Interface 35 input device 36 Output device

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kazuhiro Ajimura             Sumitomo Osaka Center 6-28 Rokubancho, Chiyoda-ku, Tokyo             Mento Co., Ltd. (72) Inventor Kei Kato             Sumitomo Osaka Center 6-28 Rokubancho, Chiyoda-ku, Tokyo             Mento Co., Ltd. (72) Inventor Masato Nakajima             Kei, 3-14-1, Hiyoshi, Kohoku Ward, Yokohama City, Kanagawa Prefecture             Faculty of Science and Technology, University of Tokyo F-term (reference) 5C054 FC01 FC12 FC13 FC14 FC15                       FD00 HA19                 5C087 AA02 AA03 AA04 AA09 AA24                       AA25 DD03 EE02 EE05 EE08                       EE15 GG02 GG19 GG36

Claims (13)

[Claims] 1. A first image pickup device having a first number of scanning lines for picking up an image of the monitoring target region; provided in parallel with the first image pickup device in the scanning line direction, and picking up the monitoring target region. A second image pickup device having a second number of scanning lines; a first image pickup device and a second image pickup device from two image pickup devices of the first image pickup device and the second image pickup device. A scanning line extracting means for extracting a third smaller number of corresponding scanning lines; a distance to the object based on an image formed on the scanning line extracted by the scanning line extracting means; Distance information acquisition means for acquiring information;
A detection processing unit that monitors the object using the distance information acquired by the distance information acquisition unit; a monitoring system. 2. The distance information acquisition means; a difference image from image information of images formed on each of the extracted scanning lines of images captured at different times for each of the two image pickup devices. A difference image forming unit that forms a difference image forming unit; and a region extracting unit that extracts a maximum region among regions divided by pixels whose absolute values of pixel values forming the difference image are larger than a predetermined threshold value. The monitoring system according to claim 1. 3. The distance information acquisition means calculates a correlation output value between the maximum areas extracted by the area extraction means from a difference image corresponding to each image picked up by the two image pickup elements. The monitoring system according to claim 2, further comprising: a correlation output calculating unit; acquiring distance information to the object based on the correlation output value. 4. The distance information acquisition means uses the threshold value in each of the maximum areas extracted by the area extraction means from the difference images corresponding to the images captured by the two image sensors. Seeking correspondence of large pixels,
The monitoring system according to claim 2, wherein the distance information to the object is acquired based on the positional deviation of the corresponding pixels. 5. The surveillance system according to claim 1, wherein the images formed on the scan lines corresponding to each other are extracted in synchronization. 6. The monitoring system according to claim 1, wherein the scanning line extracting means extracts at least two scanning lines. 7. A means for calculating a correlation value with respect to a scanning line of the first image sensor which scans the object and a scanning line of the second image sensor which scans the object. 7. The apparatus according to claim 1, wherein the positional relationship between the first image sensor and the second image sensor is determined so that the calculated correlation value is maximized. The monitoring system according to item. 8. A means for calculating a correlation value with respect to a scanning line which scans the object in the first image sensor and a scanning line which scans the object in the second image sensor. A corresponding scanning line selecting means for defining a positional correspondence between the scanning line of the first image sensor and the scanning line of the second image sensor so that the calculated correlation value is maximized. The monitoring system according to any one of claims 1 to 6. 9. The monitoring system according to claim 1, wherein the image pickup device is a CCD camera or a CMOS sensor. 10. An image pickup device for picking up an image of a region to be monitored, the image pickup device comprising a plurality of one-dimensional light-receiving device arrays corresponding to each other between the two image pickup devices; Distance information obtaining means for obtaining distance information to the object based on an image formed on the array; detection processing for monitoring the object using the distance information obtained by the distance information acquiring means A monitoring system. 11. An imaging step of imaging a monitoring target area by two image pickup devices having two image pickup devices having a first number of scanning lines and a second image pickup device having a second number of scanning lines. A scanning line extracting step of extracting from the two image pickup devices a third number of scanning lines corresponding to each other, which is smaller than either the first number or the second number, and the scanning line extracting step; A distance information acquisition step of acquiring distance information to the object based on the image formed on the scanning line; and monitoring the object using the distance information acquired by the distance information acquisition step. A detection processing step; a monitoring method. 12. A scan line for scanning the object in a first image sensor of the two image sensors, and a scan line for scanning the object in a second image sensor corresponding to the scan line. 12. The monitoring method according to claim 11, wherein the positional relationship between the first image pickup device and the second image pickup device is determined so that the same exists on the same plane; 13. Among the two image pickup devices, a correlation value is set for a scanning line of the first image pickup device for scanning the object and a scanning line of the second image pickup device for scanning the object. 12. The method according to claim 11 or claim 1, further comprising a step of calculating; a positional relationship between the first image sensor and the second image sensor is determined so that the calculated correlation value is maximized.
The monitoring method described in 2.