JP2004120009A - Monitoring system and monitoring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring system and a monitoring method for monitoring a monitoring object intruding into a three-dimensional monitoring space at night or in the dark. <P>SOLUTION: The monitoring system is provided with: an infrared ray emitting device 17 for emitting an infrared ray to the monitoring space 10; a first imaging apparatus 16 for imaging the infrared ray reflected in the monitoring object 12 at a first time and a second time delayed from the first time by a prescribed time; a second imaging apparatus 18 installed toward the monitoring space 10 apart from the first imaging apparatus 16 by a prescribed interval and for imaging the infrared ray reflected in the monitoring object 12 at the first and second times; difference image forming sections 24, 25 for forming a difference image between the image imaged at the first time and the image imaged at the second time; a position arithmetic section 30 for the monitoring object for calculating the position of the monitoring object 12 on the basis of the difference images 24, 25; and a proximity notice section 32 for detecting the approach of the monitoring object 12 in a direction of the monitoring space 10 to provide an output of a detection signal. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a monitoring device corresponding to nighttime or darkness, and in particular, changes in height and moving direction of a person, a finger, a hand, an arm, a body, etc. of a person, a human body, and the like entering a monitoring space using a CCD camera or a CMOS sensor. The present invention relates to a monitoring device and a monitoring method for monitoring a device.
[0002]
[Prior art]
2. Description of the Related Art In general, a monitoring apparatus that measures an image of a monitoring target at night or in the dark without being affected by the posture of the monitoring target or room illumination light is known. The conventional monitoring apparatus includes an imaging unit that emits a predetermined illumination pattern and continuously captures light of the emitted wavelength. The illumination pattern is obtained from images of two frames acquired at different times by the imaging unit. After calculating the inter-frame movement amount, the inter-frame movement amount is arranged in time series to generate movement amount waveform data, and the movement of the monitoring target is detected from the movement amount waveform data.
[0003]
[Patent Document 1]
Patent Publication No. 2002-175582 (paragraph number 0029, FIG. 1)
[0004]
[Problems to be solved by the invention]
However, as described above, according to the conventional monitoring device, since the movement of the monitoring target is detected based on the movement amount waveform data arranged in time series, the calculation process of the moving direction of the monitoring target becomes complicated, and the monitoring target moves to a predetermined area. It has been difficult to detect in a short time that an object to be monitored detects proximity. In addition, an imaging device that monitors an object to be monitored using artificial indoor illumination light or natural light has a problem in that, at night or in the dark, the edge detection accuracy of the object to be monitored is reduced and the movement of the object to be monitored is accurately measured. Was difficult.
[0005]
Therefore, the present invention provides a monitoring device and a monitoring method for measuring a distance between a monitoring target that enters a three-dimensional monitoring space at night or in the dark and an imaging device, and monitoring a monitoring target that approaches a predetermined area. It is an object.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, a monitoring device 1 according to the first aspect of the present invention, as shown in FIG. 1, for example, detects a monitoring target entering a three-dimensional monitoring space and outputs a detection signal. An infrared irradiating device 17 that irradiates the monitoring space 10 with infrared light in response to a predetermined threshold signal indicating a decrease in the amount of visible light in the monitoring space; A first imaging device 16 that captures infrared light reflected from the monitoring target 12 at a second time that is delayed by a predetermined time from the first time, and a predetermined interval from the first imaging device 16 A second imaging device 18 that is installed facing the monitoring space 10 and captures infrared light reflected from the monitoring target 12 at a first time and a second time; Forming a difference image between the captured image and the image captured at the second time Monitoring based on the difference image forming units 24 and 25 and the difference image 24 of the image formed by the first imaging device and the difference image 25 of the image formed by the second imaging device formed by the difference image forming units 24 and 25. Based on the calculation result of the position calculation unit 30 of the monitoring target which calculates the position of the target 12 and the position calculation unit 30, the monitoring target 12 detects the approach to the monitoring areas 42 and 43 and outputs a detection signal. And an approach notifying unit 32.
[0007]
With this configuration, the monitoring target 12 calculates the position of the monitoring target 12 and calculates whether the monitoring target 12 approaches the monitoring areas 42 and 43 based on the calculation result of the position calculation unit 30. Since the apparatus includes the approach notification unit 32 that detects and outputs a detection signal, the monitoring target 12 can determine a position change to the monitoring areas 42 and 43 and output a detection signal.
[0008]
In order to achieve the above object, the monitoring device 1 according to claim 1 of the invention according to claim 2 further includes, for example, as shown in FIG. 1, an illuminance measuring device 29 that detects visible light in the monitoring space 10. .
[0009]
With this configuration, the infrared irradiating device 17 can irradiate the monitoring space 10 with infrared light in response to a predetermined threshold signal indicating a decrease in visible light transmitted from the illuminance measuring device 29.
[0010]
In order to achieve the above object, the monitoring device 1 according to any one of claims 1 to 2 according to the invention according to claim 3 includes, for example, as shown in FIG. The movement direction 27 of the monitoring target 12 is calculated by detecting a temporal change in the position of the target 12.
[0011]
With this configuration, the position of the monitoring target 12 is detected and the moving direction 27 of the monitoring target 12 is calculated to detect the change over time. Twelve positions can be detected.
[0012]
In order to achieve the above object, a monitoring method according to the invention according to claim 4 is, for example, a monitoring method for detecting a monitoring target approaching a three-dimensional monitoring space and outputting a detection signal as shown in FIG. There is an infrared irradiation step of measuring the amount of visible light in the monitoring space 10 and emitting infrared light when a predetermined undervisible light amount state is reached, and a plurality of images obtained by imaging the monitoring space 10 with infrared light. The monitor object 12 is monitored based on the movement calculation process of the monitor object 12 that forms the difference images 24 and 25 and calculates the height 26 and the movement direction 27 of the monitor object 12, based on the calculation result of the movement calculation process. An approach notification step of detecting approach to the regions 42 and 43 and outputting a detection signal 39; measuring an amount of visible light in the monitoring space 10; and stopping the emission of infrared light when a predetermined amount of visible light is reached. An irradiation stopping step.
[0013]
With this configuration, it is possible to detect the approach of the monitoring target 12 to the monitoring areas 42 and 43 and output the detection signal 39.
[0014]
In order to achieve the above object, a monitoring method according to a fifth aspect of the present invention is a monitoring method for detecting a monitoring object approaching a three-dimensional monitoring space and outputting a detection signal as shown in FIG. Measuring the amount of visible light in the monitoring space 10, and stopping the infrared light emission 17 when a predetermined amount of visible light is reached, and a plurality of images obtained by imaging the monitoring space 10 with visible light. Based on the calculation results of the monitoring object moving and calculating the height 26 and the moving direction 27 of the monitoring object 12, the monitoring object 12 An approach notification step of detecting approach to the monitoring areas 42 and 43 and outputting a detection signal 39.
[0015]
With this configuration, based on a plurality of images obtained by capturing the monitoring space 10 with visible light, the monitoring target 12 detects the approach of the monitoring areas 42 and 43 based on the calculation result of the movement calculation step, and the detection signal 39 Can be output.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding members are denoted by the same or similar reference numerals, and overlapping description will be omitted.
[0017]
FIG. 1 is a schematic partially broken perspective view of a monitoring device 1 according to an embodiment of the present invention. In the figure, a monitoring target 12 stands upright on the floor, and observes the direction of a monitoring space 10 in which a security target 38 such as a showcase, a safe, a product storage, and a computer is installed. In the present embodiment, the monitoring target 12 is a person 12, a finger, a hand, an arm, and a body of the person.
[0018]
The monitoring device 1 includes a first imaging device 16 installed toward the monitoring space 10 and a second imaging device installed toward the monitoring space 10 at a predetermined distance w from the first imaging device 16. The apparatus includes a device 18 and a control device 2 electrically connected to the first and second imaging devices.
[0019]
The control device 2 includes a calculation unit 30, a memory unit 31, an interface I / F 19, a notification unit 32, and an illuminance measurement unit 29 as an illuminance measurement device.
[0020]
The monitoring device 1 receives the imaging data transmitted from the first or second imaging device via the interface I / F 19, extracts the illuminance data by the operation unit 30, and transmits the illuminance data to the illuminance measurement unit 29. The illuminance measurement unit 29 compares the received illuminance data with a pre-stored predetermined threshold indicating a decrease in visible light, and responds to the illuminance data below the threshold from the illuminance measurement unit 29 to indicate a signal indicating a decrease in the amount of visible light. Is transmitted to the arithmetic unit 30.
[0021]
The operation unit 30 responds to the reception of the signal indicating the decrease in the amount of visible light, via the I / F 19, the first CCD camera 16 as the first imaging device and the second CCD camera as the second imaging device. A control signal for causing the infrared irradiating device 17 including a plurality of infrared light emission devices LED or infrared laser diode attached to the CCD camera 18 to emit light is transmitted to the first and second CCD cameras, and the infrared irradiating device 17 transmits the control signal. Light is irradiated toward the monitoring space 10. Further, the drive power supply of the infrared irradiation device 17 may supply DC power according to the universal serial bus USB standard from the I / F 19. The driving power of the infrared irradiation device 17 may be supplied directly from a DC power adapter (not shown).
[0022]
Here, the first and second CCD cameras have an automatic switching function of increasing or decreasing the amount of video signals so as to output a color video signal when shooting at daytime and to output a black and white video signal when shooting at night. May be prepared. The first and second CCD cameras may be constituted by monochrome CCDs, and may be configured to output a monochrome video signal regardless of day and night.
[0023]
Further, when each of the infrared irradiation devices 17 is exemplified, an infrared irradiation device 17 having an illuminance of about 10 lux (Lux) is attached to the first and second CCD cameras, and a total of 24 infrared irradiation devices 17 are used in darkness. Secure a monitoring distance of about 10m. Similarly, a monitoring distance of about 15 m is ensured in the dark by a total of 56 infrared irradiation devices 17. Similarly, a monitoring distance of about 20 m can be secured in the dark by a total of 104 infrared irradiation devices 17. The turning on and off of the infrared irradiation device 17 is automatically controlled by the control signal transmitted from the arithmetic unit 30 as described above.
[0024]
The first CCD camera 16 transmits imaging information of the monitoring space 10 taken at the first time to the control device 2 via the I / F 19, specifies an address by the arithmetic unit 30, and takes an image of the inside of the memory unit 31. The signal is stored in the signal storage area 20. Further, the second CCD camera also transmits imaging information of the monitoring space 10 taken at the first time to the control device 2 via the I / F 19, and designates an address by the computing unit 30 to take an image of the inside of the memory unit 31. It is stored in the signal storage area 22.
[0025]
The first CCD camera 16 transmits to the control device 2 via the I / F 19 imaging information of the monitoring space 10 captured at a second time delayed by a predetermined time from the first time, and the arithmetic unit 30 An address is designated and stored in the imaging signal storage area 21 in the memory unit 31. The second CCD camera also transmits to the control device 2 via the I / F 19 imaging information of the monitoring space 10 taken at a second time delayed by a predetermined time from the first time, and the arithmetic unit 30 An address is designated and stored in the imaging signal storage area 23 in the memory unit 31.
[0026]
In the surveillance space 10, a security object 38 is installed substantially at the center, a cylindrical mask area is set around the security object 38, and a space between the security object 38 and the outer periphery of the monitoring area is placed on the floor. Thus, a curtain region that cannot be seen in the vertical direction is formed.
[0027]
The invisible curtain area has an outer edge 42, an inner edge 43, a width D, and a height H surrounding the security object 38 from a horizontal 360-degree direction, and includes the first CCD camera 16 and the second CCD camera 18. Are imaged from directly above.
[0028]
Typically, a CCD camera can be used for the first image sensor and the second image sensor, but CMOS sensors may be used. The CCD cameras 16 and 18 are typically used as a passive distance detecting device having two image pickup devices and measuring distance by trigonometry.
[0029]
The first and second CCD cameras simultaneously capture the infrared light reflected from the security object 38 from directly above, or capture the infrared light from above with a predetermined angle. The first and second CCD cameras 16 and 18 are set apart from each other by a predetermined distance w, and both CCD cameras also image a curtain area that cannot be seen with a predetermined parallax.
[0030]
The first CCD camera 16 and the second CCD camera 18 are connected to the control device 2 via a signal line and an I / F 19 and transmit an image pickup signal to the control device 2 by the above-described USB standard serial method. .
[0031]
The control device 2 includes an interface I / F 19, a calculation unit 30, a memory unit 31, and a notification unit 32 therein, and converts an image signal received from the first CCD camera 16 and the second CCD camera 18 into the I / F 19. The calculation unit 30 sends the imaging signal to the memory unit 31 for storage.
[0032]
The memory unit 31 includes an image signal storage unit 20 at a first time for storing an image signal obtained by infrared imaging with the first CCD camera 16 at a first time, and a first image signal storage unit 20 at a second time. A second time imaging signal storage unit 21 that stores an imaging signal obtained by infrared imaging at a first time, and a first time imaging signal storage unit 22 that stores an imaging signal captured by the second CCD camera 18 at a first time. And an image signal storage unit 23 at a second time for storing an image signal captured by the second CCD camera 18 at a second time.
[0033]
The operation unit 30 has a first difference image storage unit 24 for logically operating and storing the difference image obtained by the first CCD camera 16, and a second operation for logically operating and storing the difference image obtained by the second CCD camera 18. A difference image storage unit 25, a height storage unit 26 that logically calculates and stores the height of the person 12 or the height of the floor surface that has entered the invisible curtain area, and a movement that logically calculates and stores the moving distance of the person 12 A distance storage unit 27 and a mask address storage unit 28 for storing a mask address of a mask area in the monitoring space 10 set from the outside are provided.
[0034]
The notification unit 32 outputs a detection signal to the line 39 in response to the control signal output from the calculation unit 30, activates the external alarm generation unit 37, and generates an alarm sound. Similarly, in response to a control signal output from the arithmetic unit 30, a detection signal is output to a line 34, and an alarm signal is automatically sent to a security company located at a remote place via a ground line 36. The detection signal can be transmitted via a line 34 to a terminal 35 inserted and arranged between the telephone 33 and a land line 36. Further, it is also possible to make a radio report to a mobile phone carrier instead of the ground line 36 used by the telephone 33. In short, as long as it is a means that can contact the security company, the alarm signal may be transmitted using a communication means such as a public line, a ground line, a wireless carrier, and the Internet.
[0035]
The monitoring space 10 in the figure is set in an independent room separated by a wall 44. A passage 41 is provided in front of the wall 44, and the security guard 15 carries a flashlight at night to guard. The passage 41 is monitored by a security device (not shown) at night, but when the security guard 15 patrols the passage 41, the security guard is released by the security guard 15.
[0036]
The monitoring device 1 monitors the monitoring space 10 even when the security device on the side of the passage 41 is released, detects a person entering the monitoring space 10 even if it is the security guard 15, and detects the intruder approaching the security object 38. It is configured to issue a warning by judging a person. That is, the surveillance space 10 is designated as an off-limits zone except for the persons concerned.
[0037]
With reference to the schematic plan view of FIG. 2, the mask processing of the monitoring space 10 applied to the second embodiment of the present invention will be described. The monitoring device 1 irradiates infrared light from the infrared irradiation device 8 connected to the I / F 19 toward the security object 38, and reflects the infrared light from the monitoring space 10 with the first CCD camera 16 and the second CCD camera 18. Of the person 12 entering the angle of view of the first and second CCD cameras, the invisible curtain area, and the height information of the security object 38 at the same time.
[0038]
The monitoring device 1 can set an arbitrary range to the invisible curtain region and the mask region by the control device 2 of FIG. The control device 2 is constituted by hardware capable of changing the control process by a software program such as a personal computer or a microprocessor, and the first and the second positions are set at positions where the security object 38 is imaged directly above or from a predetermined angle. While the second CCD cameras 16 and 18 are installed and the image of the monitoring space 10 is viewed on the monitor, the mask area 50 surrounded by the broken line in the figure can be set at an arbitrary position.
[0039]
In the present embodiment, the maximum value of the radius around the security object 38 is set at the outer peripheral position of the mask area 50, and the image pickup signals in the mask area 50 imaged by the first and second CCD cameras are imaged. Can be removed from processing.
[0040]
Also, the outer edge 42 and inner edge 43 of the invisible curtain area, which is shown by hatching in the figure and surrounds the mask area 50 from the horizontal 360-degree direction, are set by a horizontal address (XY orthogonal coordinates) and a vertical address (Z coordinate). Then, the mask area 50 is surrounded by a curtain area invisible with a width D, and the temporal change in the height of the person 12, the finger, the hand, the arm, the torso, etc. of the person 12, which has entered the range of the width D, is measured.
[0041]
Accordingly, the infrared light reflected from the person 12 standing upright outside the invisible curtain area is imaged by the first and second CCD cameras. It can be removed from image processing.
[0042]
In the above-described embodiment, the first and second CCD cameras 16 and 18 as distance detecting devices that detect the approach of the person 12 by measuring the distance between the CCD cameras 16 and 18 and the person 12 are typically The apparatus is provided with two imaging devices for imaging the reflection of infrared light emitted from the infrared irradiation device 8, and is used as a passive distance detection device that measures distance by trigonometry.
[0043]
The above-described curtain region has been described as a plane or a cylindrical curved surface standing upright on the floor surface, but the distance to be set is arbitrary, so the curtain region is formed into a free curved surface, for example, a hemispherical or oval shape. You can also.
[0044]
Further, the curtain region has been described as a region having a thickness D sandwiched between the outer edge portion 42 and the inner edge portion 43. However, the present invention is not limited thereto, and the outer edge portion 42 and the inner edge portion 43 coincide with each other, or only the outer edge portion 42, or It may be a region where only the inner edge portion 43 has zero thickness. At that time, a detection signal is output depending on whether the monitoring target 12 passes through the curtain area.
[0045]
The operation of the monitoring device 1 will be described with reference to FIGS. The first CCD camera 16 has a predetermined number of scanning lines, the second CCD camera 18 has a predetermined number of scanning lines, and the two CCD cameras 16 and 18 are commonly installed in parallel in the scanning line direction. I have. Further, in the present embodiment, the first CCD camera 16 and the second CCD camera 18 will be described as separate bodies, but may be configured integrally. The curtain region surrounding the mask region 50 is a region having a thickness D sandwiched between the outer edge portion 42 and the inner edge portion 43, and the first and second CCD cameras move through the curtain region at night or in a dark state. The infrared light reflected from the moving person 12 is imaged. Further, in a state where natural light or artificial light due to fluorescent lamp irradiation is incident, irradiation of infrared light is stopped and reflection of visible light is imaged.
[0046]
Also, the number of scanning lines of the first and second CCD cameras is typically the same, and the number of scanning lines of each CCD camera is, for example, about 500, and each scanning line has about 500 Pixel to pixel. In the present embodiment, the first CCD camera 16 and the second CCD camera 18 are installed on the ceiling. However, the first CCD camera 16 and the second CCD camera 18 may be installed facing the monitoring space with an angle on the wall.
[0047]
The control device 2 includes an arithmetic unit 30 and controls the entire monitoring system 1 as a monitoring device. Further, inside the calculation unit 30, a reference distance to be compared with the measurement distance measured by the first and second CCD cameras is stored in, for example, the height storage unit 26. The reference distance may be a distance at a time point in the past of the monitoring time point. Typically, the reference distance is a distance of the background (floor) in a state where the person 12 does not exist in the invisible curtain area. The distance may be one frame before when the background distance is periodically detected.
[0048]
The control device 2 receives imaging signals from the first CCD camera 16 and the second CCD camera 18, and extracts images formed on scanning lines corresponding to each other in synchronization. Further, as the scanning lines to be synchronized, 500 scanning lines of each CCD camera may be extracted. The number of the invisible curtain areas may be thinned out at predetermined intervals.
[0049]
For example, when the torso of the person 12 is detected, a scanning line for capturing an image at an interval of about 100 mm is extracted. When a hand of the person 12 is detected, a scanning line for capturing an image at an interval of about 50 mm is extracted. In the case of detecting the finger, it is also possible to individually set so as to extract a scanning line for picking up an image of about 10 mm intervals, and to detect that any part of the person 12 has entered the invisible curtain area. For example, when at least about 500 scanning lines of each CCD camera are used, one scanning line may be selected at regular intervals and about 20 scanning lines may be extracted.
[0050]
Further, the curtain region has been described as a plane or a cylindrical curved surface erected in the direction perpendicular to the floor surface, but the distance to be set is arbitrary, so the curtain region can be formed into a free curved surface, for example, a hemispherical shape. .
[0051]
The curtain region has been described as a region having a thickness D sandwiched between the outer edge portion 42 and the inner edge portion 43. However, the present invention is not limited thereto, and the outer edge portion 42 and the inner edge portion 43 coincide with each other, or only the outer edge portion 42, or It may be a region where only the inner edge portion 43 has zero thickness. At that time, a detection signal is output depending on whether or not the monitoring target passes through the curtain area.
[0052]
Further, as for the scanning lines to be synchronized, it is preferable to extract each scanning line in time series. Further, when the scanning lines are extracted from the first CCD camera 16 and the second CCD camera 18, if the extraction processing is performed using an external synchronization signal, a circuit for extracting and processing the corresponding scanning lines is provided. Can be simplified, and software processing can be facilitated.
[0053]
The control device 2 receives the imaging signals captured at different times T0 and T1 for the first CCD camera 16 and the second CCD camera 18, respectively, and the first CCD camera 16 captures the image at the time T0. The imaging signal is stored in the imaging signal storage unit 20 at the first time, and the imaging signal captured by the second CCD camera 18 at the time T0 is stored in the imaging signal storage unit 22 at the first time.
[0054]
Further, the control device 2 stores the imaging signal captured by the first CCD camera 16 at the time T1 in the imaging signal storage unit 21 at the second time, and the imaging signal captured by the second CCD camera 18 at the time T1. Is stored in the imaging signal storage unit 23 at the second time.
[0055]
The arithmetic unit 30 performs a logical operation process based on image information of each image formed on a scanning line included in each image pickup signal, and stores the difference image data of the first CCD camera in the internal difference image storage unit 24 ( CCD1: difference), and the difference image data of the second CCD camera is stored in the internal difference image storage unit 25 (CCD2: difference). Subsequently, a logical operation is performed based on the two difference image data to perform a correlation process. An image data processing function is provided for storing the height H of the monitored object in the height storage unit 26 (height H) by this correlation processing. Note that the image information is typically a pixel value of a pixel constituting the image.
[0056]
The arithmetic unit 30 reads out the image signals of T0 and T1 from the image signal storage units 20 and 21 that store the image signals of the first CCD camera 16, respectively, and reads out the images formed on the scanning lines included in the image signals. A difference image is formed from the image information and stored in the internal first difference image storage unit 24. Further, the imaging signals of T0 and T1 are read from the imaging signal storage units 22 and 23 for storing the imaging signals of the second CCD camera 18, respectively, and the image information of each image formed on the scanning line included in each imaging signal is read out. A difference image is formed and stored in the internal second difference image storage unit 25.
[0057]
The two images for forming the difference image are acquired at an imaging time T1 that is delayed by a predetermined time from the imaging time T0, but the delay time to be shifted is substantially the same as the amount of movement of the person 12 is not too large. May be set to a time that can be regarded as substantially the same position, for example, about 0.1 second. Alternatively, it is set to 1 to 10 cycles (1/30 sec to 1/3 sec) of the television cycle. When such a difference image is calculated and obtained, the background of the person 12 is removed, and only the edge image of the moving person 12 can be extracted.
[0058]
In addition, the calculation unit 30 has a function as an area extracting unit that extracts a maximum area from areas divided by pixels whose absolute values of pixel values constituting the difference image are larger than a predetermined threshold. ing.
[0059]
Since the background of an image formed on the scanning line of each CCD camera does not move, the pixel value of the boundary of the moving person 12 changes rapidly. Therefore, the arithmetic unit 30 reads each difference image from the internal first difference image storage unit 24 and second difference image storage unit 25, and determines that the absolute value of each pixel value forming the difference image is greater than a predetermined threshold value. Can perform a logical operation that regards the largest area as a boundary of the person 12 among the areas divided by large pixels. Here, the boundary is set to the maximum area because the pixel value of the threshold value is small in the area divided by the movement of the clothes and gloves worn by the person 12 in the area other than the boundary. This is because it may change.
[0060]
In addition, the arithmetic unit 30 calculates the maximum region between the maximum regions extracted from the difference images corresponding to the respective images in which the infrared light or the visible light reflected by the first CCD camera 16 and the second CCD camera 18 is captured. It has a logical operation function as correlation output calculation means for calculating a correlation output value. The calculating unit 30 calculates the distance between the CCD camera and the person 12 based on the correlation output value obtained by performing the logical operation.
[0061]
The calculation unit 30 extracts, for example, image information of the largest region extracted from the difference image formed by the scanning lines, and specifies a point where the contrast of the person 12 greatly changes due to the movement of the person 12. Further, similarly to the scanning line of the first CCD camera 16, the image information of the maximum area of the difference image formed on the scanning line of the second CCD camera 18 is extracted. When the image information of the maximum area of each of the extracted difference images is compared, it means that the area images where the position of the maximum area of the difference image is slightly different from the parallax relationship of the two CCD cameras are extracted.
[0062]
Subsequently, the arithmetic unit 30 performs a logical operation of the correlation processing between the difference image obtained by the first CCD camera and the difference image obtained by the second CCD camera, and measures the position where the correlation peak is maximized, thereby obtaining two values. An average correlation output value between the difference images or a correlation output value substantially at the center of the extracted region is logically obtained and obtained.
[0063]
Here, the correlation output value is a relative imaging position difference generated by the parallax between the first CCD camera 16 and the second CCD camera 18, and is typically output in the number of pixels by the correlation processing. Value. The arithmetic unit 30 calculates the distance between the CCD camera and the person 12 using trigonometry based on the parallax between the scan line of the first CCD camera and the scan line of the second CCD camera based on the correlation output value.
[0064]
Further, the correlation processing means that one of the images obtained from the scanning lines of the first CCD camera and the scanning lines of the second CCD camera is shifted by a pixel unit until the two images substantially match. This is a process of calculating the shifted amount by indicating the amount by the number of pixels, for example. The determination as to whether or not the two difference images match is performed based on the overall strength of the signals that overlap when one difference image is fixed and the other difference image is compared while being shifted. The coordinates at which the signal peaks are coincident points and the correlation peak positions.
[0065]
In the correlation process, a difference image corresponding to each image captured by the first CCD camera 16 and the second CCD camera 18 is binarized with an appropriate value, and an edge portion thereof is extracted, whereby there is movement. A configuration may be adopted in which a region portion is extracted and a correlation process is performed only on the extracted region. With this configuration, the correlation processing uses the binarized difference image, so that the calculation processing speed can be improved.
[0066]
The arithmetic unit 30 is configured to output predetermined infrared light reflected by the first CCD camera 16 and the second CCD camera 18 from a maximum area extracted from a difference image corresponding to each of the captured images. A process may be performed such that a correspondence between pixels larger than the threshold value is obtained, and a distance between the CCD camera and the person 12 is logically calculated based on the displacement of the corresponding pixel. In this case, the calculation unit 30 does not calculate the correlation output value between the largest regions extracted from the difference image as described above, but obtains the correspondence of the pixels larger than the threshold in the largest region, and The displacement of the pixel is calculated.
[0067]
The calculation unit 30 calculates the distance from the CCD camera to the person 12 using the trigonometry based on the displacement of the corresponding pixel, that is, the parallax of the CCD camera. Thereby, the calculation unit 30 can, for example, number the pixels exceeding the threshold value from the same direction, and obtain the parallax from the positional shift between the pixels of the corresponding numbers.
[0068]
With such a configuration, the calculation unit 30 can calculate the parallax by a very simple process, omitting the correlation process. Further, for example, when many pixels exceeding the threshold value are detected due to a change in contrast due to a pattern or the like in the extracted area, the arithmetic unit 30 performs processing so that numbering is not performed on portions other than the above-described edge portions. . Accordingly, it is possible to reduce a phenomenon in which a pixel having a corresponding number is shifted from a pixel to be originally handled due to the influence of parallax, thereby causing an error in a value of a position shift. This is also effective when the parallax increases due to the short distance to the person 12.
[0069]
FIG. 3 is a schematic partially broken perspective view of a monitoring device according to another embodiment of the present invention. In the figure, a person 12 stands upright on an indoor floor such as a painting hall and faces a wall 52 on which a painting 53 is hung. Also in the present embodiment, the monitoring target is the person 12, the finger, the hand, the arm, the body, and the like of the person.
[0070]
A wall 54 erected opposite the wall 52 embeds the first CCD camera 16 and the second CCD camera 18, and monitors the person 12 approaching the painting 53. The infrared irradiating device 17 provided so as to surround the lens portions of the first and second CCD cameras emits light at night or in a dark state, and directs infrared light toward the person 12, the wall 52, and the picture 53. Irradiate.
[0071]
The first CCD camera 16 and the second CCD camera 18 are set apart from each other by a predetermined distance w, and their optical axes are set parallel to each other. Also, the video output signals of the first CCD camera 16 and the second CCD camera 18 are transmitted to the control device 2 in the same manner as in the above-described embodiment, and thus the duplicate description will be omitted.
[0072]
The first CCD camera 16 and the second CCD camera 18 have an angle of view covering the passage in front of the wall 52, and monitor an invisible curtain area 57 shown by hatching in the figure. The invisible curtain area 57 is set in the vicinity of the painting 53, has a height H and a width D, and captures infrared light reflected from the person 12 approaching the painting 53 from the passage.
[0073]
Further, a mask area may be set on the side of the passage adjacent to the invisible curtain area 57. In this case, the stereo video output signal of the infrared light transmitted from the first CCD camera 16 and the second CCD camera 18 can be configured so that the control device 2 removes the imaging signal of the mask area. . The same processing as that of the above-described embodiment can be used for the processing of the mask area, and thus the overlapping description will be omitted.
[0074]
The control device 2 identifies a person who only walks in the passage and a person who enters the invisible curtain area 57 and approaches the painting 53. That is, as the person 12 approaching the picture 53 by measuring the change in the position of the person 12 approaching the picture 53 over time while the person 12 walking on the passage enters the invisible curtain area 57 in front of the picture 53. judge. Based on the result of the determination, the control device 2 that has detected the approach of the person 12 outputs a detection signal via the notification unit, and issues an alarm signal to the outside.
[0075]
The invisible curtain area 57 is set to a distance close to the painting 53 from a range where the hand of the person 12 cannot reach the painting 53. For example, the approaching area of the person 12 with a width D of 10 to 100 cm before the painting 53 is set. Is configured to be detected. With this configuration, an alarm can be issued before the person 12 contacts the painting 53.
[0076]
In the above embodiment, two CCD cameras 16 and 18 are buried in the wall 54 facing the picture 53 to monitor the front of the picture 53 as a three-dimensional monitoring space. For example, two CCD cameras can be installed so as to monitor the picture 53 from above or obliquely at a predetermined angle.
[0077]
Typically, the edge of the person 12 is extracted from the person 12 approaching the picture 53 by irradiating infrared light and the contrast difference between the wall 52 and the picture 53 or the floor.
[0078]
With this configuration, the contrast between the background and the person 12 can be obtained with high probability. In addition, since an image corresponding to the scanning line of the CCD camera corresponding to the head to the toe of the person 12 is obtained, image information of a plurality of scanning lines can be averaged, and monitoring accuracy can be improved.
[0079]
For example, if the number of scanning lines of the CCD camera is α, the standard deviation of the image information can be obtained by averaging the distance values corresponding to the α number, and the accuracy of the standard deviation of the route α can be obtained. Performance can be improved. In this case, the movement of the person 12 and the other person walking along the passage are simultaneously imaged. However, based on the infrared light image information captured by the two CCD cameras 16 and 18, the other person and the person The distance between the object 12 and the CCD camera is calculated, and the approaching distance of only the person 12 as the monitoring target existing at a predetermined distance or more (the width D of the invisible curtain area) is extracted, and other moving image information is obtained. Is configured to be removed.
[0080]
In the above embodiment, based on the contrast between the wall 52 and the painting 53 that are stationary due to the irradiation of the infrared light and the moving person 12, an edge is detected and a difference image is formed, and then the correlation processing is performed. Although the approaching state was monitored, as another embodiment, the first and second CCD cameras were installed indoors, an outdoor scene was imaged by infrared light through the glass window, and the person approaching the glass window was observed. It can also be detected based on distance information.
[0081]
In this case, trees and flowers moving in the scenery, a moving car, and the approaching person are simultaneously imaged. Based on the infrared light image information captured by the two CCD cameras 16 and 18, The moving background and the distance between the person 12 and the CCD camera are calculated, the approaching distance of only the person 12 as a monitoring target which is shorter than a predetermined distance is extracted, and other moving image information is removed. I do.
[0082]
In this way, the curtain area and the mask area which are not visible at a distance from the glass window are set, and the distance information of the person approaching the area less than the predetermined distance is extracted from the two CCD cameras to detect the approach of the person. Can be detected.
[0083]
FIG. 4 is a schematic perspective view of the imaging device 16 and the infrared irradiation device 17 used in the embodiment of the present invention. The imaging device 16 has the lens 11 disposed on the front surface, and the plurality of infrared irradiation devices 17 are disposed so as to surround the lens 11 by 360 °. The imaging device 16 and the infrared irradiating device 17 are electrically connected to the interface I / F 19, and are configured to transmit imaging data of visible light received by the imaging device 16 to the interface I / F 19. The infrared irradiation device 17 receives a light emission signal responsive to a predetermined threshold signal indicating a decrease in visible light through the I / F 19 and irradiates the monitoring space with infrared light.
[0084]
When the visible light received by the imaging device 16 reaches a predetermined visible light amount (illuminance), the infrared irradiation device 17 receives a light emission stop signal via the I / F 19 and stops emitting infrared light.
[0085]
FIG. 5 is a schematic perspective view of another infrared irradiation device 8 used in the embodiment of the present invention. The infrared irradiating device 8 is a stand-alone infrared irradiating device supported by the stand 4 and erected in the vertical direction. A housing is installed on the upper part of the stand 4, and infrared irradiation units 7 arranged in a matrix on one main surface of the housing are provided in 8 rows and 4 columns.
[0086]
An illuminance measuring device 6 for measuring visible light is arranged below the infrared irradiating units 7 arranged in a matrix. The illuminance measuring device 6 can detect an amount of light by using an optical sensor such as a photodiode, providing an infrared cut filter (not shown) in front, and transmitting only visible light.
[0087]
The infrared irradiator 8 receives power from a commercial power supply via the AC power plug 5 and independently measures the visible light in the monitored space, responds to a predetermined threshold signal indicating a decrease in visible light, Irradiates the monitoring space with infrared light from the infrared irradiating section 7 of. On the other hand, when the amount of visible light in the monitoring space increases and reaches a predetermined amount of visible light, the emission of infrared light is independently stopped.
[0088]
With reference to the flowchart of FIG. 6, a method for monitoring a three-dimensional monitoring space will be exemplified. Here, the monitoring device can use a common processing method corresponding to visible light and infrared light. The left image is an image captured by the first CCD camera 16, and the right image is described as an image captured by the second CCD camera 18. The process starts from a start step S10, and in step S11, the control device 2 converts the left image and the right image captured by the first CCD camera 16 and the second CCD camera 18 via the imaging device interface I / F19. To obtain a stereo image. Further, the acquired stereo image is subjected to mask processing to remove image information outside the mask region 50 surrounded by the invisible curtain region and the invisible curtain region.
[0089]
The control device 2 acquires the left image of the t frame, which is the first image of the person 12 in the invisible curtain area at the time point t (step S13). Next, a left image of the t + Δt frame, which is the second image captured at the time point t + Δt at which the predetermined time Δt has elapsed, is acquired (step S14). Subsequently, a difference image between the left image of the t frame and the left image of the t + Δt frame is extracted (step S15). The difference image is subjected to a dynamic region extraction process (Step S16) in which the background is removed from the extracted difference image in correspondence with one of the t frame and the t + Δt frame, and then is subjected to the correlation process of the subsequent Step S21. Move to
[0090]
Further, the control device 2 acquires a right image of a t-frame which is a third image of the person 12 in the invisible curtain area at the time point t (step S17). Next, the right image of the t + Δt frame, which is the fourth image captured at the time point t + Δt after the elapse of the predetermined time Δt, is acquired (step S18). Subsequently, a difference image between the right image of the t frame and the right image of the t + Δt frame is extracted (step S19). The difference image is subjected to a dynamic region extraction process (step S20) in which the background is removed from the extracted difference image in correspondence with either one of the t frame and the t + Δt frame, and then the correlation process of the subsequent step S21. Move to
[0091]
In the present embodiment, the first CCD camera and the second CCD camera perform imaging simultaneously and in parallel at each time point (t, t + Δt). In this case, Δt may be appropriately determined according to the setting conditions of the counting device, but is desirably the time from when one frame N is imaged until the next frame N + 1 is imaged.
[0092]
Next, the corresponding point of the right image of the t frame is searched for using the outline of the person 12 having the strong contrast of the left image of the t frame, and the correlation process is executed (step S21). Height information is calculated from the corresponding points obtained by the correlation processing (step S22). For example, the height of the person 12 is calculated and output based on the number of pixels for matching the left image and the right image.
[0093]
The height information of the person 12 obtained in the above step S22 is compared with the coordinate information of the invisible curtain area, and the person 12 or the height of the part enters the invisible curtain area and approaches the security object 38. It is determined whether or not it has been performed (step S23).
[0094]
For example, the height information of the person 12 obtained at the immediately preceding time is compared with the height information of the current person 12, and it is determined whether the current person 12 or a part thereof approaches the security object 38 or not. The determination is made based on the pixel address indicating the outline of the part. When the contour address of the person 12 is away from the address of the outer edge 42 of the invisible curtain area width D shown in FIG. 1 and approaches the address of the inner edge 43 (Yes in step 23), the person 12 or a part thereof is a security target. 38, the process branches to step 24, and outputs a detection signal as a detection signal, and then proceeds to step 25 to end the process.
[0095]
In addition, in the case of application to a painting theater, when the contour address of the person 12 is away from the passage-side address of the invisible curtain area 57 shown in FIG. 3 and approaches the address on the wall 52 side (Yes in step 23), the person 12 Alternatively, it is determined that the part is approaching the picture 53, and the process branches to step 24, outputs a detection signal, and then proceeds to step 25 to end the process.
[0096]
On the other hand, the contour address of the person 12 is compared with the address of the outer edge portion 42 and the address of the inner edge portion 43 of the invisible curtain region width D shown in FIG. When approaching the address of the outer edge portion 42 (No in Step 23), the process branches to the node N12 to acquire a frame at the next time. The height information of the newly acquired person 12 is calculated (step S22), and the approach of the person 12 is determined in association with the curtain area that cannot be seen in step 23 (step S23).
[0097]
The determination conditions in step 23 will be described using the monitoring device shown in FIG. When the person 12 advances in the direction of the painting 53 and enters the invisible curtain area width D, the contour address of the person 12 moves away from the passage-side address of the invisible curtain area width D shown in FIG. (Step 23 is Yes), it is determined that the person 12 or its part is approaching the picture 53, and the flow branches to step S24 to output a detection signal. Thereafter, the process proceeds to step 25 to end the processing.
[0098]
On the other hand, the contour address of the person 12 is compared with the passage-side address of the invisible curtain area width D and the address of the painting 53 shown in FIG. When approaching the address (No in step 23), the process branches to node N12 to acquire a frame at the next time. Further, the height information of the person 12 is calculated to detect the moving direction within the width D of the invisible curtain area (step S23).
[0099]
Here, with reference to the schematic block diagram of FIG. 7, a method of calculating the distance between the CCD camera and the person 12 using trigonometry will be described with reference to the distance (height) when the person 12 is imaged from vertically above. An example of a measurement method will be described. Here, w is the distance between optical axes (base line length) between the first CCD camera 16 and the second CCD camera 18, f is the focal length of each CCD camera when the light receiving lens is a single lens, d is the parallax on the image plane of the CCD camera. When a commonly used combination lens is used, the focal length f here is the focal length f of the combination lens. Thus, the distance a between the target person 12 and the CCD camera can be calculated by the following equation.
a = w × f / d (1)
In this way, the sensor control unit 60 that has acquired the stereo images from the first and second CCD cameras 16 and 18 uses the difference image forming unit 62 inside to detect the difference image of the first CCD camera 16 and the second CCD camera. The difference image of the camera 18 is formed, the two difference images are subjected to correlation processing by the correlation output operation unit 64, and a correlation processing signal is output to the control device 2, thereby calculating the distance from the CCD camera to the person 12. The height z of the person 12 is calculated by subtracting the distance between the person 12 and the reference distance h between the floor and the CCD camera measured in advance, and the height z value is stored in the height storage unit 26. Further, a configuration may be adopted in which video signals are directly transmitted from the two CCD cameras to the control device 2 without going through the sensor control unit 60.
[0100]
Returning to FIG. 1, the operation of the monitoring system 1 will be further described. The arithmetic unit 30 can monitor the person 12 based on the height value z stored in the internal height storage unit 26.
[0101]
The arithmetic unit 30 stores the height value z of the person 12 stored in the internal height storage unit 26 in time series, and determines the position information, the moving direction, the posture, and the like of the person 12 based on the time value z. To be configured. That is, by comparing the reference distance h with the height value z of the person 12 at a certain point in time, the presence of the person 12, its position information, the moving direction, the posture, and the like can be determined.
[0102]
Further, the calculation unit 30 determines the final existence of the person 12, its position information, the moving direction, the posture, and the like from the determination results respectively corresponding to a plurality of scanning lines extracted from the scanning lines of the CCD camera. The position information of the person 12 is processed. In addition, the moving direction of the person 12 includes not only a horizontal change in the position of the person 12 but also a vertical change in which the person 12 stands or sits, for example.
[0103]
In the above embodiment, the monitoring device 1 that monitors the monitoring space 10 with the two CCD cameras installed vertically above the person 12 has been described. However, as another embodiment, the two CCD cameras are connected to the monitoring space 10. It can be installed on a wall or a pillar located in the vicinity of the camera, and can monitor a curtain area that is not visible toward the monitoring space 10 at a predetermined angle.
[0104]
For example, two CCD cameras are installed on a wall or a pillar having a height H from the floor surface, and a horizontal distance from the two CCD cameras to a curtain area invisible to the two CCD cameras is L. When the distance between the 12 parts such as fingers, hands, arms and torso and the two CCD cameras 16 and 18 is measured as A, the distance from the part of the person 12 to the installation position of the CCD cameras 16 and 18 is determined. The height H1 in the direction and the distance L1 from the wall or the pillar can be obtained by the following equation.
H1 = AH / (H²+ L²)^{1 / 2}……… (2)
L1 = AL / (H²+ L²)^{1 / 2}……… (3)
[0105]
The position information of the position of the person 12 moving toward the monitoring space 10 is sequentially calculated based on the calculated height H1, the distance L1, and the distance measurement directions of the two CCD cameras 16 and 18, and the change over time is calculated. By comparing the front and the rear, it is possible to detect a state in which the part of the person 12 approaches or stops in the direction of the security object 38 and moves away in the curtain area where the part of the person 12 is not visible.
[0106]
Further, the position of the part of the person 12 can be roughly grasped from the position of the invisible curtain region corresponding to each of the arbitrarily extracted number of scanning lines. Thus, the position information of the part of the person 12 existing in the invisible curtain area can be calculated. If two CCD cameras 16 and 18 are arranged in a direction perpendicular to the floor surface, the position at which the cameras are attached is the position of the person 12 and the distance from the CCD camera is the height. Can also be easily obtained.
[0107]
The detection processing of the person 12 processed by the arithmetic unit 30 is delayed by a predetermined time from the position information of the person 12 calculated at the immediately preceding time by the same or different scanning line of the CCD camera, for example, the position information immediately before the same. The moving distance and the average moving speed, which are the moving information of the person 12, are calculated by comparing the positional information with the position information at the time.
[0108]
For example, the moving distance is determined by the difference between the latest position information and the data of the height H1 and the distance L1 of the position information at the immediately preceding time, and the average moving speed is delayed by a predetermined time from the difference between the data. The value is divided by the distance measurement interval time. Further, the moving distance, moving direction, and moving speed of the person 12 calculated and output by the calculating unit 30 are stored in the internal moving distance storage unit 27.
[0109]
In this way, the calculation unit 30 can determine the presence, the posture, the position, and the moving state of the person 12 in the invisible curtain area from the calculated position information and movement information of the person 12.
[0110]
For example, a state where the part of the person 12 is stopped, a state where the part of the person 12 is approaching in the direction of the monitoring space 10, and a state where the person 12 moves away from the monitoring space 10 can be accurately determined. Is present at a low height and is stopped, it can be determined that this person 12 is in a state of falling down in the invisible curtain area.
[0111]
In this way, the monitoring system 1 can accurately monitor the portion of the person 12 that has entered the invisible curtain area, the presence and the presence state of the person 12, and the like, by measuring image information over time.
[0112]
Since the CCD camera can easily have a wide angle of view, the use of a CCD camera with a wide angle of view can provide a sufficiently large monitoring target area in the horizontal direction. By applying such a monitoring system 1 to monitoring of various facilities, by increasing or decreasing the angle of view of the first CCD camera 16 and the second CCD camera 18 and setting the invisible curtain area to an arbitrary width, For example, detecting a state in which the hand of the person 12 is extended in front of the zoo cage, detecting a state in which an outsider approaches an off-limits area other than a person concerned, or detecting a person approaching an automatic door. Can be.
[0113]
When the monitoring device 1 is installed vertically above the automatic door, the automatic door opening control is executed when a state is detected in which a person approaching the door enters the invisible curtain area and is walking in the door direction. Further, when the state where the person is stopped in the invisible curtain area is detected, the door is kept in the open state, so that unnecessary opening and closing operations can be prevented from being repeated. In addition, even if another person enters the mask area surrounding the invisible curtain area, the automatic door does not perform useless opening operation, so that the power consumption of the automatic door can be reduced.
[0114]
A method of detecting the hand 12a of the person 12 imaged in the invisible curtain area will be described with reference to the schematic plan view of FIG.
[0115]
The arithmetic unit 30 performs the scanning line extraction processing so as to distribute the images of the hands 12a of the person who has entered the invisible curtain area from the image information of the two CCD cameras 16 and 18 at appropriate intervals. The scanning lines 71 to 77 are extracted. Further, the distance to the floor surface when the hand 12a does not enter is preferably set as the reference distance.
[0116]
By setting the two CCD cameras 16 and 18 in this way, the distance between the CCD camera and the hand 12a is measured, and at the same time, the presence of the hand 12a and the floor surface are determined based on which scanning line detects the hand 12a. Height can be determined. Further, by acquiring the difference image of each CCD camera and extracting the image of the hand 12a on the captured image, the position information within the invisible curtain area can be obtained from the position of the extracted image of the hand 12a on the image. Can be obtained.
[0117]
In the image frame 80 at the left end in the figure, the image of the hand 12a captured at the first time is correlated, and displayed in correspondence with the scanning line. The hand 12a has passed through the outer edge 42 and has advanced toward the inner edge 43. This is an image at a stage when height information is obtained corresponding to the scanning lines of the first CCD camera 16 and the second CCD camera 18, respectively. The hand 12a is imaged, for example, on the first scanning line 71 and the second scanning line 72.
[0118]
An image frame 81 captured at a second time delayed by a predetermined time is shown on the right side of the image frame 80. Since the image of the hand 12a is formed on, for example, the first to fourth scanning lines 71 to 74, the hand 12a moves toward the inner edge 43 of the invisible curtain area.
[0119]
An image frame 82 captured at a third time delayed by a predetermined time is shown on the right side of the image frame 81. Since the image of the hand 12a is formed on, for example, the first to seventh scanning lines 71 to 77, the hand 12a further moves toward the inner edge 43 of the curtain area which is not visible. The arithmetic unit 30 detects such a change in the position of the hand 12 a moving toward the inner edge 43, and transmits a control signal to the notifying unit 32 to output a detection signal.
[0120]
An image frame 83 captured at a fourth time later than the image frame 82 by a predetermined time is shown. Since the image of the hand 12a is formed on, for example, the first to eighth scanning lines 71 to 78, it can be seen that the hand 12a is further moved toward the inner edge 43 of the curtain area which is not visible. The arithmetic unit 30 has already transmitted the control signal to output the detection signal to the notification unit 32. However, as another embodiment, at the time when the image frame 82 is arithmetically output, the processing enters the detection signal transmission preparation process, and Control may be performed so that the detection signal is output after the change of the position in which the hand 12a is moving in the direction of the inner edge 43 is confirmed again by calculating and outputting the frame 83.
[0121]
With reference to the schematic plan view of FIG. 9, a method of detecting the hand 12a of the person 12 imaged in the invisible curtain area will be further described. In the image frame 84 at the left end in the figure, the image of the hand 12a captured at the first time is correlated, and displayed in correspondence with the scanning line. The hand 12a has passed through the outer edge 42 and has advanced toward the inner edge 43. This is an image at a stage when height information is obtained corresponding to the scanning lines of the first CCD camera 16 and the second CCD camera 18, respectively. The hand 12a is imaged, for example, on the first scanning line 71 and the second scanning line 72.
[0122]
An image frame 85 taken at a second time delayed by a predetermined time is shown on the right side of the image frame 84. Since the image of the hand 12a is formed on, for example, the first to fourth scanning lines 71 to 74, the hand 12a moves toward the inner edge 43 of the invisible curtain area.
[0123]
An image frame 86 captured at a third time delayed by a predetermined time is shown on the right side of the image frame 85. The hand 12a is stopped at the same position as the image frame 85. At this stage, the arithmetic unit 30 is on standby so as not to output the detection signal.
[0124]
An image frame 87 captured at a fourth time delayed by a predetermined time is shown on the right side of the image frame 86. The hand 12a is stopped at the same position as the image frame 86. The arithmetic unit 30 stands by so as not to output the detection signal even at this stage, compares the pixel information on the search line of the image frame 87 with the pixel information of the subsequent image frame, and determines whether the hand 12a approaches the inner edge 43. Monitor whether it is.
[0125]
Here, a method of installing the two CCD cameras 16 and 18 will be described. First, the monitoring apparatus 1 sets the first CCD camera 16 so that the scanning line of the first CCD camera 16 and the scanning line of the second CCD camera 18 corresponding to the scanning line are substantially on the same plane. And the second CCD camera 18 are arranged.
[0126]
Further, the calculation unit 30 calculates a correlation value for a scan line that scans the hand 12a with the first CCD camera 16 and a scan line that scans the hand 12a with the second CCD camera 18. A corresponding scanning line selection process for determining the positional correspondence between the scanning line of the first CCD camera 16 and the scanning line of the second CCD camera 18 is executed so that the calculated correlation value is maximized, The first address of the scanning line is determined so that each address of the scanning line stored in the difference image storage unit 24 and the address of the scanning line stored in the second difference image storage unit 25 match. The head address is stored in the unit 24, the second difference image storage unit 25, or a head address storage area (not shown).
[0127]
Here, when the hand 12a forms an image on each scanning line of the two CCD cameras, the first CCD camera 16 and the second CCD camera 18 image the hand 12a from slightly different directions. So that an image that is almost unchanged forms on both scan lines.
[0128]
The arithmetic unit 30 receives stereo images from the two CCD cameras and selects, for example, any one of the scanning lines of the first CCD camera 16. Correlation processing is performed between the line signal of the selected scanning line and the line signal of the first scanning line of the second CCD camera 18.
[0129]
In the correlation processing by the arithmetic unit 30, the correlation processing with the line signal of the scanning line selected in the first CCD camera 16 is performed in order from the first scanning line of the second CCD camera 18 to the last line. . The correlation processing here is determined based on a correlation value that is the strength of the entire signal. Further, the correlation processing is repeatedly executed until the correlation value between the line signal of the scanning line selected in the first CCD 16 and the line signal of the n-th scanning line of the second CCD 18 becomes maximum. It may be configured as follows. n is a natural number of 1 to 500 when the number of scanning lines is 500, for example.
[0130]
Further, the arithmetic unit 30 executes the corresponding scanning line selection process, and determines that the line having the maximum correlation value by the correlation process is the corresponding scanning line, and determines the scanning line of the first CCD camera 16. And the scanning line of the second CCD camera 18 are determined. For example, when the arithmetic unit 30 determines that the fourth scanning line of the first CCD camera 16 and the seventh scanning line of the second CCD camera 18 are the corresponding scanning lines, both of them are determined. In the position correspondence relationship of the scanning lines of the CCD camera, the fifth scanning line of the first CCD camera 16 and the eighth scanning line 14 of the second CCD camera correspond to each other. The 100th scanning line corresponds to the 103rd scanning line of the second CCD camera 18.
[0131]
In addition, the monitoring device 1 determines the correlation between the same scan line of the first CCD camera 16 and the scan line of the second CCD camera 18, for example, the fourth scan line 13 of the first and second CCD cameras. The positional relationship may be determined so that the value becomes maximum. In this case, the same lines of the first CCD camera 16 and the second CCD camera 18 are set so that the scanning lines on the same plane correspond to each other, and the positional relationship is determined so that the correlation value becomes maximum. The two CCD cameras 16 and 18 can be installed efficiently and with high accuracy.
[0132]
After determining the positional relationship between the two CCD cameras 16 and 18 or the positional correspondence between the two scanning lines, the monitoring device 1 determines that the base line length w of the two CCD cameras 16 and 18 is smaller than the set value. Even in the case of deviation, the deviation in the scanning line direction can be corrected by measuring the predetermined distance A1. In this case, assuming that the position of the optical axis of the first CCD camera 16 and the position of the optical axis of the second CCD camera 18 are shifted by S in the scanning line direction, the above equation (1) is obtained.
a = {w} × {f / (d + S)} (4)
By doing so, the shift in the scanning line direction can be corrected.
[0133]
In this way, the monitoring device 1 can not only accurately measure the distance to the person's hand 12a, but also, for example, a floor pattern or the like even if the CCD camera slightly moves due to temperature change or temporal change. The accuracy can be maintained by measuring the predetermined distance A1 using. This correction may be repeatedly performed at predetermined intervals.
[0134]
In the above description, the case where the two CCD cameras 16 and 18 are installed so as to correspond to each other or the corresponding scanning line is selected has been described. It is also possible to use a fixed and fixed scanning line. In this way, the two CCD cameras 16 and 18 can be handled with the same feeling as a distance sensor, so that the installation of the two CCD cameras 16 and 18 becomes very simple.
[0135]
As described above, the monitoring apparatus 1 determines how large the person 12 or the hand 12a enters the invisible curtain area and what state of the person 12 or the hand 12a (in which position, standing, sitting, The person 12 can follow a series of movements such as whether the person 12 is moving, has stopped, or has left, with a simple device. In this case, since the difference from the reference distance is obtained, it can be used to determine the state even if the distance is relatively inaccurate.
[0136]
In addition, since the monitoring apparatus 1 uses the difference image, for example, if the person 12 stops moving, the area cannot be specified. However, a series of methods such as extending the interval of the image frames with time are used. The state of the person 12 can be determined comprehensively from the movement.
[0137]
Therefore, the presence of the person 12 that is stationary or slowly moving can be automatically monitored. For example, this is effective when the person 12 or the hand 12a that has entered the invisible curtain area has a small amount of movement.
[0138]
Furthermore, the distance to the floor or wall when the person 12 or the hand 12a does not exist in the invisible curtain area is set as a reference distance, and the change of the monitoring target from that state is followed. Thus, the state of the person 12 can be determined.
[0139]
According to the present embodiment as described above, the state of the person 12 or the hand 12a in the invisible curtain area is determined, and the person 12 or the hand 12a is moving in a predetermined direction. It is possible to accurately and very easily monitor that a computer has entered. In addition, since the information of the entire image frame is not acquired and processed from the CCD camera, complicated and large-capacity image processing is not required, and a relatively simple digital circuit or analog circuit, or a combination thereof, is extremely useful. An inexpensive device can be constructed. Further, the scanning lines to be extracted are of such an extent that the privacy of the person 12 and the hands 12a can be protected, which is very effective in monitoring the state in public facilities. Further, since the monitoring device 1 uses a wavelength of infrared light close to visible light, the monitoring device 1 can perform a monitoring activity without being alerted by an intruder.
[0140]
In the above embodiment, the monitoring device is configured using a CCD camera that receives at least visible light and infrared light and outputs an imaging signal. However, as another embodiment, for example, an infrared CCD camera (IRCCD) is used. In addition, it is possible to provide a monitoring device that captures infrared light during the day and night and monitors the state where the monitoring target 12 enters the monitoring areas 42 and 43. In this case, since the monitoring area can be imaged even if the amount of visible light decreases, the infrared irradiation device 17 can be omitted. The IR CCD also captures infrared light (heat source) radiated from the person 12 as the monitoring target, but since the light amount difference is small compared to the contrast between the person 12 and the background, the person 12 This does not hinder the detection of the boundary between the image and the background, and the edge detection processing of the person.
[0141]
It should be noted that the monitoring device and the monitoring method of the present invention are not limited to the illustrated example described above, and it is needless to say that various changes can be made without departing from the gist of the present invention.
[0142]
【The invention's effect】
As described above, according to the present invention, the infrared irradiation device 17 that irradiates the monitoring space 10 with infrared light in response to a predetermined threshold signal indicating a decrease in the amount of visible light in the monitoring space, and the infrared irradiation device 17 installed toward the monitoring space 10 A first imaging device 16 for imaging infrared light reflected from the monitoring target 12 at a first time and a second time delayed by a predetermined time from the first time; and a first imaging device. A second imaging device 18 that is installed toward the monitoring space 10 at a predetermined interval from the imaging device 16 and captures infrared light reflected from the monitoring target 12 at a first time and a second time; Difference image forming units 24 and 25 for forming a difference image between the image captured at the first time and the image captured at the second time, and a first image formed by the difference image forming units 24 and 25. Based on the difference image 24 of the image by the imaging device and the difference image 25 of the image by the second imaging device. The monitoring target 12 calculates the position of the monitoring target 12, and based on the calculation result of the position calculating unit 30, detects that the monitoring target 12 approaches the monitoring areas 42 and 43 and detects the detection signal. , Which measures the distance between a monitoring target that enters a three-dimensional monitoring space at night or in the dark and the imaging device, and monitors a monitoring target that approaches a predetermined area. There is an excellent effect of providing an apparatus and a monitoring method.
[Brief description of the drawings]
FIG. 1 is a schematic perspective view of a monitoring device according to an embodiment of the present invention.
FIG. 2 is a schematic plan view of a monitoring device according to an embodiment of the present invention.
FIG. 3 is a schematic perspective view of a monitoring device according to another embodiment of the present invention.
FIG. 4 is a schematic perspective view of an imaging device and an infrared irradiation device used in the embodiment of the present invention.
FIG. 5 is a schematic perspective view of another infrared irradiation device used in the embodiment of the present invention.
FIG. 6 is a flowchart of the monitoring device according to the embodiment of the present invention.
FIG. 7 is a block diagram of a monitoring device according to an embodiment of the present invention.
FIG. 8 is a schematic plan view of the embodiment of the present invention.
FIG. 9 is a schematic plan view of the embodiment of the present invention.
[Explanation of symbols]
1 Monitoring device
2 Control device
6 Illuminance measuring device
7 Infrared irradiation unit
8 Infrared irradiation device
10 Monitoring space
12 people
13 scanning line
14 scanning line
15 Security guard
16 CCD camera
17 infrared irradiation device
18 CCD camera
19 Interface for imaging device
20 ° imaging signal storage unit
21 Image signal storage
22 Image signal storage
23 Image signal storage
24 ° difference image storage
25 ° difference image storage
26 height storage
27 Moving distance storage
28 Mask address storage
29 illuminance measurement unit
30 arithmetic unit
32 Notification Department
38 Security object
42 monitoring area
50 ° mask area
57 curtain area
60 ° sensor control unit
62 ° difference image forming unit
64 ° correlation output operation unit

Claims (5)

A monitoring device that detects a monitoring target entering a three-dimensional monitoring space and outputs a detection signal;
An infrared irradiation device that irradiates the monitoring space with infrared light in response to a predetermined threshold signal indicating a decrease in the amount of visible light in the monitoring space;
A first imaging device installed toward the monitoring space and imaging infrared light reflected from the monitoring target at a first time and at a second time delayed by a predetermined time from the first time; When;
A second sensor that is set toward the monitoring space at a predetermined interval from the first imaging device and captures infrared light reflected from the monitoring target at the first time and the second time. Two imaging devices;
A difference image forming unit that forms a difference image between the image captured at the first time and the image captured at the second time;
Based on the difference image of the image by the first imaging device and the difference image of the image by the second imaging device formed by the difference image forming unit, A position calculation unit;
An approach notification unit that detects, based on a calculation result of the position calculation unit, that the monitoring target approaches a monitoring area and outputs the detection signal;
A monitoring device comprising: The monitoring device according to claim 1, further comprising an illuminance measurement device that detects visible light in the monitoring space. The monitoring device according to claim 1, wherein the position calculation unit detects a temporal change in a position of the monitoring target and calculates a moving direction of the monitoring target. A monitoring method for detecting a monitoring target approaching a three-dimensional monitoring space and outputting a detection signal;
An infrared irradiation step of measuring a visible light amount in the monitoring space and emitting infrared light at a stage when a predetermined undervisible light amount state is reached;
A monitoring object movement calculating step of forming a difference image based on a plurality of images of the monitoring space captured by the infrared light and calculating a height and a moving direction of the monitoring object;
An approach notification step of detecting the approach of the monitoring target to a monitoring area based on the calculation result of the movement calculation step and outputting the detection signal;
Measuring an amount of visible light in the monitoring space, and stopping irradiation of the infrared light when a predetermined amount of visible light is reached;
A monitoring method comprising: A monitoring method for detecting a monitoring target approaching a three-dimensional monitoring space and outputting a detection signal;
An infrared irradiation stopping step of measuring a visible light amount in the monitoring space and stopping emission of infrared light when a predetermined visible light amount state is reached;
A monitoring object movement calculating step of forming a difference image based on a plurality of images of the monitoring space captured by the visible light, and calculating a height and a moving direction of the monitoring object;
An approach notification step of detecting the approach of the monitoring target to a monitoring area based on the calculation result of the movement calculation step and outputting the detection signal;
A monitoring method comprising:

Images