JP4287647B2 - Environmental status monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an environment situation grasping apparatus suitable for grasping a moving object in a predetermined monitoring area and a situation change of the monitoring area.
[0002]
[Related background]
When monitoring an intruder in a predetermined monitoring area or grasping the situation, a monitoring camera is generally used conventionally. However, a camera image obtained using a surveillance camera is generally easily affected by the weather (meteorology) and is difficult to monitor at night.
[0003]
On the other hand, recently, it has been proposed to monitor an intruder or the like in a predetermined monitoring area using a laser radar (see, for example, Patent Documents 1 and 2). According to the monitoring system using this laser radar, object detection can be performed without being affected by the weather (meteorology) or the like.
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 11-227608 [Patent Document 2]
Japanese Patent Laid-Open No. 11-352245
[Problems to be solved by the invention]
However, although the above-described monitoring system using a laser radar can accurately detect the spatial coordinates (distance and azimuth from the monitoring point) of a moving object or the like that has entered the monitoring area, it generally knows what the object is. Difficult to do. In this regard, in the case of an image obtained by the monitoring camera described above, the camera image itself is in accordance with human visual sensibility, and therefore it is possible to easily determine what each object is from the image information. However, distance information to the detection target cannot be obtained from the camera image unless a stereo camera or the like is used, for example.
[0006]
The present invention has been made in consideration of such circumstances, and an object thereof is to provide an environmental condition grasping device that can easily obtain information on a moving object or the like in a predetermined monitoring area together with its spatial position information. There is to do.
[0007]
[Means for Solving the Problems]
In order to achieve the above-described object, an environmental condition grasping apparatus according to claim 1 of the present invention includes a laser radar that scans a predetermined monitoring area and a radar that creates a radar image that visualizes distance information detected by the laser radar. An image creating means, and a monitoring camera provided at a position that can be regarded as the same as the installation position of the laser radar and imaging a monitoring area of the laser radar;
In particular, plane recognition means for detecting a surface of an invariant object in the monitoring area from the radar image, and a plurality of feature points associated with each other between the camera image captured by the monitoring camera and the radar image Image matching means for superimposing the coordinate system of the camera image and the coordinate system of the radar image based on the position coordinates of the image, and a plane recognized by the plane recognition means on the basis of the image matching result. An image display means for displaying a line or polygon diagram on the camera image, a moving object recognition means for recognizing a moving object in the monitoring area from the laser radar image, and a movement recognized by the moving object recognition means Auxiliary information display means for superimposing and displaying auxiliary information for specifying an object on the camera image is provided.
[0008]
Preferably, the radar image creating means is configured to create a radar image obtained by visualizing three-dimensional information by performing grayscale conversion or color conversion on the detected distance information. Further, the image matching means superimposes the coordinate system of the camera image and the coordinate system of the radar image based on the position coordinates of a plurality of feature points mutually associated between the camera image and the radar image. Composed.
[0009]
According to the environmental situation grasping device configured as described above, since the camera image and the radar image can be associated with each other, for example, while easily identifying each object in the monitoring target region from the camera image, Spatial coordinates (distance and direction from the monitoring point) can be easily and accurately obtained from the radar image. Conversely, even when the camera image is unclear due to the influence of the weather or the like, the unclearness can be compensated for by the radar image, so that it is possible to accurately grasp the monitoring target area.
[0010]
Preferably , a marker surrounding the recognized moving object may be used as auxiliary information for specifying the moving object to be displayed superimposed on the camera image .
[0011]
According to the environmental situation grasping device configured as described above, by displaying the auxiliary information for identifying the object surface and / or the moving object from the laser radar image, for example, the contour line and the identification mark, superimposed on the camera image, It is possible to clearly specify an invariant fixed structure in the monitoring target area, and to easily identify and display an object of interest in the monitoring target area from other objects. As a result, it is possible to accurately grasp various situations in the monitoring target area without being influenced by the weather or the like.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an environment situation grasping device according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a schematic configuration of an environmental condition grasping apparatus according to this embodiment, in which 10 is a laser radar that scans a predetermined monitoring target area, and 20 is provided in the vicinity of the laser radar 10 to monitor the monitoring. It is a surveillance camera (for example, a TV camera) that images a target area. The laser radar 10 and the monitoring camera 20 are housed integrally in a single casing 30 to form a sensor unit, and are installed at a monitoring point of a predetermined height such as a wall surface of a building or a utility pole. It is provided so as to overlook a certain ground surface. That is, the laser radar 10 and the monitoring camera 20 are provided so as to respectively view the same monitoring target area from monitoring points that can be regarded as substantially the same.
[0013]
Incidentally, the laser radar 10 is provided, for example, with a main scanning motor 12 that rotates the polyhedral mirror 11 at a constant speed, and a rotation axis of the polyhedral mirror 11 that is provided so as to be orthogonal to the main scanning motor 12. And a sub-scanning motor 13 that swings the rotating surface of the polyhedral mirror 11 at a predetermined speed by being tilted within the angle range. By irradiating the polyhedral mirror 11 with laser pulse light emitted from the laser light source 14 via the half mirror 15, the laser pulse light is applied to a predetermined monitoring region according to the rotation and tilt of the polyhedral mirror 11. The reflected light from the monitoring region of the laser pulse light is received and detected by the light receiver 17 from the polyhedral mirror 11 through the condenser lens 16.
[0014]
That is, the laser radar 10 scans the scanning surface in the sub-scanning direction (y direction) while scanning the irradiation direction of the laser pulse light at high speed in the main scanning direction (x direction) by rotating and swinging the polyhedral mirror 11. ), Thereby scanning the entire area of the predetermined monitoring target area as illustrated in FIG. Then, the reflected light from various objects existing in the monitoring target area is received in synchronization with the irradiation of the laser pulse light, and from the light reception timing (the elapsed time from the laser pulse light irradiation timing to the reflected light reception timing). The distance information to the object (reflection point) is obtained.
[0015]
The radar image creating means 40 for creating a radar image in which distance information detected using such a laser radar 10 is visualized is realized as a part of the function of the monitoring apparatus main body 50 composed of a personal computer, for example. Specifically, the radar image creating means 40 receives the synchronization signal from the laser pulse scanning circuit 41 that drives the laser light source 14 as shown in FIG. 3, for example, and receives the reflected light by the object of the laser pulse light. The reflected light detection unit 42 that measures the distance, the distance information conversion unit 43 that converts the measurement time into distance information, and the gray level conversion unit 44 that converts the distance information into gray level information. Of course, it is possible to directly convert the measurement time into grayscale information. In this embodiment, an example is shown in which distance information is converted into light and dark information whose density increases with distance, for example, but tone conversion in which the color changes stepwise according to the distance may be performed.
[0016]
In the laser radar image, the grayscale information (distance information) obtained as described above is sequentially mapped onto a predetermined image memory 45 in accordance with the main scanning angle and subscanning angle information obtained from the laser pulse scanning circuit 41. It is created by. That is, the density information (distance information) obtained at that time is written in the coordinates (address of the image memory 45) specified by the main scanning angle and the sub-scanning angle indicating the scanning direction of the laser pulse light. A laser radar image that three-dimensionally identifies the reflection point of the laser pulse light on the above-described monitoring target region on 45 is created.
[0017]
The environmental condition grasping device according to the present invention is characterized by the coordinates of the laser radar image A of the monitoring target area obtained as described above and the monitoring target area imaged using the monitoring camera 20 described above. The image matching means 60 is provided for associating the coordinates of the camera image B with each other. This image matching means 60 is also realized as a part of the function of the monitoring apparatus main body 50 described above. The image matching means 60 is an initial setting process when the sensor unit 30 including the laser radar 10 and the monitoring camera 20 is installed so as to view a predetermined monitoring target area on the above-described power pole or the like. It is activated.
[0018]
Specifically, the image matching unit 60 displays the laser radar image A and the camera image B on the monitor (display) 70 provided in the monitoring apparatus main body 50, and in the monitoring target area in each of these images A and B. When the representative feature points corresponding to each other are designated over a plurality of points, the coordinate system of the laser radar image A and the coordinate system of the camera image B are mutually set according to the position coordinates of the designated feature points. Execute the association process.
[0019]
That is, as shown in FIG. 4, the coordinates indicating the positions of a plurality of objects that can be clearly identified in the laser radar image A are designated as the representative feature points a1, a2, a3,. Coordinates indicating the positions of the objects in the image B are designated as feature points b1, b2, b3,... Corresponding to the feature points a1, a2, a3,. For these feature points a1, a2, a3,... (B1, b2, b3,...), For example, based on the experience of the operator and the image recognition sensation, The points may be selectively designated as being at a short distance from the monitoring point and as being at a long distance from the monitoring point. Then, the coordinate system of one image (laser radar image A or camera image B) is transformed so that the coordinates of these designated feature points a1, a2, a3,... (B1, b2, b3,...) Overlap each other. Thus, the laser radar image A and the camera image B may be superimposed on each other.
[0020]
More specifically, as shown in FIG. 5, first, the coordinates of an object that is vertical in the laser radar image A, specifically, the horizontal dot position and the vertical dot position in the laser radar image A are obtained. Each of the four locations is designated <Step S1>. Next, in the camera image B, four coordinates (horizontal dot position and vertical dot position) indicating the same part of the same object as the laser radar image A are designated <step S2>. Then, the gain (spatial slope comparison value) of each of the images A and B is obtained from the plurality of designated coordinates (step S3).
[0021]
Next, the horizontal dot difference and the vertical dot difference indicating the deviation between the coordinates so that the coordinates of the specific part of the object in the laser radar image A coincide with the coordinates of the same part of the same object in the camera image B. The offset indicated by is input <step S4>. And based on the gain and offset, for example
[Laser radar screen coordinates] = [Camera screen coordinates] / [Gain]-[Offset]
Then, the coordinate system of the laser radar image A is matched with the coordinate system of the camera image B <Step S5>.
[0022]
Thus, if the coordinate system of the laser radar image A and the coordinate system of the camera image B are matched in this way, the camera image B that can generally easily grasp the state of the imaging target region during the subsequent monitoring operation. It is possible to assign distance information indicated by the laser radar image A to each of the components. In the camera image B, it is difficult to identify at a glance, for example, even if there are a plurality of similar-colored objects that overlap with each other, or due to the influence of weather conditions, Even when it is difficult to identify each object, it is possible to clearly identify each object according to the distance information obtained from the laser radar image A. Therefore, while making up for the disadvantages of the laser radar image A and the camera image B, and taking advantage of the advantages of the laser radar image A and the camera image B, respectively, the situation of moving objects and the like in a predetermined monitoring target area can be easily achieved. Moreover, it becomes possible to accurately grasp.
[0023]
By the way, in addition to the image matching processing means 60 described above, the environmental condition grasping apparatus according to the present invention further includes an object recognition means 80 for recognizing a surface of a fixed object or a moving object in the monitoring target area from the laser radar image A. ing. Specifically, as shown in FIG. 5, the object recognition unit 80 includes a plane recognition processing unit 81 that recognizes a surface constituting a fixed object from information on a group of fixed objects in the monitoring target area, and the monitoring target area. The moving object detection processing unit 82 recognizes a moving object from information whose detection coordinates change for each scan.
[0024]
In other words, the plane recognition processing unit 81 examines the direction of the arrangement of spatial coordinates that are continuously connected in the laser radar image A, for example, so that the surface of a fixed structure such as a road surface or a building that is invariable in the monitoring target area. It has a recognition function. The result recognized by the plane recognition processing unit 81 is displayed on the laser radar image A or the camera image B as auxiliary information such as a contour line or a polygon diagram specifying the surface, for example. Further, as described above, the moving object detection processing unit 82 recognizes that the object is a moving object such as a person or a car from the information of the object whose detection coordinates sequentially change for each scan of the laser radar. A function of displaying a rectangular marker or the like surrounding a moving object as auxiliary information on the laser radar image A or the camera image B is provided. Such auxiliary information may be selectively displayed under an operator's instruction or may be displayed only for a moving object to be noticed.
[0025]
Thus, according to the environmental situation grasping device configured to include such an object recognition unit 80 and the above-described image matching unit 60, the monitor image subjected to the image matching process is displayed on the monitor 70, and the status of a predetermined monitoring target region is displayed. For example, as shown in FIGS. 6A to 6D, representative display examples appropriately display information necessary for grasping the situation according to the environment (situation) of the monitoring target area. It can be used for monitoring.
[0026]
That is, since the monitoring image is obtained by associating the coordinates of the camera image B and the coordinates of the laser radar image A as described above, for example, as shown in FIG. 6A, the pointer is used by using a coordinate input device such as a mouse. It is possible to display the spatial position coordinates of the designated part, specifically, the distance from the monitoring point, only by moving P and specifying the point of interest on the monitoring image. When a moving object is detected, for example, as shown in FIG. 6B, a marker (auxiliary information) M surrounding the recognized moving object is displayed on the monitoring image so as to be noticed. Objects can be clearly identified and displayed.
[0027]
Furthermore, when it is desired to recognize the size and shape of a certain object, for example, as shown in FIG. 6C, the plane recognition described above can be performed by specifying a plurality of points indicating the corners of the object of interest. Since it is possible to grasp the size and shape of the main surface of the object by processing, it is possible to easily estimate the three-dimensional size of the object from this recognition result and easily find its existence position. It becomes possible.
[0028]
Further, if a polygon diagram (auxiliary information) for specifying each plane of the object obtained by the above-described plane recognition processing is displayed superimposed on the monitoring image as shown in FIG. Even when the camera image B cannot be clearly obtained due to the influence of the conditions, it is possible to grasp the outline of the monitoring target area from the polygon diagram (auxiliary information). As a result, it is possible to easily and accurately grasp the status of the monitoring target area without being influenced by the monitoring time zone, weather conditions, or the like.
[0029]
The present invention is not limited to the embodiment described above. Here, an example of grasping the situation of a predetermined monitoring target area outdoors has been shown, but for example, by using the monitoring image shown in FIG. 6A, a camera having a ranging function of each part can be realized, It can be applied to a ship monitoring camera, a robot eye, a distance measuring device, and the like. In addition, by using the monitoring image shown in FIG. 6 (b), a monitoring system covering a wide range of areas such as outdoors is constructed, and an alarm is triggered by detection of a moving object, and the monitoring image is recorded. Can be taken. Furthermore, it can be applied to a traffic volume grasping system at an intersection or the like, a crossing pedestrian grasping system, or a tracking system for a moving body.
[0030]
Further, by using the image shown in FIG. 6C, it is possible to apply to positioning of a container crane, size determination processing of various loads flowing on a belt conveyor, and the like. Furthermore, by using the image shown in FIG. 6 (d), it is possible to perform unevenness determination and topographic surveying of various objects. In addition, the present invention can be variously modified and implemented without departing from the scope of the invention, and can be applied to various object recognition fields.
[0031]
【The invention's effect】
As described above, according to the present invention, the coordinates of the radar image obtained from the laser radar and the coordinates of the camera image captured by the monitoring camera provided in the vicinity of the laser radar are associated with each other and the predetermined monitoring target. Since it is used for the area monitoring process, it is possible to easily and stably monitor the situation of the monitoring target area without being affected by the weather conditions and the monitoring time zone. In addition, the situation can be accurately monitored while obtaining the distance information to the attention point.
[0032]
Furthermore, since information on the object surface recognized from the laser radar image and information on the moving object are displayed as auxiliary information superimposed on the monitoring image, the situation can be grasped more appropriately. A great effect is produced.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of an environmental situation grasping apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram showing a scanning form of a monitoring target area by a laser radar.
FIG. 3 is a block diagram showing a schematic creation processing means for a laser radar image.
FIG. 4 is a diagram showing a concept of matching processing between a laser radar image A and a camera image B.
FIG. 5 is a diagram showing an example of a specific processing procedure of image matching processing.
6 is a diagram showing a typical display example of a monitoring image in the environmental situation grasping apparatus shown in FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Laser radar 20 Monitoring camera 40 Laser radar image preparation means 50 Monitoring apparatus main body 60 Image matching means 70 Monitor (display)
80 Object recognition means 81 Plane recognition processing unit 82 Moving object detection processing unit A Laser radar image B Camera image

Claims (2)

A laser radar that scans a predetermined monitoring area;
Radar image creating means for creating a radar image in which distance information detected by the laser radar is visualized;
Plane recognition means for detecting a surface of an invariant object in the monitoring area from the radar image;
A monitoring camera that images the monitoring area of the laser radar provided at a position that can be regarded as the same as the installation position of the laser radar;
Based on the position coordinates of a plurality of feature points associated with each other between the camera image captured by the monitoring camera and the radar image, the coordinate system of the camera image and the coordinate system of the radar image are superimposed. an image matching means to match,
The recognized plane in the previous SL plane recognizing means based on the image matching result, image display means for displaying superimposed on the camera image as a contour or polygon diagrams,
Moving object recognition means for recognizing a moving object in the monitoring area from the laser radar image;
An environmental condition monitoring apparatus comprising: auxiliary information display means for displaying auxiliary information for identifying a moving object recognized by the moving object recognition means so as to overlap the camera image. The environmental condition monitoring apparatus according to claim 1, wherein the auxiliary information for specifying a moving object to be displayed superimposed on the camera image includes a marker surrounding the recognized moving object.

Images