JPS62264390A - Visual sense recognizing device for supervisory robot - Google Patents

Abstract

PURPOSE:To clearly judge whether or not a human being invades into an invading prohibiting area even under the high temperature environment by adopting an image recognizing processing system based upon the comparison of the line drawing of an observing object and a model line drawing. CONSTITUTION:Prior to supervising, the variable density image of an observing object at the time of normality is obtained by an image pickup part and the line drawing of the variable density image is prepared and filed as a model line drawing at a line drawing preparing part 2. At the time of the supervising, the variable density image of the supervisory object is obtained in such a same way, after the line drawing is executed for this by the line drawing preparing part 2, it is compared with the model line drawing and the line drawing part is extracted which is not included in the model line drawing. Next, in a stereoscopic vision part 1, the stereoscopic vision is executed for respective points on the observing object loaded to the line drawing part extracted by the comparing part 3, three-dimensional coordinates are observed, and an area discriminating part 4 discriminates whether or not the three-dimensional coordinates are included in the data area to specify an invading prohibiting area.

Description

[Detailed Description of the Invention] <Industrial Application Field> The present invention relates to a monitoring robot, such as a vbFX robot, that patrols a building to detect the presence of illegal intruders and suspicious objects. The present invention relates to a visual recognition device used for the vision of this type of monitoring robot.

<Prior Art> Conventionally, as this type of visual recognition device, a structure in which various sensors such as an ultrasonic sensor, a microwave sensor, a proximity switch, an infrared sensor, etc. are appropriately arranged around the head or torso has been proposed. (Nikkei Mechanical September 1985 issue). This visual recognition device uses an ultrasonic sensor to detect objects.

A microwave sensor and a proximity switch function, as well as an infrared sensor to detect intruders.

<Problems to be Solved by the Invention> The above-mentioned infrared sensor distinguishes between humans and objects constituting the surrounding environment based on temperature differences, so it becomes difficult to use it in environments where the ambient temperature (air temperature) is high. Furthermore, even if a person could be detected, it would be impossible to determine whether the person is an illegal trespasser, or in other words, whether the person's location is within the prohibited area.

This invention eliminates the conventional method using infrared sensors and adopts a new image recognition processing method based on comparing the line drawing of the observation target with a model line drawing.This invention can be used even in high-temperature environments, and can be used in areas where humans are prohibited from entering. An object of the present invention is to provide a visual recognition device for a surveillance robot that can clearly determine whether or not a robot has invaded.

Means for Solving the Problems〉 The structure of the present invention that achieves the above object will be explained using FIGS. 1 and 2, which correspond to one embodiment. In this invention, the observation target is An imaging unit such as a television camera 31 that captures an image and generates a grayscale image thereof; A line drawing creation unit 2 that performs line drawing processing on the grayscale image to create a line drawing of the observation target; Comparing the line drawing of the observation target with a model line drawing. a comparison unit 3 that extracts line drawing parts that are not included in the model line drawing, and a stereoscopic viewing unit 1 that stereoscopically views each point of the observation target related to the line drawing part extracted by the comparison unit and measures its three-dimensional coordinates. A visual recognition device for a monitoring robot is composed of the following: and an area discriminating unit 4 that discriminates whether or not the three-dimensional coordinates of each measured point are included in a predetermined prohibited area.

Function> First, prior to monitoring, the imaging unit obtains a grayscale image of the observation target under normal conditions (specifically, the scenery of the monitoring area including the prohibited area), and the line drawing of this l@light image is sent to the line drawing creation unit. 2 and file it as a model line drawing in advance.

During monitoring, a grayscale image of the observation target is similarly obtained by the imaging unit, and after this is converted into a line drawing by the line drawing creation unit 2, this line drawing (this is referred to as an “observation line drawing”) is compared with the model line drawing by the comparison unit 3. and compare them to extract line drawing parts that are not included in the model line drawing. Next, the stereoscopic viewing unit 1 stereoscopically views each point on the observation target related to the line drawing portion extracted by the comparing unit 3 and measures its three-dimensional coordinates, and the area determining unit 4 measures the three-dimensional coordinates.
It is determined whether the dimensional coordinates are included in a predetermined prohibited area, that is, a data area that defines a prohibited area.

Therefore, according to this method, it is possible to detect a person even in a high temperature environment, and it is also possible to clearly determine whether or not a person has entered the prohibited area.

<Embodiment> FIG. 3 shows a monitoring robot according to an embodiment of the present invention.

The illustrated surveillance robot is a mobile type security robot that patrols inside a building to detect the presence of illegal intruders and suspicious objects; however, the present invention is not limited to this, and the robot may be installed toward a specific surveillance area. It can also be applied to fixed type monitoring robots.

The robot in Fig. 3 has a moving mechanism 11 (+i)
It has a structure in which a body part 13 is installed vertically on the ground, and a head part 14 is rotatably mounted on the body part 13. A speaker 15 is installed in the corner 13, and a communication antenna 16 is installed in the head 14.
and a pair of optics 17, 18, respectively.
Inside each visual section 17, 18 there is a first section, which will be described later. Each second observation system 34, 36 is in a corresponding position.

A mark 9 for measuring the current position of the robot is placed on the floor surface on which the robot runs, and the robot reads this mark 19 with a mark reading device (not shown) installed on the underside of the body 12. ) while moving along a predetermined path.

FIG. 1 shows an example of the overall configuration of a visual recognition device incorporated into the robot.

The figure-neutral stereoscopic viewing unit t closely images the observation target (specifically, the scenery of the monitoring area including the prohibited area) and generates a gray scale image of the observation target, and also generates a gray scale image of each point on the observation target (hereinafter referred to as "object"). The three-dimensional coordinates of a point are measured.

The line drawing creation unit 2 performs line drawing processing on the gray scale image to create a line drawing of the observation target. This line drawing includes a line drawing (this is called a "model line drawing") obtained from the observation target during normal operation prior to monitoring.
) and line drawings (referred to as "observation line drawings") obtained from the observation target during monitoring, and the number of model line drawings corresponding to the observation target is stored in the line drawing file 22.

Figure 5 (1) shows an example of a model line drawing, and Figure 5 (2)
shows an example of an observation line drawing. In the illustrated model line drawing, only the wall portion a that defines the no-trespassing area appears, whereas in the observation line drawing, there is a vehicle outside the no-trespassing area, but a trespasser C is inside the no-trespassing area. Each is appearing. Furthermore, the fifth
In Figure (2), each point that makes up the car and the trespasser C means new points that are not included in the model line drawing (these are called "new feature points"), and correspond to these new feature points. The three-dimensional coordinates of the object point are measured by the stereoscopic viewing unit 1.

Next, the comparison unit 3 compares the observed line drawing with the model line drawing,
This is for extracting line drawing parts that are not included in the model line drawing (in the example of FIG. 5, the line drawing parts for the car and the illegal trespasser C).

The model management unit 21 is a part that stores and retrieves the model line drawing from the line drawing file 22.

The area determination unit 4 compares the three-dimensional coordinates of the object point measured by the stereoscopic vision unit 1 with the determination standard data defining the no-entry area, and determines whether the winter object point is located within the no-entry area. Determine whether or not. Note that the judgment standard data is set in advance in the judgment standard setting section 23.

The monitoring data control unit 24 manually inputs information regarding the current robot position and direction, retrieves a model line drawing according to the human power situation u via the model management unit 21, and also outputs predetermined judgment standard data according to the input information. Judgment criteria setting section 23
The data is output to the area discriminating unit 4.

FIG. 2 shows an example of the configuration of the stereoscopic viewing section 1.

The illustrated example is a television camera 31. Half mirror 32
It also includes a first observation system 34 including a laser scanner 33, a second observation system 36 including a PSD camera 35, and a triangulation control section 37 electrically connected to these observation systems 34 and 36, respectively.

The television camera 31 includes a lens 38 and an area sensor 39 made of a two-dimensional COD or the like, and a grayscale image of the observation target 40 is generated on the imaging surface of the two-dimensional area sensor 39. The half mirror 32 is arranged at an intermediate position between the television camera 31 and the observation object 40, and transmits the light from the observation object 40, and also reflects the laser beam 41 output from the laser scanner 33 toward the observation object 40 side. It is something. The laser scanner 33 is for scanning a laser beam 41 and projecting it onto the (bead) point of the half mirror 32, and the reflected light 44 is directed to a predetermined point on the observation target 40, as will be described later. To the object point!11r1.It is emitted to the light point P
is now generated.

The PSD camera 35 includes a lens 42 and a semi-solid optical device detector 43 (Po5ition 5 sensitive).

The light point P is imaged on the detection surface of the PSD 43 to generate an image point W. This PSD 43 detects the coordinates of the image point W on the detection surface, converts this into an analog quantity such as voltage, and outputs it to the triangulation control 1 section 37.

The triangulation unit 37 is a device including a microcomputer, and performs various controls and calculations such as operation control of the laser skier/ner 33: JIl and calculation of three-dimensional coordinates of an object point by triangulation. An operation of sequentially specifying image points Q on the area sensor 39 is performed based on the new feature point data.

By the way, in the first observation system 34, when the image point Q is specified by the triangulation control section 37, the laser scanner 33 is adjusted so that the helaser light 41 hits the point on the half mirror 32 corresponding to this image point Q. The amount of scanning is controlled.

Also, TV camera 31. The half mirror 32 and the laser scanner 33 are arranged with a predetermined positional relationship, so that the reflected light 44 at the point is irradiated to the object point corresponding to the image point Q, and the light point P is configured to be generated.

That is, the center point of the half mirror 32 is now 0, and the lens 38 is
, and the scanning center point of the laser scanner 33 is S, the television camera 31 . Half mirror 32 and laser scanner 3
3 is determined. Furthermore, line segments OL and OS
When the X-coordinate axis and the X-coordinate axis are set for , the half mirror 32 maintains its posture so that it is inclined at 45 degrees with respect to these coordinate axes. With this configuration, the half mirror corresponding to the image point Q on the area sensor 39
The point K on 32, that is, the visual %iQL is the half mirror 32
When the laser beam 41 is applied to the intersection point, the optical path of the reflected light 44 at point K coincides with the visual field ``41AQL'', and it is irradiated to the object point on the observation target 40 corresponding to the image point Q. A light point P is generated.Therefore, a table is created showing which point on the half mirror 32 each point on the area sensor 39 corresponds to, and this is calculated by triangulation system?111
By storing the image point Q in the memory in the section 37 in advance, each time the image point Q on the area sensor 39 is specified, the image point Q
The point K on the half mirror 32 corresponding to the point K can be automatically selected. Then, if the scanning direction of the laser scanner 33 is automatically controlled by the triangulation side'
Reflected light 44 whose optical path coincides with L can be easily generated.

Figure 4 shows that if the above positional relationship is set, the half mirror 3
FIG. 2 is an explanatory diagram for proving that the optical path of reflected light 44 at a point on point 2 coincides with visual field 4iQL.

In the figure, the center point of the lens 38 of the television camera 31 is located at a distance of 10 in the X coordinate axis direction from the center +'4Q (corresponding to the coordinate origin) of the half mirror 32, and the area sensor 39 captures and images The scanning center point S of the laser scanner 33 is located at a distance q0 in which the surfaces are in the same direction.
is the distance l in the direction of the X coordinate axis. Assuming that they are located at the respective positions, the equation of the imaging plane of the Eri-7 sensor 39 is
=Qo, the equation of half mirror 32 is y=)(.

In addition to the xy coordinate axes, if we assume two coordinate axes at the center point 0 of the half mirror 32, the coordinates of the center point of the lens 38 are (0, β., 0), and the laser scanner 3
The coordinates of the scanning center point S of No. 3 are <1. , O, O) ~ Furthermore, the coordinates of the image point Q with the Eli-7 sensor 39 are (q
x −Qo −O2), the equation of the line of sight QL is as shown in the following equation 0.

Therefore, if the spatial coordinates of the intersection between the visual vAQL and the half mirror 32 are (KQ, K, , K, .), then the following equation 0 to 0 is obtained.

Next, if the vector LK along the line of sight is LK, the vector KS along the optical path of the laser beam 41 is KS, and the unit vectors along the x and y coordinate axes are i and j, then from the above equations 0 to 0, the following relational expression is obtained. 〇 is guided.

L K + K S = io (t J )・
−m−・■ In the above equation, (i − j) on the right side corresponds to the normal vector n of the half mirror 32, and from this equation 0, LK + KS is on the same plane as the normal vector n of the half mirror 32. It can be seen that this normal vector n is the axis of symmetry. Therefore, the laser beam 41 irradiated onto the rehalf mirror 32 from the scanning center point S of the laser scanner 33 is reflected by the half mirror 32 along the line of sight toward the object point.

Next, the operation of the visual recognition device having the above configuration example will be explained.

First, prior to monitoring, the robot is guided to predetermined monitoring positions one after another, the observation target at each monitoring position is imaged by the television camera 31 of the stereoscopic viewing unit 1, and the grayscale image is displayed on the imaging surface of the area sensor 39. generate. This T-comic image is sent to the line drawing creation section 2 and converted into a line drawing, and the model management section 21 stores this line drawing in the line drawing file 22 as a model line drawing.

During monitoring, the robot is rotated along a predetermined route, and when it reaches a predetermined monitoring position, the robot is stopped and set and held in a predetermined posture and orientation, and then the observation target is captured by the TV camera 1 at each monitoring position. , and obtain a grayscale image in the same manner as described above. After this grayscale image is converted into a line drawing by the line drawing creation section 2, this observed line drawing is sent to the comparison section 3.

On the other hand, the monitoring data control section 24 controls the operation of the model management section 21 to take out a model line drawing corresponding to the current monitoring position and send it to the comparison section 3. Then, the comparison unit 3 compares the observed line drawing with the model line drawing, extracts line drawing parts that are not included in the model line drawing, and extracts each point (referred to as "new feature point") that constitutes the line drawing part. and outputs it to the stereoscopic viewing section 1.

The stereoscopic viewing unit 1 specifies the image point Q on the area sensor 39 corresponding to the new feature point, and displays the corresponding half mirror.
The point K on the laser beam 32 is stored in a memory in advance and determined by referring to a table, and the scanning amount of the laser beam 33 is controlled so that the laser beam 41 hits that point.

Next, the laser scanner 33 is activated to emit laser light 41.
When output, the laser beam 41 hits the point K on the half mirror 32 and is reflected toward the viewing object 40, and the reflected light 44 reaches the object point corresponding to the image point Q along the line of sight QL and becomes a light point P. generate. This light point P is imaged on the optical device detector 33 of the PSD camera 35, and the coordinates where the image point W is located are outputted to the triangulation control section 37 as an analog quantity.

Next, the triangulation control unit 37 calculates the direct vAQ L from the point where the center point of each lens 38.
and the straight line WR, and then calculate the three-dimensional coordinates of the intersection (light point P) of these two straight lines based on the principle of triangulation.

These three-dimensional coordinates are sent to the area determination unit 4, where they are compared in size with criteria and quasi-data that define the prohibited area at the monitoring position, and whether or not the light point P is located within the prohibited area. Determine.

By sequentially repeating the same process as above for other new feature points, the three-dimensional coordinates of all new feature points are determined, and the object point (light point) related to each new feature point is placed in the prohibited area. (determine whether or not it is located within).

As a result, when it is determined that the object point is located within the no-trespassing area, the area determination unit 4 activates an alarm means (not shown) to generate an alarm sound from the speaker 15, or activates a communication means (not shown). 1) to output an abnormal signal from the antenna 16.

<Effects of the Invention> Since the present invention is configured as described above, unlike infrared sensors, there is no risk of the sensor becoming unusable in a high temperature environment, and moreover, it can detect whether or not a person has entered a prohibited area. It has a remarkable effect of achieving the purpose of the invention, such as being able to make a clear judgment.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a visual recognition device according to an embodiment of the present invention, FIG. 2 is an explanatory diagram showing an example of the configuration of a stereoscopic viewing section, FIG. 3 is an external view of a security robot, and FIG. FIG. 5 is an explanatory diagram showing the principle of the stereoscopic viewing section in the figure, and FIG. 5 is an explanatory diagram showing a model line drawing and an observation line drawing.

Claims (4)

[Claims] (1) A monitoring robot that monitors an observation target and determines whether there is an abnormality in the observation target includes an imaging unit that images the observation target and generates a grayscale image thereof, and a line drawing of the observation target that processes the grayscale image. a line drawing creation unit that compares the line drawing of the observation target with a model line drawing and extracts line drawing parts that are not included in the model line drawing, and each point on the observation target that corresponds to the line drawing part extracted by the comparison unit. A monitoring device comprising: a stereoscopic viewing unit that views the object stereoscopically and measures its three-dimensional coordinates; and an area determination unit that determines whether the three-dimensional coordinates of each measured point are included in a predetermined prohibited area. Visual recognition device for robots. (2) The visual recognition device for a monitoring robot according to claim 1, wherein the monitoring robot is a movable security robot that patrols inside a building. (3) The visual recognition device for a monitoring robot according to claim 1, wherein the stereoscopic viewing unit includes an imaging unit in its configuration. (4) The stereoscopic viewing unit includes an imaging device for generating a grayscale image of the observation target on an imaging surface, a half mirror disposed at an intermediate position between the observation target and the imaging device, and a half mirror disposed on the half mirror. A light projection device that emits light to a point and irradiates the reflected light from a half mirror to a point on an observation target to generate a light point, and a light that images the light point on a detection surface and determines the coordinates of the image point. Claims comprising: a device detector; and an image processing device that calculates the three-dimensional coordinates of a point on the observation target based on the image point position coordinates on the imaging surface and the image point position coordinates on the detection surface. The visual recognition device for a monitoring robot according to item 1 or 3.