JPH06253312A - Space recognition device - Google Patents

Abstract

PURPOSE:To securely discriminate and recognize changes of plural objects to be monitored in a depth direction by representing the three dimensional coordinate position of an abnormality occurrence point in a monitor space as an internal or external ratio to the segments connecting two monitor points in the monitor space. CONSTITUTION:While a monitor image PIC1 is obtained by a 1st television camera TV1, an object to be monitored in the monitor space monitored through the image PIC1 is picked up by a 2nd television camera TV2 arranged at a different position from the camera TV1 to obtain a 2nd monitor image PIC2. In this case, two of plural reference points are merely selected by using plane image information obtained from the images PIC1 and PIC2 obtained from the cameras TV1 and TV2 and a point is specified on the segments between the two reference points by internally dividing two segments at the same ratio to specifies the object to be monitored in the depth direction of the image PIC1 and PIC2, thereby securely deciding their changes.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a space recognition device, and in particular, monitors a predetermined monitoring space for changes in the presence or absence of an object, changes in the state, occurrence of an abnormal state (hereinafter referred to as abnormality). It is applicable to a space monitoring device.

[0002]

2. Description of the Related Art Conventionally, as this type of space recognizing device, a space monitoring device has been proposed in which two cameras are used to detect an abnormality at a specific position within a monitoring field of view (Japanese Patent Laid-Open No. Sho 60). -264181).

In this space monitoring device, when an attempt is made to monitor a change in the field of view by using one television camera, the surveillance image picked up by the television camera provides a basically flat image. Therefore, in consideration of the fact that the change of the monitored object in the three-dimensional position cannot be monitored, the two monitoring images obtained from the two television cameras arranged at different positions are combined. It is configured to be able to recognize changes in the monitored object at a plurality of positions in the depth direction in each monitoring image.

[0004]

As described above, in the three-dimensional space, a large number of objects to be monitored are provided in the depth direction in the monitoring images obtained from the two television cameras, and a plurality of objects to be monitored in each monitoring image. Even when the target image portions appear to overlap each other, it is desirable to be able to determine this with the highest possible accuracy.

The present invention has been made in consideration of the above points, and is intended to propose a spatial recognition device capable of surely distinguishing and recognizing changes in a plurality of monitored objects in the depth direction. is there.

[0006]

In order to solve such a problem, in the present invention, first and second image pickup means TV1 and TV2 for picking up images of the monitoring spaces to be monitored at different monitoring positions, and the first and second image pickup means. 2 image pickup means TV1,
First and second monitoring images PIC1 and PIC2 in which subject images in the monitoring space are displayed on two-dimensional coordinates based on the image outputs VD1 and VD2 obtained from the TV2.
Image processing means 16 for forming the image data of the first and second object images M1 to M4 in the monitoring space, and the first and second coordinate values on the first and second monitoring images PIC1 and PIC2. Reference point position data forming means 14, SP8 for forming the position data of the two reference points P11 and P12 and the third and fourth reference points P21 and P22.
Regarding the image data of the first monitoring image PIC1, when an abnormality occurs on the first straight line L1 passing between the first and second reference points P11 and P12 in the first monitoring image PIC1, the abnormality is detected. 1st detection point P13 at which
On the second straight line L2 passing between the first and second abnormality determination means 14 and SP9 for detecting the coordinate values (x13, y13) of the second and the third and fourth reference points P21 and P22 in the second monitoring image PIC2. , Has the same internal division ratio or external division ratio as the internal division ratio or external division ratio of the first abnormality occurrence point P13 to the line segment between the first and second reference points P11 and P12 of the first straight line L1. The coordinate value of the second detection point P23 (x23, y2
3) is detected and the coordinate value (x2) of the second detection point P23 is detected.
(3, y23), second abnormality determining means 14, SP11, SP12 for determining whether or not an abnormality has occurred in the image data of the second monitoring image PIC2, and second abnormality determining means 14, SP11, When the occurrence of an abnormality is detected by SP12, the abnormality is detected in the subject portion at the position represented by the first and second detection points P13 and P23 on the straight line passing through the two subject portions in the monitoring space. An abnormality occurrence determination means 14 and SP13 for determining that an abnormality has occurred are provided.

[0007]

When an abnormality occurs at a point on a straight line passing through two subject parts in the surveillance space to be monitored, the three-dimensional coordinate position of the abnormality occurrence point in the surveillance space is 2 in the surveillance space.
It can be expressed as an internal division ratio or an external division ratio with respect to a line segment connecting two monitoring points. This means that in the first and second monitoring spaces PIC1 and PIC2, the line segment between the reference points P11, P12 and P21, P22 corresponding to the two reference points in the monitoring space, respectively, is the internal division ratio in the monitoring space. An abnormal point corresponding to the point of the same internal division ratio (or external division ratio) as (or external division ratio) occurs.

In the present invention, by utilizing this relationship, if there is a point on the straight line L1 passing through the two reference points P11 and P12 in the first monitoring image PIC1, there is an abnormality,
The occurrence point P of the abnormality is an internal division ratio (or an external division ratio) with respect to the line segment L1 between the first and second reference points P11 and P12.
The position P13 having the same internal division ratio (or external division ratio) as this internal division ratio (or external division ratio) is determined to be the third and fourth reference points P21 in the second monitoring image PIC2. P
A straight line passing through 22 is specified and the presence or absence of the abnormality is determined.

Thus, corresponding monitoring points P13 in the first monitoring image PIC1 and the second monitoring image PIC2,
When an abnormality occurs in the image data of P23, it can be determined that an abnormality has occurred at the monitoring point in the monitoring space corresponding to the first and second abnormality occurrence points.

Thus, two television cameras TV1
In addition, based on the planar image data obtained by the video outputs VD1 and VD2 of the TV 2, it is possible to reliably recognize the occurrence of an abnormality at the monitoring point in the depth direction within the monitoring space.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the space recognition device according to the present invention will be described in detail below with reference to the drawings.

(1) Principle Configuration As shown in FIG. 1, in a state in which the monitoring image PIC1 is obtained by the first television camera TV1, a monitoring space in the monitoring space monitored by the monitoring image PIC1 is displayed. Consider a case where the monitored object is imaged by the second television camera TV2 arranged at a position different from that of the first television camera TV1 to obtain the second monitoring image PIC2.

In this case, the first television camera TV
1. In the monitoring image PIC1 obtained by the method 1, a plurality of, for example, four reference points K1j (j = 1, 2 ... J, J = 4) are four monitored objects Mj (j = 1, 2 ……
J, J = 4) is set on the image portion, and thus the monitored point Mj (j = 1, 2 ... J) is determined by the coordinate value of the reference point K1j (j = 1, 2 ... J). It is designed to represent the coordinate value of the video portion.

The monitored object Mj actually in the monitored space
(J = 1, 2, ... J) to the first television camera T
The image light reaching V1 passes through the focal position on the optical axis of the lens of the first television camera TV1 and is incident on the light receiving surface of the image pickup device (for example, CCD, charge coupled device), and the incident light. Position is surveillance image PIC
It can be specified by the xy coordinate value on 1.

In this way, the surveillance image PIC2 obtained by picking up the surveillance space by the second television camera TV2 arranged at a position different from the first television camera TV1 is the surveillance space. Is the same as the monitoring image PIC1 except that the line-of-sight angle to the first television camera TV1 is different.
A monitoring image PIC2 including the image of J) is obtained, and the projection position of the monitoring point K2j (j = 1, 2 ... J) can be specified by the xy coordinate values on the second monitoring image PIC2. become.

By the way, the reference point K1 is thus set on the first and second monitoring images PIC1 and PIC2.
j (j = 1, 2 ... J) and K2j (j = 1, 2 ... J).
J) set monitored object Mj (j = 1, 2 ... J)
Since the positions in the monitoring space have different depths based on the distances from the first and second television cameras TV and TV2, respectively, each reference point K1j (j =
X of 1,2 ... K) and K2j (j = 1,2 ... K)
The y coordinate value substantially has the depth information.

The space recognition apparatus according to the present invention utilizes the depth information included in the coordinate values on each of the monitoring images PIC1 and PIC2 to draw lines and planes passing through a plurality of reference points on the monitoring images PIC1 and PIC2. By designating a point in the monitor point as the monitoring point using the internal division ratio or external division ratio for the line segment connecting the reference points, the change in the brightness information at any point in the depth direction in the monitoring space can be changed. Allow extraction.

(1-2) Designation of the position of transfer First and second television cameras TV1 and TV
2 PIC1 (Fig. 2) and PIC
2 (FIG. 3), reference points K1j and K2j (j =
Two points P1, P12 and P of 1, 2 ... J)
21 and P22 are specified as reference information for designating the monitoring points P13 and P23, respectively, and straight lines L1 and L2 connecting the two reference points P11 and P12 and P21 and P22 are respectively m1 to n1 and m2.
Points internally divided into the pair n2 are designated as monitoring points P13 and P23.

Here, the first and second monitoring images PIC1
And PIC2 at the first reference points P11 and P2
If the monitored objects specified as 1 are the same as each other and the monitored objects specified as the second reference points P12 and P22 are the same as each other, the reference point P11 and the monitoring point P
The ratio of the distance between 13 and the distance between the monitoring point P13 and the reference point P12 (that is, m1: n1) is the second monitoring image PIC.
2, the ratio of the distance from the first reference point P21 to the monitoring point P23 and the distance from the monitoring point P23 to the second monitoring point P22 (that is, m2: n2) becomes equal to

[Equation 1] It becomes a relationship.

In practice, the line segment connecting the two reference points in the surveillance space is connected to the first and second television cameras T.
The difference seen from V1 and TV2 is that the line-of-sight angle to the line segment is different, and when one monitoring point is specified on the line segment, the ratio of the distances from the monitoring point to the reference points at both ends is Even when viewed from the first television camera TV 1 at that line-of-sight angle, the second television camera TV 1
It is the same as when viewed from 2 from this line-of-sight angle.

Therefore, the first and second monitoring images PIC1
When the monitoring points P13 and P23 are unknown in PIC2 and PIC2, the coordinate positions on the line segments L1 and L2 such that the relationship of the equation (1) is established are monitored images P.
By selecting from IC1 and PIC2 and designating the selected coordinate positions as the monitoring points P13 and P23,
After all, any position on the line segment connecting the two reference points in the monitoring space can be designated as the monitoring point.

In practice, in the object recognition device, the first and second monitoring images PIC1 and PIC are
2. A monitoring area of a predetermined size is set at the monitoring points P13 and P23 positions designated on 2 and the brightness data of the pixels included in the monitoring area is extracted (integral calculation or average value calculation is performed). , The monitoring points P13 and P23 by determining whether or not the processing result is larger than the reference value.
It is determined whether or not there is a change in the monitored object at the position.

According to the above construction, the first and second television cameras TV1 and TV1 can be obtained from the first and second television cameras TV1 and TV2.
In addition, by using the planar image information obtained from the second monitoring images PIC1 and PIC2, two reference points are simply selected from the plurality of reference points, and the two points are displayed on the line segment between the two reference points. The monitoring image PIC1 is provided by a simple configuration in which the points obtained by internally dividing the line segments of the book at the same ratio are designated.
In addition, the subject to be monitored in the depth direction of the PIC 2 can be designated and its change can be reliably determined.

(1-3) Designation of Position between Three Points The second method of designating the monitored object in the depth direction is the first method.
And the monitoring images PIC1 and PIC2 obtained from the second television cameras TV1 and TV2,
As shown in FIGS. 4 and 5, three reference points are selected, and common monitoring points are designated on the triangular surface segment including the three reference points to designate the position of the monitored object.

That is, as shown in FIGS. 4 and 5, three arbitrary points are selected from the reference points K1j (j = 1, 2, ... J) set in the monitoring image PIC1 of the first television camera TV1. 4 to select triangular vertices P31, P32, and P33 as shown in FIG. 4, and similarly from the reference point K2j (j = 1, 2 ... J) of the monitoring image PIC2 obtained from the second television camera TV2. Surveillance image PIC1
The same reference points as the three reference points selected in 3 are selected on the monitoring image PIC2 to select the vertices P41 and P4 of the triangular surface.
2 and P43.

On the first monitor image PIC1, one of the three line segments connecting the vertices, for example, the line segment L3 connecting the points P32 and P33 is internally divided into r3: s3.
Along with designating 35, a point P34 is obtained by internally dividing the line segment connecting the vertex P31 and the designated point P35 into m3: n3.

Similarly, in the second monitoring image PIC2, one of the line segments connecting the vertices P41, P42 and P43, for example, the line segment connecting the vertices P42 and P43 is r4:
A point P45 internally divided at a ratio of s4 is designated, and a point P44 obtained by internally dividing a line segment connecting the designated point P45 and the apex P41 facing the line segment L4 at a ratio of m4: n4 is designated.

In this way, if the designated points P34 and P44 on the first and second monitoring images PIC1 and PIC2 are the same points in the monitoring space, the first and second monitoring images PIC1 and P44 are displayed. Monitoring images PIC1 and PIC2
The top two triangles P31, P32 and P33 and P4
Between 1, P42 and P43

[Equation 2] , The frequency division ratio r3: s in the line segments L3 and L4
3 and r4: s4 are equal to each other,

[Equation 3] , The internal ratio m of the line segment between points P31 and P35
3: n3 and the ratio m4: n4 that internally divides the line segment between the points P41 and P45 have the same relationship.

Incidentally, the two monitoring images PIC1 and PIC
The three reference points corresponding to the vertices of the triangle designated on the second screen are set to the first and second television cameras TV1 and T1.
When viewed from V2, the difference is TV Jeon Camera TV1
And the TV2 have different line-of-sight angles, the ratios of the distances divided by the internal division in the monitor images PIC1 and PIC2 are equal to each other.

Thus, the coordinates (x ₃₅ , y ₃₅ ) and (x ₄₅ , y ₄₅ ) of the points P35 and P45 and the coordinates (x ₃₄ , y ₃₄ ) of P34 and P44 such that the expressions (2) and (3) are satisfied. ) And (x ₄₄ , y ₄₄ ), the point P3
4 and P44 mean that one point in the depth direction in the surveillance space is imaged by the first and second television cameras TV1 and TV2, respectively.

Accordingly, the first and second monitoring images PIC1
In addition, a monitoring area having a predetermined size is set at the points P34 and P44 on the PIC2, and it is determined whether the brightness information of the pixels included in the monitoring area exceeds the reference value. It is possible to reliably monitor the change of a certain monitoring target.

The relationships described above with reference to FIGS. 4 and 5 are the internal division ratios r3: s3 and r4: s4 and m3: n3 and m.
4: Since it holds even if n4 is set to an arbitrary value as necessary, eventually, all points in the triangular surface segments P31, P32 and P33 and P41, P42 and P43 can be designated, so that the corresponding monitoring space in the corresponding monitoring space can be specified. It is possible to specify all the monitoring points within the corresponding triangle surface segment.

(1-4) Designation of External Position on Line Passing Two Points In FIGS. 2 and 3, of the plurality of reference points K1j and K2j (j = 1, 2, ... J) in the monitoring space. Although the points P13 and P23 that internally select the line segments L1 and L2 connecting the two points by selecting the two points are designated as the monitoring points, the line segments L1 and L2 are extended to the outside of the two points. Even if the external dividing points are designated on the straight lines passing through the reference points K1j and K2j obtained by the above, and are used as the monitoring points, 1 in the monitoring space is processed in the same manner as described above with reference to FIGS. A point can be designated to monitor the change in brightness of the designated monitored object.

(1-5) Designation of External Position of Triangle Connecting Three Points In FIGS. 4 and 5, three points out of a plurality of reference points in the surveillance space are designated and the respective surveillance images PIC1 and PIC2 are designated.
Regarding the triangle formed above, the case where the monitoring point is specified inside the triangle by specifying the point that internally divides the line forming the triangle was described, but instead, the line forming the triangle is replaced. Even if the external dividing point is designated on the straight line extending outward and is set as the monitoring point, the monitoring point in the monitoring space can be designated in the same manner as in the above case.

Thus, a monitoring point can be designated as necessary on the surface of the triangular surface connecting the three reference points selected in the monitoring space and extending outward.

(1-6) In the embodiments shown in FIGS. 4 and 5, the monitoring points are designated on the monitoring images PIC1 and PIC2 by selecting three points from a plurality of reference points in the monitoring space. Although a triangle is formed, the number of reference points to be selected is not limited to three, and even when a large number of reference points are designated in a polygonal shape on a flat surface or curved surface extending in a strip shape as shown in FIG. , The specified plane or curved surface is divided into a set of triangles, and the divided triangles are shown in FIG.
And, in the same manner as described above with reference to FIG. 5, a straight line connecting the insides of the planes constituting the triangle from one vertex to the inside dividing points (or outside dividing points) of the opposite sides is internally divided (or outside). By doing so, it is possible to specify a monitoring point in the monitoring space in the depth direction as required in the same manner as described above with reference to FIGS. 4 and 5.

(2) Embodiment As shown in FIG. 7, the space recognition apparatus 1 of this embodiment has a structure shown in FIG.
As described above, the first and second views FLD1 and FLD2 having different view angles with respect to the surveillance space are provided.
Of the first and second video signals VD obtained by imaging with the television cameras TV1 and TV2 of
1 and VD2 to the first and second image memories 11A and 1A
Take in 1B.

In the case of this embodiment, the first and second television cameras TV1 and TV2 are provided with the synchronization signal generating circuit 12.
The video signals VD1 and VD2 are formed in synchronization with each other by the synchronization control signal CONT1 transmitted from the synchronization signal generator circuit 12 and the synchronization signal generator circuit 12 is synchronized with the central processing unit (CPU) 14 via the bus 13. By giving the control signal CONT2, the CPU 14 supplies the memory control signal CONT3 via the bus 13 to the first and second image memories 11A and 11B, and thus the first and second image memories 11A and 11B.
B video signals VD1 and VD at the same timing
By making it possible to read 2, the storage positions of the first and second image memories 11A and 11B (hence the xy on the monitoring images PIC1 and PIC2) when the monitoring target in the monitoring space is moving. It is designed not to cause a position shift in the coordinate position).

In this embodiment, the first and second image memories 11A and 11B store 8-bit brightness data as frame (or field) image information at addresses corresponding to 512.times.512 pixels, for example. It is done like this.

First and second image memories 11A and 11
The monitor image brightness data DT1 and DT2 representing the monitor images PIC1 and PIC2 stored in B are the switching control signals CONT4 provided from the CPU 14 via the bus 13.
The monitor image PIC is supplied to the image processing device 16 directly via the switching circuit 15 that performs the switching operation according to the above, and is also supplied to the display device 18 via the adding circuit 17.
1 and PIC2 are displayed on the display screen of the display device 18 as an image that can be visually confirmed by the operator.

When the reference point position designation information PDA is given from the position designation device 21 via the bus 13, the CPU 14 supplies the display control signal CONT5 to the display control device 22, which causes the display control device 22 to generate. The mark display signal MKD is added to the display device 1 via the adder circuit 12.
8 gives the first and second monitoring images PIC.
1 and PIC2 on the reference points K1j and K2j, respectively.
A mark display of a predetermined shape (for example, a square) indicating that (j = 1, 2, ... J) has been set is displayed in the video portion of the monitoring space displayed as the monitoring images PIC1 and PIC2, and the monitoring is performed. When a change occurs in the monitored object on the images PIC1 and PIC2, a display mark indicating that the change has occurred is superimposed on the image portion of the monitored object in which the change has occurred.

In the case of this embodiment, reference points K1j and K2j
When setting (j = 1, 2, ... J), the operator positions the cursor at a predetermined position on the display device 18 using the keyboard provided in the position designation device 21, and then moves the cursor to the position of the positioned cursor. A display mark having a predetermined shape is set and displayed.

The CPU 14 executes the monitoring processing procedure RT shown in FIG.
0 is executed by executing a program stored in the fixed memory 25 while using various registers provided in the work memory 27 according to command information input by the operator via the keyboard 26. There is.

When the CPU 14 enters the monitoring processing procedure RT0, the first monitoring image PIC1 is displayed on the display device 18 in step SP1, and then the operator uses the position specifying device 21 to display the first monitoring image PIC1 in step SP2. It waits for setting the mark display MK of the reference point K1j (j = 1, 2 ... J) on the PIC1.

A reference point K is displayed on the first monitoring image PIC1.
When the setting of the mark display of 1j (j = 1, 2 ... K) is completed, the CPU 14 proceeds to the next step SP3 to display the second monitoring image PIC2 on the display device 18, and then at step SP4. The first on the second monitoring image PIC2
Reference image K2j (j =
Reference points K2j (j = 1, 2, ...
The operator displays the mark display indicating J) by the position specifying device 2
Wait for setting using 1.

When the mark display MK is set on the second monitor image PIC2, the CPU 14 proceeds to step SP.
Moving to step 5, the operator waits for the operator to define, via the keyboard 26, a specification method for specifying a monitoring point where an abnormality has occurred.

Here, as described above, the method of designating the abnormality detection point is as follows: the position between two points on a straight line, the outer position on a line passing through the two points, the inside or the outside of a triangular surface segment, and a large number of reference points. It defines which of the methods to find the position inside or outside of the polygonal line or curved surface connecting between is adopted.

Thereafter, the CPU 14 determines in step SP6 whether or not the setting process is completed. When a negative result is obtained, the CPU 14 returns to step SP1 described above, and then S
P1-SP2-SP3-SP4-SP5-SP6-SP
The setting process is continued by executing the loop LP1 of 1.

On the other hand, when a positive result is obtained in step SP6, the CPU 14 waits for the operator to operate the monitor key of the keyboard 26 in step SP7.

When the operator operates the monitor key, the CP
U enters the monitoring processing operation, and in step SP8, the first
And brightness data based on the video signals VD1 and VD2 from the second television cameras TV1 and TV2.
After the data is loaded into the second image memories 11A and 11B, it is determined in step SP9 whether or not there is an abnormality in the brightness data loaded from the first television camera TV1.

In the abnormality determination processing in step SP9, the CPU 14 executes the image processing apparatus 16 via the bus 13.
The image processing control signal CONT5 is given to the image processing device 16 so that the image processing device 16 executes the processing shown in FIG.

That is, the image processing device 16 changes from the judgment start point PST consisting of the selected reference points K1j to the judgment end point PE.
For the line segment L0 up to D, a monitoring area M of a predetermined size
Kh (h = 1, 2 ... H) is sequentially set at a predetermined interval, and the monitoring area MKh (h = 1, 2 ... H) is set, so that the image portion of the superimposed image is displayed. The average value or integrated value of the brightness data of the pixel is calculated to obtain the difference from the brightness data in the normal state, and when the value of the difference becomes equal to or larger than the determination reference value, the monitoring area MKh (h = 1, 2). …
It is determined that an abnormality has occurred at the position of (H).

As described above with reference to FIGS. 4 and 5, when the image processing apparatus 16 determines abnormality for each point in the triangular area, the point P35 designated by internal division is used.
And the position of P45 are set to the internal division ratios r3: s3 and r4: s4.
By changing the line segments L3 and L4 to the reference point P32.
From P33 to P33 and from P42 to P43, it is finally determined whether or not an abnormality has occurred on the entire triangular plane.

The CPU 14 determines that the result of the determination in step SP9 is the bus 1 in the next step SP10.
Determination result data DDT from the image processing device 16 via
Judge by taking in.

When a negative result is obtained in step SP10, this means that no abnormality has occurred in the first monitored image PIC1, and at this time the CPU 14
Returns to step SP8 described above and then step SP10
Until a positive result is obtained in step SP8-SP
By repeating the processing of the loop LP2 of 9-SP10-SP8, the monitoring for the occurrence of the abnormality in the monitoring space is continuously executed.

On the other hand, when a positive result is obtained in step SP10, this means that the occurrence state of the abnormality is displayed on the first monitoring image PIC1 due to the occurrence of abnormality in the monitoring space. At this time, CPU1
Step 4 moves to step SP11 to enter the processing for calculating the position of the monitoring target where the abnormality has occurred.

That is, the CPU 14 gives the image processing control signal CONT5 to the image processing device 16 via the bus 13 in step SP11, so that the corresponding first detection image PIC1 is detected based on the abnormality detection position. The coordinates on the second monitoring image PIC2 are calculated.

Subsequently, the CPU 14 in step SP12 monitors the monitoring area M of the point P23 on the second monitoring image PIC2.
It is determined whether or not an abnormality has occurred with respect to Kh. When an abnormality has occurred, this is detected in step SP13, and the process proceeds to step SP14 to execute the abnormality processing.

When the abnormality processing is completed, the CPU 14 returns to the above-mentioned step SP8 and starts a new monitoring operation.

On the other hand, when no abnormality has occurred in step SP12, a negative result is obtained in step SP13, and this is confirmed and the process returns to step SP8.

As described above, even if an abnormality is detected at a point on the first monitor image PIC1 of the first television camera, the point corresponding to the abnormality generation point is obtained from the second television camera. When no abnormality has occurred on the acquired second monitoring image PIC2, it is evaluated that no abnormality has occurred as a comprehensive judgment.

Thus, the first and second monitoring images PIC1
When both PIC2 and PIC2 can confirm the occurrence of an abnormality at the corresponding point, it is evaluated that this has occurred at a point in the depth direction, as described above with reference to FIGS.

According to the abnormal configuration, the monitoring images PIC1 and PIC2 of the first and second television cameras TV1 and TV2 are used as planar images, respectively, to monitor the monitoring target at a point in the depth direction within the monitoring field of view. The state can be reliably determined.

(3) Other Embodiments (3-1) FIGS. 11 to 14 show other embodiments.
In this case, the first and second television cameras TV1 and TV2 are, as shown in FIG. 11, the first ridgeline R of the mountain area.
It is installed at two camera installation positions ST1 and ST2 on L1.

First and second television cameras TV1
, And TV2 image through the telephoto lens the scenery of the second ridge line RL2 which is continuous with the first ridge line RL1 across the river RIV, and thus the first and second television cameras TV1 and TV2. The first and second monitoring images PIC11 and PIC12 as shown in FIGS. 12 and 13, respectively, are stored in the first and second image memories 11A and 1A.
1B (FIG. 7).

Two peaks within the second ridge line RL2 are selected as the monitored points M1X and M2X, and the first and second monitored points M1 in the first monitoring image PIC11 are selected.
The first and second reference points K11 on the X and M2X image portions.
X and K12X are set and the first monitoring image PI is set.
Third and fourth reference points K21X and K22X are set in the image portions of the first and second monitored target points M1X and M2X in C12.

In addition to this, in the case of this embodiment, the first
Of the first and second reference points K in the monitoring image PIC11 of
Directly above 11X and K12X (eg 1000
[M]), the first and second auxiliary reference points K11Y and K12Y are respectively set at positions corresponding to points above, and the third and fourth reference points are similarly set in the second monitoring image PIC12. Third and fourth auxiliary reference points K21Y and K22Y corresponding to K12X and K22X are set.

Thus, the CPU 14 (FIG. 7) executes the monitoring processing procedure RT0 described above with reference to FIGS. 8 and 9 in the first and second steps.
Of the monitoring images PIC11 and PIC12, the first and second monitoring images P
When the image of the flying object MCL is displayed on the IC 11 and the PIC 12, the flying object is monitored by the target points M1X and M2.
Auxiliary reference points K11Y and K12Y on the straight line connecting X
When passing through a region having a predetermined height (that is, a height of 1000 [m]) corresponding to K21Y and K22Y, it can be detected as if the flying object has passed through the barrier.

In the case of this embodiment, the first monitoring image PIC
11, the first and second reference points K11X and K12
A line segment L11 connecting between X and the first and second reference points K11
X and K12X and line segments L12 and L13 that connect the first and second auxiliary reference points K11Y and K12Y set above them, and the first and second auxiliary reference points K11Y and K12
The first monitoring image P is formed by the line segment L14 connecting the 12Ys.
A quadrangular surface segment formed in a plane on the IC 11 is defined by, for example, a first reference point K11X and a second auxiliary reference point K12Y.
It is divided into two triangular surface segments by a line segment L15 connecting the two, and these two triangular surface segments are monitored at the monitoring points on the first monitoring image PIC11 described above with reference to FIG. 4, that is, in the flying object MCL. The position search process is executed by the same method as described above with reference to FIG.

Similarly, the second monitoring image PIC12
At the third and fourth reference points K21X and K22X
A line segment L21 connecting between the third and fourth reference points K21X
And K22X and the line segments L22 and L23 connecting the third and fourth auxiliary reference points K21Y and K22Y set above them, and the third and fourth auxiliary reference points K12Y and K2.
A quadrangular surface segment formed by a line segment L24 connecting between 2Y is a third reference point K21X and a fourth auxiliary reference point K2.
It is divided into two triangular facet segments by a line segment L25 connecting between 2Ys, and the search for the coordinates where the flying object MCL exists for each triangle facet segment is performed by the same method as described above in FIG. Search for.

Thus, the auxiliary monitored object set at a predetermined height (for example, an altitude of 1000 [m]) above the two monitored object points M1X and M2X set on the second ridge line RL2. When a flying object MCL flew in such that a quadrangular surface area surrounded by dots penetrates this as a barrier stretched over the mountainous area, the flying object is displayed on the first monitoring image PIC11. By obtaining the coordinate value, an internal division ratio of the coordinate value and the line segment L16 passing through the first auxiliary reference point K11Y is obtained, and a coordinate having the same internal division ratio as the internal division ratio is displayed on the second monitoring image PIC12. Flying object MC
A line segment L connecting the image of L and the third auxiliary reference point K21Y
26, it can be recognized that the flying object MCL has penetrated through the barrier formed on the mountain.

In the case of this embodiment, the first, second, third and fourth auxiliary reference points K11Y, K12Y, K21Y and K2 on the first and second monitoring images PIC11 and PIC12 are used.
The method of FIG. 14 is used to set 2Y.

That is, the monitored points M1X and M2X on the second ridge line RL2 are controlled by the first and second television cameras TV1 and TV2 installed on the first ridge line RL1.
As shown in FIG. 14, the first and second television cameras TV1 and TV2 use the telephoto lens LNS when capturing an image of
If the television cameras TV1 and TV2 are set so that their optical axis R1 becomes horizontal through,
And a point B intersecting the optical axis R1 on a line extending in the vertical direction through the point A representing the optical positions of the second monitored target points M1X and M2X, and the first and second monitored target points. M1
Consider point C, which represents the position of X and M2X.

In this optical system, the images of the first and second monitored points M1X and M2X are taken by the telephoto lens LNS.
If an image is formed at a point a representing the positions on the image planes of the television cameras TV1 and TV2 along an optical line R2 passing through the center of, the image of the point B on the optical axis R1 is on the image plane. Point b
And the images of the first and second monitored points M1X and M2X at the point C are imaged at the point c on the image plane through the optical line R3. In FIG. 14, the distance H between points A and B represents the height of the first and second monitored points M1X and M2X, and the distance h between points A and C is the first and second monitored points. Height from M1X and M2X to auxiliary monitoring points set above (1000 [m] in this embodiment)
Represents

On the other hand, a point a corresponding to points A and B on the image planes of the television cameras TV1 and TV2.
The distance between points a and b is ya, and the point a corresponding to points A and C is
And the distance between c and y is yc, and the cameras TV1 and TV
2 from L to the monitored points M1X and M2X, that is, from the television cameras TV1 and TV2 to the monitored points M1X and M2X, and L is the focal length of the telephoto lens LNS. For the distance ya between a and b on the image plane, the following equation,

[Equation 4] Therefore, the focal length f is

[Equation 5] Can be expressed as

Furthermore, the distance yc between the points a and c on the image plane
For the following equation,

[Equation 6] And the distance Yc is solved from the expression (6) and the expression (5) is substituted, the following expression,

[Equation 7] Can get a relationship.

Thus, the first and second monitoring images PIC1
1 and the distance between the first reference point K11X and the first auxiliary reference point K12Y on the screen of the PIC12, the second reference point K1.
The distance between 2X and the second auxiliary reference point K12Y, the distance between the third reference point K21X and the third auxiliary reference point K21Y, and the fourth reference point K22X and the fourth auxiliary reference point K22Y.
The distance between them can be obtained by calculation based on the equation (7), and thus the first to fourth auxiliary reference points K11Y to K
The coordinate position on the screen of 22Y can be easily set.

(3-2) In FIGS. 11 to 14, when a barrier having a height of 1000 [m] is provided on a straight line connecting two monitored points M1X and M2X on a ridgeline in a mountainous area. However, the present invention is not limited to this, and for example, two points on the top surface of the fence are selected as the monitored points M1X and M2X, and a monitoring barrier of a predetermined height is provided above the two points on the fence. Also when setting, by setting auxiliary reference points on the screens of the first and second monitoring images PIC11 and PIC12 in the same manner as described above with reference to FIGS. It is possible to easily realize a recognition device capable of detecting an intruder when it appears on the formed barrier.

(3-3) In the embodiment described above, FIG.
As shown in 0, the brightness data of the pixels in the monitoring area MKh (h = 1, 2 ... H) centered on the point on the line segment L0 connecting the two reference points is calculated as an average value or an integration operation. Although the presence or absence of abnormality is determined by doing so, instead of this, as shown in FIG. 15, a predetermined number (8 × 8) of pixels (for example, 512 × 512 pixels) on the monitoring image PICX are used.
64 × 64 monitoring mesh areas AR are formed by dividing each of them into mesh shapes, and all the data of the monitoring mesh area AR through which the line segment between the reference points passes or only a part of the data are stored in the first and second image memories. Even if the occurrence of an abnormality is determined for each surveillance camera area AR by incorporating it into 11A and 11B, the same effect as in the above case can be obtained.

In practice, the image memory 1
Reference numerals 1A and 11B are composed of an analog digital converter, an adder, and an integrator having a memory. The integrated data of the brightness of 8 × 8 pixels included in the sequentially designated surveillance camera area AR are first and By storing in the second image memories 11A and 11B, for example, 64 × 64 × 8 bits or 64 × 64 × 16 bits of image data are stored in the first and second image memories.
Image memory 11A and 11B.

(3-4) In the above embodiment, the monitoring points K1j and K2j on the monitoring images PIC1 and PIC2.
As a means for setting (j = 1, 2, ... J), the case where the cursor is positioned by the keyboard to set the display mark has been described. However, instead of this, a mouse, a joystick, a light pen, A touch sensor, a digitizer (pen input), or the like may be used.

(3-5) Monitoring images PIC1 and PIC2
, The reference points K1j and K2j (j = 1, 2, ...
J) as means for setting the monitoring images PIC1 and PI
The singular point included in the image of C2 may be automatically extracted and set. For example, first, a thin light beam (visible light beam, far-infrared beam or infrared beam) is applied to a desired object in the monitoring field of view, and the reflected light is reflected by the first and second television cameras TV1 and TV2. An incident high-luminance light spot is formed in the monitoring images PIC1 and PIC2, the coordinate position of the high-luminance light spot is detected by the position designation device 21, and a reference point is set at the detection position. To do so.

Secondly, an illumination light source is arranged at a predetermined position in the dark surveillance space, or a reflecting mirror is provided in the bright surveillance space, and the light from the illumination light source or the reflecting mirror is supplied to the first and second televisions. It enters the Jyon cameras TV1 and TV2, and the coordinate position on the monitor images PIC1 and PIC2 is automatically calculated by the position specifying device 21 and the reference point is set.

Thirdly, when an outdoor space in which no object is present is used as the monitoring space, a flying object,
For example, a helicopter is caused to fly and the first and second television cameras TV1 and TV2 are imaged to store a monitoring image sequence composed of a plurality of monitoring images PIC1 and PIC2 in the image storage device according to the passage of time. The flight trajectory of the helicopter is detected from the surveillance image sequence as coordinates on the surveillance image, and the reference point is determined based on the coordinate position on the trajectory.

In this case, in the image sequence of the monitoring images PIC1 and PIC2, the video signals of the first and second television cameras TV1 and TV2 are recorded in the corresponding video tape recorders together with the time code, and the video signals from the respective video tape recorders are recorded. At the time of reproduction, the coordinate positions of the helicopters of the frames having the same time code are made to correspond to each other.

When this outdoor space is used as the monitoring area, the flying helicopter is monitored by the monitoring images PIC1 and PIC.
2 may be automatically recognized and the locus may be represented by each recognized position on the screen.

(3-6) In the above embodiment (FIG. 7), the first and second television cameras TV1 and TV are used.
The first and second image memories 11A and 11B are provided as two image capturing devices for each of the two image capturing devices, and one image processing device 16 is provided for the two image capturing devices.
However, instead of this, a dedicated image processing device may be provided for each of the first and second image memories 11A and 11B, or the first and second image memories 11A and 11B may be provided.
One image memory and one image processing device are provided for each of the television cameras TV1 and TV2, and these image memories and image processing devices are sequentially and alternately used for the first and second television cameras by time division. Even in this case, the same effect as the above case can be obtained.

(3-7) In the above-described embodiment (FIG. 7), the case where the two television projection cameras TV1 and TV2 are used has been described, but instead of this, three television projection cameras or more are used. It may be provided so that the blind spot may be eliminated.

(3-8) In the above embodiment, the monitoring images PIC1 and PIC2 are picked up by the television camera.
However, the image pickup of various configurations such as obtaining a video signal by a photoelectric conversion element (for example, a CCD element) arranged on a plane instead of the television cameras TV1 and TV2 has been described. Means can be applied.

[0090]

As described above, according to the present invention, the depth as necessary is determined based on the coordinate positions of the same two reference points displayed on the two monitoring images obtained from the two image pickup means. It is possible to realize a space recognition device that can surely obtain position information regarding a point in a direction.

[Brief description of drawings]

FIG. 1 is a schematic diagram front view showing a monitoring image for explaining an operation principle of a space recognition device according to the present invention.

FIG. 2 is a schematic diagram for explaining a monitoring point on a line segment of a first monitoring image.

FIG. 3 is a schematic diagram for explaining a monitoring point on a line segment of a second monitoring image.

FIG. 4 is a schematic diagram for explaining a monitoring point on a triangular surface segment of a first monitoring image.

FIG. 5 is a schematic diagram for explaining a monitoring point on a triangular surface segment of a second monitoring image.

FIG. 6 is a schematic diagram for explaining a case of setting a monitoring point on a plane or a curved surface.

FIG. 7 is a block diagram showing the overall configuration of a space recognition device according to the present invention.

FIG. 8 is a flow chart showing a monitoring processing procedure.

FIG. 9 is a flow chart showing a monitoring processing procedure.

FIG. 10 is a schematic diagram used for explaining a method of capturing data from an abnormality occurrence point.

FIG. 11 is a plan view showing an embodiment in which a flying object monitoring buffer is set in a mountain area.

12 is a front view showing a first monitor image obtained by the first television camera TV1 of FIG. 11. FIG.

13 is a front view showing a second monitoring image obtained by the second television camera TV2 of FIG. 11. FIG.

14 is an optical path diagram showing an optical system of the first and second television cameras TV1 and TV2 of FIG. 11. FIG.

FIG. 15 is a schematic diagram showing a monitoring image in another embodiment.

[Explanation of symbols]

TV1, TV2 ... First and second television cameras, PIC1, PIC2, PIC11, PIC12 ...
First and second monitoring images, K11 to K14, K21 to K
24, K11X to K22X ... Reference point, K11Y to K2
2Y ... Auxiliary reference points, M1-M4 ... Monitoring target, 1 ...
Space recognition device, 11A, 11B ... First and second image memories, 13 ... Bus, 14 ... Central processing unit (CP
U), 12 ... Synchronous signal generating circuit, 15 ... Switching circuit,
16 ... Image processing device, 18 ... Display device, 21 ... Positioning device, 22 ... Display control device, 25 ... Fixed memory, 26 ... Keyboard, 27 ... Work memory.

Claims (1)

[Claims] 1. A first and second image pickup means for picking up an image of a monitoring space to be monitored at different monitoring positions, and two-dimensional coordinates based on image outputs obtained from the first and second image pickup means, respectively. Image processing means for forming image data of first and second monitoring images by displaying subject images in the monitoring space above, and the first of the subject image parts representing two subject parts in the monitoring space. Reference point position data forming means for forming coordinate values on the first and second monitoring images as position data of the first and second reference points and the third and fourth reference points, and the first monitoring image Regarding the image data, when an abnormality occurs on the first straight line passing between the first and second reference points in the first monitoring image, the coordinate value of the first detection point where the abnormality occurs is detected. The first difference On the second straight line passing between the determination means and the third and fourth reference points in the second monitoring image, with respect to the line segment between the first and second reference points of the first straight line, The coordinate value of the second detection point having the same internal division ratio (or external division ratio) as the internal division ratio (or external division ratio) of the first abnormality occurrence point is detected, and the coordinate value of the second detection point is detected. The occurrence of an abnormality is detected by the second abnormality determining means for determining whether or not an abnormality has occurred in the image data of the second monitoring image corresponding to the coordinate value, and the second abnormality determining means. At this time, an abnormality occurrence determination means for determining that an abnormality has occurred in the subject portion at the position represented by the first and second detection points on the straight line passing through the two subject portions in the monitoring space. A space recognition device characterized by comprising.