JP2002094969A - Supervisory system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a supervisory system having no dead angle. SOLUTION: The supervisory system includes an omnidirectional image photographing device 2 that is installed in a supervisory area for omnidirectionally picking up images and transmitting the picked-up omnidirectional images, and an omnidirectional image supervisory device 4 that wirelessly receives image data from the omnidirectional image photographing device 2 and stores the received image data. The omnidirectional image photographing device 2 includes an omnidirectional photographing section 20 that photographs the omnidirectional images at the same time, an external sensor 12 that senses an external intruder, an image discrimination section 14 that discriminates whether or not any intruder exists in the supervisory area on the basis of an output from the external sensor 12 and the images photographed by the omnidirectional image supervisory device 4, and an alarm generating section 18 that issues an alarm on the basis of an instruction from the image discrimination section 14.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surveillance system, and more particularly to a surveillance system capable of simultaneously observing 360 degrees around.

[0002]

2. Description of the Related Art In a conventional monitoring system for monitoring the intrusion of a suspicious person, a wide range of images are photographed using a plurality of cameras in order to eliminate blind spots in a monitoring area. in this way,
In order to eliminate the blind spot in the monitoring area, a plurality of cameras must be used, and the installation work is also difficult.

[0003] Therefore, a surveillance system using a revolving camera has been developed. In this surveillance system, the camera is turned 360 degrees to capture an omnidirectional image and eliminate blind spots.

[0004]

However, in order to capture an omnidirectional image with a revolving camera, it takes a time for the camera to make one round. For this reason, it is not possible to capture an omnidirectional image at the same time, and a blind spot always occurs when viewed at a specific time.

Further, since a mechanism for turning the camera is required, there is a problem that the camera is easily broken.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and an object of the present invention is to provide a monitoring system free from blind spots.

[0007]

A monitoring system according to an aspect of the present invention is provided in a monitoring area, captures an omnidirectional image, and transmits the captured omnidirectional image. An omnidirectional image monitoring device that receives image data from the omnidirectional image capturing device and records the received image data. The omnidirectional imaging device includes an omnidirectional imaging unit that captures an omnidirectional image at the same time, a first information recording unit that is connected to the omnidirectional imaging unit, and stores an image captured by the omnidirectional imaging unit. , A first transmitting and receiving unit connected to the first information recording unit for transmitting and receiving data to and from the omnidirectional image monitoring device, and connected to the first information recording unit and photographed by the omnidirectional photographing unit. An image judging unit for judging whether or not an intruder is present in the monitoring area based on the image and controlling to transmit the image from the first transmitting / receiving unit; and an image judging unit connected to the image judging unit. And an alarm generation unit for generating an alarm based on the information. The omnidirectional imaging unit includes a camera and a light condensing unit that is disposed at a position having a predetermined relationship with the camera and condenses an omnidirectional image to the camera.

[0008] The omnidirectional photographing section can monitor an image of 360 degrees around at a time. For this reason, a monitoring system without blind spots can be provided. Further, it is possible to determine whether or not there is an intruder outside based on the captured image. For this reason, an alarm can be immediately generated and the intruder can be threatened.

[0009] Preferably, the omnidirectional image photographing apparatus further includes an external sensor for detecting an intruder from outside. The image determination unit is connected to the first information recording unit and the external sensor, and detects an external intruder based on an image captured by the omnidirectional imaging unit and an output of the external sensor.

The image determining unit determines whether there is an intruder based on not only the image but also an external sensor. For this reason, it is possible to reduce the malfunction of the alarm generation unit.

[0011] More preferably, the first transmitting / receiving section transmits an image. The omnidirectional image monitoring device includes a second transmitting / receiving unit that transmits / receives data to / from the omnidirectional image capturing device, a second information recording unit that records information, a second transmitting / receiving unit, and second information. An image determination unit connected to the recording unit and recording the image received by the second transmission / reception unit together with the time in the second information recording unit.

[0012] The transmitted image is recorded in the second information recording section with time. Therefore, it is possible to know later when the intruder has entered.

[0013] More preferably, the image judging section records the image in the second information recording section only when a change has occurred in the image.

Since there is no need to store many similar images, the storage capacity of the second information recording unit can be reduced.

More preferably, when a change occurs in the image, the image determination unit records only the image of the portion where the change has occurred in the second information recording unit.

An image of a portion where a change has occurred, that is, an image of only an intruder is recorded. For this reason, the storage capacity of the second information recording unit can be further reduced.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG. 1, a monitoring system according to an embodiment of the present invention is provided in a monitoring area, captures an omnidirectional image, and transmits the captured omnidirectional image. It includes an image capturing device 2 and an omnidirectional image monitoring device 4 that receives image data from the omnidirectional image capturing device 2 via wireless and stores the received image data.

The omnidirectional image photographing device 2 includes an omnidirectional photographing section 20 for photographing omnidirectional images at the same time, and an omnidirectional photographing section 2.
A / D (Analog to Digita) that is connected to the digital camera and digitizes an analog video signal output from the omnidirectional imaging unit 20
l) The conversion unit 22 is connected to the A / D conversion unit 22,
An information recording unit 24 for recording image data output from the D conversion unit 22, an external sensor 12 for detecting an intruder from outside, an output of the external sensor 12, and image data stored in the information recording unit 24. And an image determination unit 14 that determines whether an intruder exists in the monitoring area.

The omnidirectional image photographing apparatus 2 is further connected to an image determining unit 14 and an information recording unit 24, and generates an alarm based on an instruction from the image determining unit 14.
An input unit 28 connected to the information recording unit 24 for inputting various commands and the like; a display unit 30 connected to the information recording unit 24 and displaying image data recorded in the information recording unit 24; The information recording unit 24 is connected to the recording unit 24.
An image compression unit 32 for compressing image data recorded in the
An image processing unit 16 that is connected to the information recording unit 24 and processes the image data recorded by the omnidirectional photographing unit 20 recorded in the information recording unit 24 into an image viewed from a normal camera; The image data recorded in the information recording unit 24 is connected to the recording unit 24 based on a transmission request of image data received from the omnidirectional image monitoring device 4 through the communication line.
And a transmission / reception unit 26 for transmitting to the

The omnidirectional image monitoring device 4 has an input unit 54 for inputting various instructions, and is connected to the input unit 54.
Omnidirectional image capturing device 2
Connected to the transmitting and receiving unit 50 for transmitting image data from the omnidirectional image photographing apparatus 2 and the image recording unit 46 for recording the image data and other data received by the transmitting and receiving unit 50. Information recording unit 4
6 and an image decompression unit 42 for decompressing the compressed image data.

The omnidirectional image monitoring device 4 is further connected to an information recording unit 46 and a transmitting / receiving unit 50.
6, an image judging unit 48 for judging whether or not there is a suspicious person in the image data from the decompressed image data stored in the information recording unit 46; A display unit 44 connected to the input unit 54 and displaying the decompressed image data stored in the information recording unit 46 based on an instruction from the input unit 54; and the input unit 54, the display unit 44, and the information recording unit 46 , And converts the image data stored in the information recording unit 46 based on an instruction from the input unit 54, and displays the image data on the display unit 44.
And

Referring to FIG. 2, omnidirectional photographing section 20 is provided at the tip of pole 68, and has a hyperboloid mirror 64 installed vertically downward, a camera 62 turned vertically upward, and
A hyperboloid mirror 64 and a cylindrical transparent cover 66 covering the camera 62 are included.

Regarding the omnidirectional photographing section 20, see "Kazumasa Yamazawa et al .:" A Omnidirectional Visual Sensor for Navigation of Mobile Robots ", IEICE Transactions D-IIVol.
J79-D-II No. 5pp. 698-707 (19
May 1996) ".

Hereinafter, a schematic configuration of the omnidirectional photographing section 20 will be described. Referring to FIG. 3, hyperboloid mirror 64 uses a hyperboloid in a region of Z> 0 among two-lobe hyperboloid as a mirror. The two-lobed hyperboloid is a curved surface obtained by rotating a hyperbola around a real axis (Z axis). The two-lobe hyperboloid has two focal points (0, 0, + c) and (0, 0, -c). However,

[0025]

(Equation 1)

## EQU1 ## Here, a three-dimensional coordinate system O-XYZ having the Z axis as a vertical axis as shown in FIG. 3 is considered. At this time, the two-lobe hyperboloid is expressed by the following equation (1).

[0027]

(Equation 2)

The constants a and b define the shape of the hyperbola. Referring to FIG. 4, omnidirectional photographing unit 2
Numeral 0 is composed of a hyperboloid mirror 64 located vertically downward in a region of Z> 0, and a camera (not shown) vertically disposed below it. At this time, the focal point OM of the hyperboloid mirror 64 and the lens center OC of the camera are located on the two focal points (0, 0, + c) and (0, 0, -c) of the bilobal hyperboloid, respectively. A mirror 64 and a camera are arranged. The image plane xy is a plane parallel to the XY plane and separated from the lens center OC of the camera by the focal length f of the camera. The reflecting surface of the hyperboloid mirror 64,
The focal point OM of the hyperboloid mirror 64 and the lens center OC of the camera are represented by the following equation (2).

[0029]

(Equation 3)

Referring to FIG. 5, an arbitrary point P in space
The mapping point on the image for (X, Y, Z) is p (x,
When y), the azimuth θ of the point P is expressed by the following equation (3).

Tan θ = Y / X = y / x (3) That is, the azimuth θ of the point P defined by Y / X is y / x
Is obtained by calculating the azimuth angle θ of the mapping point p determined by. As described above, the azimuth angle θ of the target object in the 360-degree panoramic region directly appears as the azimuth of the mapping on the image plane of the object.

Referring to FIG. 6, assuming a vertical section including point P and the Z axis, a point P and a mapping point p
The following equation (4) holds.

[0033]

(Equation 4)

That is, the azimuth θ and the depression angle α of the point P from the focal point OM of the hyperboloid mirror 64 can be determined by setting the lens center OC of the camera at the focal position of the hyperboloid.
It is uniquely obtained from (x, y). At this time, since the focal point OM of the hyperboloid mirror 64 is fixed, the input image can be converted into an image obtained by rotating the camera as viewed from the focal point OM of the hyperboloid mirror 64 around the vertical axis or an image of a normal camera. .

Referring to FIG. 7, omnidirectional image photographing apparatus 2
Works as follows. When the monitoring system is activated (S2), the omnidirectional imaging unit 20 captures an omnidirectional image (S6). The photographed image is converted by the A / D converter 2
In step 2, the image data is converted into digital image data (S8).
The A / D converter 22 stores the digitally converted image in the information recording unit 24 (S10).

Next, an image determining subroutine for determining from the image whether an abnormality has occurred is executed (S1).
2). The details of the image determination subroutine will be described later.

The image processing unit 16 or the image compression unit 32
In response to a request from the omnidirectional image monitoring device 4, processing such as image compression, image color reduction or image deformation is performed (S14). Here, the image transformation process refers to a process of converting the image into a normal camera image by the above-described conversion method.
The processed image is transmitted from the transmitting / receiving unit 26 to the omnidirectional image monitoring device 4 (S16).

It is determined whether or not the current time is within the monitoring time set in advance in S14 (S18). If it is within the monitoring time (Y in S18), the processing from S6 is repeated. If the monitoring time has passed (N in S18), the process ends.

Referring to FIG. 8, the image determining subroutine (S12 in FIG. 7) will be described. A difference is taken between two temporally consecutive images, the difference image is binarized, and it is determined whether or not the area of the change region is larger than a predetermined threshold (S42). If the area of the change region is larger than the predetermined threshold value (Y in S42), it is highly possible that an abnormality has occurred, so that the action subroutine is executed (S44). Thereafter, the image determination subroutine ends. The details of the action subroutine will be described later. If the area of the change area is equal to or smaller than the predetermined threshold (N in S42), it is considered that no abnormality has occurred, and the image determination subroutine ends.

The action subroutine (S44 in FIG. 8) will be described with reference to FIG.

The image judging section 14 judges whether or not a condition for generating a warning has been satisfied (S52). The warning occurrence condition is a condition that becomes true when an external sensor 12 such as a door sensor, a window sensor, or an infrared sensor reacts. If the alarm generation condition is not satisfied (N in S52), the action subroutine ends.

When the alarm generation condition is satisfied (Y in S52), it is determined whether or not the generation of a warning is set by the omnidirectional image monitoring device 4 (a request for warning is set). (S54). If it is set to generate a warning (Y in S54), the image determination unit 14 generates a warning by sound or light to the approaching individual or approaching object using the alarm generation unit 18 (S56). ).

If it is not set to generate a warning (N in S54), or after the processing in S56, the omnidirectional image monitoring device 4 sets the information transmission at the time of the warning. It is determined whether or not the request for the warning information transmission setting has been set (S58).
If it is set to transmit information at the time of warning (Y in S58), the image determination unit 14 transmits various information to the omnidirectional image monitoring device 4 via the transmission / reception unit 26 (S6).
0). The transmitted information includes log information indicating where and what happened.

If it is not set to transmit information at the time of warning (N in S58), or after S60, the action subroutine ends.

Referring to FIG. 10, omnidirectional image monitoring device 4
Works as follows. When the monitoring system is activated (S72), a monitoring initial setting subroutine for setting a monitoring area and the like is executed (S74). Details of the monitoring initialization subroutine will be described later. The transmitting and receiving unit 50
In addition to parameters such as the monitoring area set in 74, a request for image compression, image color reduction, transmission of an image converted to a normal camera image, etc., in response to an input from the input unit 54, is sent to the omnidirectional image photographing device 2. It is transmitted (S78). The transmitting / receiving unit 50 receives the transmitted processed image by the processing in S16 of FIG. 7 (S80). The image determining unit 48 stores the image in the information recording unit 46 together with the time (S8).
2). At that time, it is determined whether or not the image has changed, and if the image has not changed, the image is not recorded. If a change has occurred, only the portion where the change has occurred is recorded.

The display section 44 displays the image stored in the information recording section 46 in S82 (S86). It is determined whether it is the predetermined monitoring time (S88), and if it is within the predetermined monitoring time (Y in S88), the processing after S76 is continued. If it is outside the predetermined monitoring time (N in S88), the process is terminated.

The monitoring initialization subroutine (S74 in FIG. 10) will be described with reference to FIG.

The monitoring time period is input from the input unit 54, and the monitoring time period for operating the omnidirectional image photographing device 2 is set in the information recording unit 46 (S22). The set monitoring time zone is used for the determination in the processing of S18 of FIG. 7 and S88 of FIG. Next, a monitoring area setting subroutine for setting a monitoring area is executed (S24). Details of the monitoring area setting subroutine will be described later. After the transmission of the initial image is transmitted to the omnidirectional imaging device 2, the omnidirectional imaging unit 20 captures the initial image (S26). The position of the monitoring area is confirmed on the captured initial image, and if the position of the monitoring area is incorrect, correction is performed at that time (S28).

The monitoring area setting subroutine (S22 in FIG. 11) will be described with reference to FIG.

First, the monitoring time zone set in S22 of FIG. 7 is subdivided into several (S32). For example, FIG.
In S22, the monitoring time zone is "AM 9:00 to PM 6: 0
0 is set to “AM9: 00:
PM 0:00 "and" PM 0:00 to PM 6:00 ".

Next, a monitoring area is set for each subdivided monitoring time zone (S34). The method of setting the monitoring area will be described later in detail. It is determined whether the monitoring area has been set for all the monitoring time zones (S36). If there is a monitoring time zone for which the monitoring area has not been set (N in S36), the processing from S32 is repeated. It is. If the monitoring area has been set for all the monitoring time zones (Y in S36), the monitoring area setting subroutine ends.

Next, a method of setting a monitoring area in S34 of FIG. 12 will be described. FIG. 13 is a diagram illustrating the relationship between the omnidirectional imaging unit 20 and the monitoring area in the three-dimensional space. FIG. 14 is a diagram illustrating a relationship between the input image captured by the omnidirectional imaging unit 20 and the monitoring area.

Referring to FIG. 13, P (X, Y, Z) is
The coordinates in the three-dimensional space when the focal point OM of the hyperboloid mirror 64 is set as the origin. The monitoring area is hatched in the figure. When the monitoring area to be set is projected on the input image, the monitoring area becomes a fan-shaped monitoring area as shown in FIG. The coordinates of the four corners in the three-dimensional space of the monitoring area are E1
1, E12, E21 and E22. When the coordinates of these four corners are projected on the input image, the result is as shown in FIG. The angle of the sector from the x-axis is sandwiched between θ1 and θ2.

The coordinates of E11 in the three-dimensional space are represented by (X1
1, Y11, Z1) and E12 in the three-dimensional space are (X12, Y12, Z1), E21 in the three-dimensional space are (X21, Y21, Z1), and E22 are in the three-dimensional space. Are defined as (X22, Y22, Z1).

A horizontal plane with the ground passing through the focal point OM and a straight line OM
Assuming that the angle between E11 and α1 is α1, the relationship of Expression (5) holds. Equation (6) is derived from equation (5). The angle between the horizontal plane with the ground passing through the focal point OM and the straight line OME12 is α
Assuming that 2, the relationship of equation (7) holds. Equation (8) is derived from equation (7).

The angles α and β shown in FIG.
It can be expressed by equation (9) and equation (10), respectively.

[0057]

(Equation 5)

Referring to FIG. 14, the coordinates of the projection point on the input image of E11 are (E11x, E11y), the coordinates of the projection point on the input image of E12 are (E12x, E12y), and E2
(E21x, E21x)
y), the coordinates of the projection point on the input image in E22 are set to (E22
x, E22y).

From the above relationship, E11x and E11y
Is represented by equations (11) and (12), respectively.
E12x and E12y are represented by equations (13) and (14), respectively. Similarly, E21x, E2
1y, E22x and E22y can also be determined.

Therefore, assuming that the azimuth from the x-axis on the input image is θ, the fan-shaped monitoring area is expressed by the following equations (15) to (1).
7). Therefore, in order for the user to set the fan-shaped monitoring area, the azimuth angles θ1 and θ2 and the coordinates E11, E12, and E in the three-dimensional space of the four corners of the fan-shaped area are set.
21 and E22 may be input.

[0061]

(Equation 6)

Next, a method of setting a monitoring area having a more complicated shape will be described. As shown in FIG. 15, even in the case of a monitoring area having a complicated shape, it can be regarded as a set of fan-shaped areas. Therefore, by inputting the azimuth angle and the coordinates of the four corners in the three-dimensional space in the manner described above, a complicated monitoring area can be set for each sector area.

As described above, according to the present embodiment, it is possible to capture 360-degree surrounding images at a time.
For this reason, a monitoring system without blind spots can be provided.

The image at the time of occurrence of the abnormality is stored together with the time. Therefore, it is possible to know exactly when an abnormality has occurred.

Further, it is possible to record only an image which has changed when an abnormality has occurred. For this reason, the storage capacity of the information recording unit of the omnidirectional image monitoring device can be reduced.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a hardware configuration of a monitoring system according to an embodiment of the present invention.

FIG. 2 is an external view of an omnidirectional imaging unit.

FIG. 3 is a diagram illustrating a two-lobed hyperboloid.

FIG. 4 is a diagram showing a configuration of an omnidirectional vision sensor.

FIG. 5 is a first diagram illustrating a relationship between an arbitrary point in space and a mapping point on an image.

FIG. 6 is a second diagram illustrating the relationship between an arbitrary point in space and a mapping point on an image.

FIG. 7 is a flowchart of a process executed by the omnidirectional image photographing apparatus.

FIG. 8 is a flowchart of an image determination subroutine.

FIG. 9 is a flowchart of an action subroutine.

FIG. 10 is a flowchart of a process executed by the omnidirectional image monitoring device.

FIG. 11 is a flowchart of a monitoring initialization subroutine.

FIG. 12 is a flowchart of a monitoring area setting subroutine.

FIG. 13 is a diagram illustrating a relationship between an omnidirectional imaging unit and a monitoring area in a three-dimensional space.

FIG. 14 is a diagram illustrating a relationship between an input image captured by an omnidirectional imaging unit and a monitoring area.

FIG. 15 is a diagram illustrating a relationship between an input image captured by an omnidirectional imaging unit and a monitoring area.

[Explanation of symbols]

2 Omnidirectional image capturing device, 4 Omnidirectional image monitoring device, 1
2 external sensor, 14 image determination unit, 16 image processing unit,
18 alarm generation unit, 20 omnidirectional imaging unit, 22 A / D
Conversion unit, 24, 46 information recording unit, 26 transmitting / receiving unit, 2
8, 54 input unit, 30, 44 display unit, 32 image compression unit, 42 image expansion unit, 48 image judgment unit, 50 transmission / reception unit, 52 image conversion unit, 62 camera, 64 hyperboloid mirror, 66 transparent cover, 68 pole .

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G08B 25/00 510 G08B 25/00 510G 510M 25/04 25/04 E H04N 5/225 H04N 5/225 C DF term (reference) 5C022 AA05 AC51 5C054 AA01 AA05 CA04 CC03 EA01 EA05 EA07 FC13 FF06 GB01 GD01 GD05 HA18 5C084 AA07 BB40 CC17 DD11 DD87 GG78 HH12 HH13 5C087 AA02 BB74 GG04 GG08 GG08 GG08 GG08 GG08

Claims (6)

[Claims] 1. An omnidirectional image capturing device provided in a monitoring area for capturing an omnidirectional image, transmitting the captured omnidirectional image, receiving image data from the omnidirectional image capturing device, and receiving the received image data. An omnidirectional image monitoring device that records data, the omnidirectional image capturing device is configured to capture an omnidirectional image at the same time, and is connected to the omnidirectional image capturing unit; A first information recording unit that stores an image captured by the unit, a first transmitting and receiving unit that is connected to the first information recording unit, and that transmits and receives data to and from the omnidirectional image monitoring device; It is connected to the first information recording unit, determines whether there is an intruder in a monitoring area based on an image captured by the omnidirectional imaging unit, and transmits the image from the first transmission / reception unit. An image determining unit for performing control, and the image determining unit An omnidirectional photographing unit, which is arranged at a position having a predetermined relationship with the camera, and is omnidirectional. A focusing means for focusing an image on the camera. 2. The image determination unit determines whether there is an intruder in a monitoring area based on a difference image between two temporally consecutive images captured by the omnidirectional imaging unit.
The monitoring system according to claim 1. 3. The omnidirectional image photographing apparatus further includes an external sensor for detecting an intruder from outside, wherein the image determining unit is connected to the first information recording unit and the external sensor, The monitoring system according to claim 1, wherein an intruder from outside is detected based on an image captured by the azimuth imaging unit and an output of the external sensor. 4. The omnidirectional image monitoring device transmits the image, the omnidirectional image monitoring device transmits and receives data to and from the omnidirectional image capturing device, A second information recording unit for recording, connected to the second transmission / reception unit and the second information recording unit, and an image received by the second transmission / reception unit is stored in the second information recording unit together with a time. The monitoring system according to any one of claims 1 to 3, further comprising an image determining unit that records the image. 5. The monitoring system according to claim 4, wherein the image determining unit records the image in the second information recording unit only when a change has occurred in the image. 6. The monitoring system according to claim 5, wherein when a change occurs in the image, the image determination unit records only the image of the portion where the change has occurred in the second information recording unit.