JP2000083243A - Image pickup device, image pickup system, imaging control method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image pickup system that efficiently and accurately conducts monitoring operation. SOLUTION: A control means 130 controls an imaging range by image pickup means 110, 120 so that the same area is not imaged by them to the utmost and all target points in the imaging area are imaged. Furthermore, in the case of detecting a target point where a change takes place, the control means 130 controls them so that the imaging range by the image pickup means 110 covers the imaging range including the target point at which the change is detected and the other imaging means 120 covers the imaging range including other target points than the target point above. Thus, the image pickup means 110, 120 provide a video image of an area where a change takes place and a video image of the other area where no change takes place.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to, for example, an optical camera parameter at the time of photographing (hereinafter, "optical parameter").
) And geometric camera parameters (hereinafter, referred to as “geometric parameters”) used in an imaging system that can be freely controlled by a computer, an imaging system, an imaging control method, and a method for implementing the same. The present invention relates to a storage medium in which the processing steps are stored in a computer-readable manner.

[0002]

2. Description of the Related Art Conventionally, the following two methods have been used to perform a wide range of monitoring work. The first method is to arrange a monitoring camera for each location of interest, and the second method is to use a monitoring camera capable of capturing a wide field of view.

[0003]

However, the above-mentioned first and second methods used conventionally have the following problems. First, the first method of arranging cameras for each location of interest requires an increase in the number of cameras as the number of locations of interest increases, resulting in an increased economic burden. On the other hand, in the second method using a camera capable of capturing a wide field of view, for example, even if an abnormality occurs in a certain area in the monitoring area, the abnormal part is the entire captured image (the entire monitoring area) It is only a very small area. Therefore, the details of the anomaly cannot be observed, which is a very inconvenient problem for the purpose of confirming the details of the anomaly. In order to solve this problem, it is conceivable to provide the camera with an optical zoom function to zoom up and display the area where the abnormality has occurred,
The problem remains that other areas cannot be monitored during zoom-in.

Accordingly, the present invention has been made to eliminate the above-mentioned disadvantages, and an imaging apparatus, an imaging system, an imaging control method, and an imaging control method capable of performing an efficient and accurate monitoring operation. It is an object of the present invention to provide a storage medium in which processing steps to be performed are stored in a computer-readable manner.

[0005]

For such a purpose,
According to a first aspect of the present invention, a plurality of image pickup units including at least one image pickup unit capable of changing an image pickup range, a setting unit for setting a plurality of points of interest in an image picked up by the plurality of image pickup units, Detecting means for detecting changes at a plurality of points of interest set by the means; and control means for controlling the imaging range of the imaging means capable of changing the imaging range based on the detection result of the detecting means. It is characterized by the following.

According to a second aspect, in the first aspect,
All of the plurality of imaging units can change the imaging range, and the control unit controls each imaging range of the plurality of imaging units based on a detection result of the detection unit.

[0007] A third invention is the above-mentioned first invention, wherein:
The plurality of imaging units include an imaging unit having a fixed imaging range capable of imaging all of the plurality of points of interest.

[0008] In a fourth aspect based on the first aspect,
The image processing apparatus further includes a signal processing unit that performs enlargement processing on a video signal obtained by the imaging unit whose imaging range is controlled by the control unit.

A fifth aspect of the present invention is based on the fourth aspect,
The signal processing means performs enlargement processing on a part of the video signal.

In a sixth aspect based on the first aspect,
The control means has a predetermined interface for controlling the imaging means capable of changing the imaging range.

[0011] In a seventh aspect based on the first aspect,
The detecting means performs a predetermined signal processing on the video signal obtained by the plurality of imaging means to detect a movement in the image, thereby detecting a change at a plurality of points of interest set in the image. It is characterized by automatic detection.

According to an eighth invention, in the above-mentioned seventh invention,
The predetermined signal processing includes at least image processing using a background difference method or an inter-frame difference method.

According to a ninth aspect, in the first aspect,
The control means determines the imaging means for controlling the imaging range using a predetermined evaluation function.

In a tenth aspect based on the first aspect, the control means determines the imaging range of the imaging means for controlling the imaging range based on the positional information of the plurality of points of interest. And

According to an eleventh aspect of the present invention, an imaging device, a control device for controlling an imaging operation of the imaging device, and a display device for displaying a video signal obtained by imaging with the imaging device on a screen are connected. An imaging system, wherein the imaging apparatus has at least one imaging unit capable of changing an imaging range,
The control device includes: a setting unit configured to set a plurality of points of interest in an image captured by the imaging device; a detection unit configured to detect a change in the plurality of points of interest set by the setting unit; Control means for controlling the imaging range of the imaging means capable of changing the imaging range based on the detection result, and the display device individually converts each video signal obtained by imaging with the imaging device. It is displayed on a screen.

In a twelfth aspect based on the eleventh aspect, all of the plurality of imaging means included in the imaging device can change an imaging range, and the control means is configured to perform a control based on a detection result of the detection means. And controlling each imaging range of the plurality of imaging means.

According to a thirteenth aspect, in the eleventh aspect, the imaging device includes imaging means having a fixed imaging range capable of imaging all of the plurality of points of interest.

In a fourteenth aspect based on the eleventh aspect, the control device further comprises a signal processing means for performing enlargement processing on a video signal obtained by the imaging means whose imaging range is controlled by the control means. It is characterized by having.

In a fifteenth aspect based on the fourteenth aspect, the signal processing means performs an enlargement process on a part of the video signal.

In a sixteenth aspect based on the eleventh aspect, the control means has a predetermined interface for controlling the imaging means capable of changing the imaging range.

In a seventeenth aspect based on the eleventh aspect, the detecting means performs a predetermined signal processing on the video signal obtained by the image pickup device to detect a motion in the image. It is characterized in that changes at a plurality of points of interest set in an image are automatically detected.

In an eighteenth aspect based on the seventeenth aspect, the predetermined signal processing includes image processing using at least a background difference method or an inter-frame difference method.

In a nineteenth aspect based on the eleventh aspect, the control means includes an imaging means for controlling an imaging range.
The determination is made using a predetermined evaluation function.

In a twentieth aspect based on the eleventh aspect, the control means determines the imaging range of the imaging means for controlling the imaging range based on the position information of the plurality of points of interest. And

A twenty-first invention is an imaging control method for imaging an arbitrary image with a plurality of cameras including at least one camera whose imaging range can be changed, wherein a plurality of points of interest are set in the image. Setting step, a detecting step of detecting changes at a plurality of points of interest set in the setting step, and, based on a detection result in the detecting step, changing an imaging range of the camera in which the imaging range can be changed. And a control step of controlling.

In a twenty-second aspect based on the twenty-first aspect, all of the plurality of cameras are capable of changing an imaging range, and the control step is performed based on a detection result in the detection step. Of each camera is controlled.

In a twenty-third aspect based on the twenty-first aspect, the plurality of cameras include a camera having a fixed imaging range capable of imaging all of a plurality of points of interest.

According to a twenty-fourth aspect, in the twenty-first aspect, the method further comprises a signal processing step of performing an enlarging process on a video signal obtained by a camera whose imaging range is controlled by the control step. I do.

In a twenty-fifth aspect based on the twenty-fourth aspect, the signal processing step includes a step of performing an enlarging process on a part of the video signal.

In a twenty-sixth aspect based on the twenty-first aspect, the control step includes a step of controlling the imaging range using a predetermined interface for controlling a camera capable of changing the imaging range. It is characterized by the following.

In a twenty-seventh aspect based on the twenty-first aspect, the detecting step performs predetermined signal processing on video signals obtained by the plurality of cameras to detect a motion in the image. A step of automatically detecting changes at a plurality of points of interest set in the image.

In a twenty-eighth aspect based on the twenty-seventh aspect, the predetermined signal processing includes image processing using at least a background difference method or an inter-frame difference method.

In a twenty-ninth aspect based on the twenty-first aspect, the control step includes a step of determining a camera for controlling an imaging range by using a predetermined evaluation function.

In a thirtieth aspect based on the twenty-first aspect, the control step includes a step of determining the imaging range of a camera for controlling the imaging range based on the position information of the plurality of points of interest. It is characterized by.

A thirty-first aspect of the present invention is a storage medium in which the processing steps of the imaging control method according to any one of claims 21 to 30 are readable by a computer.

[0036]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) The present invention provides, for example,
It is applied to a monitoring system 100 as shown in FIG.

As shown in FIG. 1, the monitoring system 100 includes a plurality of cameras (here, two cameras 110 and 120) capable of externally referencing and externally controlling the optical parameters and geometric parameters at the time of photographing. And the camera control device 130 are provided. Here, for the sake of simplicity, it is assumed that “zoom value” is used as an optical parameter, and “pan value” and “tilt value” representing a shooting direction are used as geometric parameters. Then, in the monitoring system 100, when one of the cameras is capturing an area (hereinafter, this operation is also referred to as “monitoring”), the other camera monitors the other area so that the monitoring ranges do not overlap. Control.
Hereinafter, the hardware configuration and principle of the monitoring system 100,
And the processing procedure will be specifically described.

Incidentally, the optical parameters and the geometric parameters are not limited to the above-mentioned "zoom value", "pan value" and "tilt value", and other camera parameters are used or used in combination. You may do so. Further, here, two cameras are used and one camera control device is used. However, the present invention is not limited to this, and a larger number of cameras may be used.

1. Hardware Configuration (see FIG. 1 above) The monitoring system 100 includes the two cameras 110 and 120 and the camera control device 130 as described above. The camera control device 130 has a pointing device 150, a keyboard 140, and a monitor 1
60 and 170 are connected.

Each of the cameras 110 and 120 has the same configuration. The camera 110 includes a camera unit 111 that captures a subject by an imaging system and outputs a captured image (video signal), and an optical parameter control unit 112 that controls optical parameters (“zoom value”) of the camera unit 111.
And a geometric parameter control unit 1 for controlling geometric parameters (“pan value” and “tilt value”) of the camera unit 111
13 is provided. Similarly, the camera 120 includes a camera unit 121, an optical parameter control unit 122, and a geometric parameter control unit 123.

The camera unit 111 directly converts a video signal (a video signal according to the NTSC system or the YC separation system) obtained by shooting under the control of the optical parameter control unit 112 and the geometric parameter control unit 113 into an external device. Monitor 1
Output to 60. Similarly, the camera unit 121
A video signal obtained by shooting under the control of the optical parameter control unit 122 and the geometric parameter control unit 123 is directly output to the external monitor 170.

Optical parameter control units 112, 122,
And each unit of the geometric parameter control units 113 and 123,
It is connected to the camera control device 130 via a control signal line. The communication of the control signal via the control signal line is performed between each unit and the camera control device 130, so that the camera control device 130 sets, acquires, and sets optical parameters and geometric parameters to each unit. Control such as optimization becomes possible. Note that, as the control signal line, for example, RS-232C, parallel IO, or the like can be used, and the standard of the communication method is not limited.

Optical parameter control units 112 and 122
Also has a function of automatically controlling optical parameters for the entire captured images in the camera units 111 and 121. This function includes a function of automatically obtaining a point image and a function of automatically adjusting the brightness balance of an image. Such a function can be easily realized by using, for example, a camera module built in a home video camera. Here, whether the adjustment (control) of the optical parameter is performed from the camera control device 130 or the automatic control function of the optical parameter control units 112 and 122 is determined by the camera via the control signal line The instruction can be given from the control device 130.

The optical parameter control units 112, 1
Reference numeral 22 has a function of controlling and referencing the zoom of the camera units 111 and 121. With this function, for example, the zoom value specified by the camera
11 or 121, and information on the current zoom value can be provided to the camera control device 130.

Geometric parameter control units 113 and 123
Also has a function of controlling and referencing the shooting direction of the camera units 111 and 121. With this function, for example, the camera 11
It is possible to change the shooting direction of the camera 1 and 121, and to provide the camera control device 130 with information on which direction is currently the shooting direction.

On the other hand, the camera control device 130 is realized by a general-purpose computer, and stores a CPU 131 for controlling the operation of the entire system, a RAM 132 including a work area of the CPU 131, a processing program and various data for various operation controls. An input / output (I / O) circuit 135 for communicating with the ROM 133, the secondary storage unit 134, the cameras 110 and 120 to be stored, and a communication interface (I / O) for communicating with the outside via a network.
F) A circuit 136 is provided. These circuits are connected by a bus 137 so as to exchange data with each other. The bus 137 has a pointing device 1 such as a keyboard 140 and a mouse.
50 and monitors (video monitors) 160 and 170 are also connected.

In such a camera control device 130, for example, a control command from the outside (remote location) is received by the communication I / F circuit 136 via the network 180. A control signal according to this control command is transmitted to the I / O circuit 13
5 through the control signal line, the optical parameter control unit 112 and the geometric parameter control unit 11 of the camera 110.
3 and the optical parameter control unit 122 and the geometric parameter control unit 123 of the camera 120. Thus, the optical and geometric parameters of the cameras 110, 120 are controlled by the control signals thus provided. Similarly, for a control command input from the keyboard 140 or the pointing device 150, a control signal according to the control command is transmitted to the optical parameter control unit 112 and the geometric parameter control unit 113 of the camera 110 and the camera 120. The optical parameters and the geometric parameters of the cameras 110 and 120 are provided to the optical parameter control unit 122 and the geometric parameter control unit 123.

2. Basic Principle Hereinafter, the basic principle of the present embodiment will be described with reference to FIGS.

FIG. 2 shows two cameras 110 and 12
0 indicates that the monitoring area is being imaged.
In the figure, “203” indicates an imaging area being monitored, and “A”
_{, 1} , A ₂ , A ₃ ,... ”Indicate points of interest scattered in the imaging region 203, that is, monitoring points (hereinafter, also referred to as“ region of interest ”).

FIG. 3 shows the image pickup area 2 shown in FIG.
3 shows a coordinate system corresponding to the shooting directions (pan direction and tilt direction) of the cameras 110 and 120. The p-axis in the figure represents displacement in the pan direction (pan value), and the t-axis represents displacement in the tilt direction (tilt value). For example, a value such as −60 ° ≦ p ≦ 60 ° and −45 ° ≦ t ≦ 45 ° is taken, which is based on the cameras 110 and 1 to be used.
It represents 20 controllable ranges and depends on the performance of the camera.

FIG. 4 is a correspondence table showing the possible zoom values z of the cameras 110 and 120 and the viewable angles of view corresponding to the zoom values z by the horizontal view angle dp and the vertical view angle dt. This depends on the performance of the imaging system provided in the cameras 110 and 120.
When the zoom value z at 0 is “2.0”, the horizontal angle of view dp is set to “82 °” and the vertical angle of view d
Reference is made such that a range where t is “53 °” can be photographed.

In this case, since the cameras 110 and 120 each having the same configuration are used, one type of correspondence table shown in FIG. 4 may be prepared, but when cameras having different performances are used. Prepares this correspondence table by the number of the types. Considering that each value in the correspondence table takes a continuous value, the correspondence table in FIG. 4 has a function "min (...)" for finding the minimum value and is replaced by the following function. It is also possible. z = PTtoZ (dp, dt) = min (tan (120 ° / 2) / tan (dp / 2), tan (90 ° / 2) / tan (dt / 2)) dp = ZtoP (z) = 2 tan ^-1 (tan (120 ° / 2) / z) dt = ZtoT (z) = 2tan- ¹ (tan (90 ° / 2) / z)

FIG. 5 shows the monitoring points A scattered in the monitored imaging area 203 as shown in FIG.
₁ , A ₂ , A ₃ ,... (Nos. 1, 2, 3,
..) Shows a table in which attributes (parameters) of positions and ranges are set (held). For example, NO.
The monitoring point A ₁ shown in FIG. ₁ has p = 52 ° and t = 20 ° in the coordinate system of FIG. 3 and a zoom value z = 8.5.
At the position and range. Since each of these parameters depends on the shooting position of each of the camera 110 and the camera 120, here, two types of tables shown in FIG. 5 are prepared.

FIG. 6 shows what areas of the two cameras 110 and 120 used here are actually monitoring (photographing) in the image pickup area 203. Here, the optical parameter control unit 112 and the geometric parameter control unit 113 included in the camera 110, and the optical parameter control unit 122 and the geometric parameter control unit 123 included in the camera 120 are each a camera unit. An optical parameter (zoom value) and a geometric parameter (pan value, tilt value, hereinafter, these are also referred to as “photographing directions”) are controlled by a control signal output from the control device 130. At this time, the control signal output from the camera control device 130 is
0 and the camera 120 shoots all the attention areas, and
"Zoom value", "Pan value", and "Tilt value" parameters for maximizing and displaying the monitoring point group assigned to the user without capturing the same area as other cameras as much as possible. . Therefore,
In FIG. 6, the monitoring points A ₁ , A ₂ , A ₃ ,.
.. With respect to the entire imaging area 203 where A ₈ is scattered, the monitoring area A _{1 to} A ₄ is included in the imaging area 208 of the camera 110, and the monitoring points A _{5 to} A ₈ are included in the imaging area 209 of the camera 120. Have been.

Therefore, how each of the cameras 110 and 120 is controlled will be described more specifically.

First, when the assigned monitoring points A ₁ , A ₂ ,..., _An are given to the camera control device 130, the camera control device 130 calculates the coordinates of the monitoring points from the center coordinates of these monitoring point groups. The shooting direction (p _c , t _c ) is calculated by the following equation (1). p _c = (max (p (A ₁ ), p (A ₂ ), ..., p (A _n )) + min (p (A ₁ ), p (A ₂ ), ..., p (A _n )))) / 2 t _c = (max (t (A ₁ ), t (A ₂ ), ..., t (A _n )) + min (t (A ₁ ), t (A ₂ ), ··, t (A _n ))) / 2 (1) In the above equation (1), “max (...)” and “min”
(...) "is a function for calculating the maximum value and the minimum value." P (...) "and" t (...) "
These are parameters relating to the position of the monitoring point, and can be obtained by referring to the table shown in FIG.

The zoom value z is calculated by the following equation (2). z = PTtoZ (2 · max ( max (p (A 1), p (A 2), ···, p (A n)) + ZtoP (z (argmax (p (A 1), p (A 2) , ・ ・ ・, P (A _n )))) / 2-p _c , p _c -min (p (A ₁ ), p (A ₂ ), ・ ・ ・, p (A _n ))-ZtoP (z (argmin (p (A ₁ ), p (A ₂ ), ・ ・ ・, p (A _n ))))) / 2), 2 ・ max (max (t (A ₁ ), t (A ₂ ), ・)・ ・, T (A _n )) + ZtoT (z (argmax (t (A ₁ ), t (A ₂ ), ・ ・ ・, t (A _n )))) / 2-t _c , t _c -min (t (A ₁ ), t (A ₂ ), ..., t (A _n ))-ZtoT (z (argmin (t (A ₁ ), t (A ₂ ), ..., t (A _n )))) / 2)) (2) In the above equation (2), “argmax (...)” and “
argmin (...) "is a function that returns" A _i "giving the maximum value and the minimum value, and" PTtoZ (...
•) ”,“ ZtoP (...) ”, and“ ZtoT (•
..) "is a function equivalent to the correspondence table shown in FIG.

According to the above equations (1) and (2), the parameters (p _c ,
t _c , z) are transmitted to the optical parameter control units 112 and 122 and the geometric parameter control units 113 and 123 provided in the cameras 110 and 120. In this way, appropriate shooting directions and zoom values are instructed to the cameras 110 and 120. That is, the photographing area 208 of the camera 110 shown in FIG.
The camera control device 130 determines the shooting area 209 of the camera 120 by using the above equations (1) and (2) for the monitoring points A _{1 to} A _{4 and} the above equations (1) and (2) for the monitoring points A _{5 to} A _8. The parameters (p _c , t _c , z) obtained by executing (2) are transferred to the camera 110 and the camera 12
This is realized by transmitting each of them to 0.

FIG. 7 shows an example of the monitoring state at a different time point (such as when an abnormality occurs at the monitoring points A ₁ and A ₂ ) with respect to FIG. “208 ′” in the figure indicates the shooting area of the camera 110 at this time, and “209 ′” indicates the camera 12 at this time.
0 shows a shooting area. As shown in FIG. 7, when the camera 110 is zoomed in to examine the detailed status of the monitoring points A ₁ and A ₂ are other camera 120, all the rest of the monitoring points A ₃ to A ₈ The shooting direction and zoom have been changed to include.

To realize such operation control, an example
For example, the camera control device 130 always keeps the current camera 11
I / O circuit 1
35, which is referred to. And abnormal
Camera control according to the situation at the time, such as when
The control device 130 has the optical parameters as described above.
And the geometric parameters are calculated and used as control signals.
Provided to cameras 110 and 120. Thereby, the camera 1
Optical and geometric parameters of 10, 120
Is controlled to be changed. That is, as shown in FIG.
Shooting area 208 'of the camera 110 and the camera 12
The camera control device 130 monitors the shooting area 209 ′ of 0.
Viewing point A₁And A _TwoEquation (1) above and
(2) and monitoring point A_Three~ A₈The above formula for
Each parameter (p obtained by executing (1) and (2)_c,
t_c, Z) to camera 110 and camera 120
It is realized by transmitting each. In this way, the turtle
The camera controller 130 controls the optical
Always reference and control parameters and geometric parameters
Thus, the operation control as shown in FIG. 7 is realized.

As described above, the monitoring system 100 externally references and externally controls the optical parameters and geometric parameters of a plurality of cameras at a certain point in time, and an abnormality occurs in either the camera 110 or the camera 120. In such a case, an optical parameter and a geometric parameter for photographing details of the abnormality occurrence area are given to a camera monitoring (photographing) the abnormality occurrence area. The optical direction and the zoom of the cameras 110 and 120 are changed by giving an optical parameter and a geometrical parameter such that there is no unphotographed area in the entire imaging area. Thereby, without increasing the number of cameras to be used, it is possible to observe the abnormality occurrence area in detail and also to reliably observe other areas,
Therefore, an efficient monitoring system can be constructed.

3. Basic Processing Hereinafter, the flow of the basic processing in the present embodiment will be described with reference to FIG. In the camera control device 130, for example, a processing program according to the flowchart shown in FIG. 8 is stored in advance in the ROM 133, and this processing program is read out and executed by the CPU 131. Is performed.

When this processing program is executed, first,
The camera control device 130 sets a point (monitoring point) to be monitored in the imaging area as an initial setting (step S301). As the setting method, various methods such as automatic setting, manual setting, and presetting can be considered, but are not particularly limited. The set monitoring points are
The position and range information processed by the CPU 131 is stored in the RAM 132 as a table as shown in FIG.
And so on.

Next, the CPU 131 performs initial settings as follows:
The areas to be monitored (photographed) by the camera 110 and the camera 120 are determined (step S302). At this time, CPU
131, the camera 110 and the camera 120, as described above, the camera 110 and the camera 120 as little as possible photographing the same area, and, so as to shoot all the region of interest, the parameters (p _c, t _c , z) are calculated respectively. These parameters are stored in the optical parameter control unit 112 and the geometric parameter control unit 113 of the camera 110, and the optical parameter control unit 12 of the camera 120.
2 and the respective components of the geometric parameter control unit 123, whereby the shooting direction and zoom of the camera 110 and the camera 120 are controlled. The specific control method of the cameras 110 and 120 (calculation method of the parameters (p _c , t _c , z) and the like) is as described above, and thus the detailed description is omitted. Similarly, in steps S305 to S308 described below, the cameras 110 and 120 are controlled by the camera control device 130. However, the details of this control method are as described above, and a description thereof will be omitted.

Thereafter, the camera 110 and the camera 12 are actually
Monitoring with 0 is started. Camera 110 and camera 12
The video signal obtained at 0 is supplied to the monitors 160 and 170 connected to the camera control device 130, and displayed on the screen. The user can use these monitors 16
Observe the display screens 0 and 170. And step S
Until an abnormality is found by the repetition loop processing (abnormality detection loop processing) in steps 303 and S310, or
The observation is performed in this state until a command indicating the end of the present process is input from the user via the pointing device 150 or the keyboard 140.

Therefore, when the user discovers an abnormality, for example, the user inputs a command to that effect using the pointing device 150 or the keyboard 140.
In addition to the above-described human observation as described above, various methods, such as automatic moving object detection (motion detection) using an image processing technique called a so-called background subtraction method or an inter-frame difference method, may be used for the abnormality detection method. There are various methods, but again there is no particular limitation. For example, when a method using the above-described image processing technique is adopted, the camera control device 130
The video signals of the cameras 110 and 120 are fetched from the / O circuit 135, and predetermined image processing is performed by the CPU 131.
Try to detect the occurrence of abnormality.

The CPU 131 determines whether an error has occurred based on a command input from the pointing device 150 or the keyboard 140 (or by the above-described image processing or the like) (step S303).

If no abnormality has occurred as a result of the determination in step S303, the CPU 131 determines whether or not a command indicating the end of this process has been input from the user by the pointing device 150 or the keyboard 140 (step S310). . If the result of this determination is that this processing has ended, this processing ends.
The process returns to step S303, and returns to the abnormality detection loop processing again.

On the other hand, as a result of the determination in step S303, if an abnormality has occurred, the CPU 131 determines which of the cameras 110 and 120 is monitoring the abnormality in the area monitored by the camera. (Step S3
04).

As a result of the determination in step S304, the camera 1
When an abnormality occurs in the 10 monitoring areas, the CPU 131
For the optical parameter control unit 112 and the geometric parameter control unit 113 of the camera 110,
A predetermined instruction (parameter (p _c , t _c , z)) is given as a control signal via the I / O circuit 135. The control signal is a signal indicating a photographing direction and a zoom value for photographing details of the attention area where the abnormality has occurred. The camera 110 receiving this changes the photographing direction and the zoom value to photograph the details of the attention area. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S305).

The CPU 131 is connected to the other camera 12.
0 to the optical parameter control unit 122 and the geometric parameter control unit 123 provided with a predetermined instruction (parameter (p _c , t _c , z)) as a control signal.
Applied through the O-circuit 135. This control signal is a signal indicating a shooting direction and a zoom value such that there is no region of interest that is not shot in the entire image sensing region. The camera 120 receiving this changes the shooting direction and the zoom value, and shoots an area according to the instruction. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S306).

After a predetermined action is taken against the above-mentioned abnormality (step S309), the CPU 131 determines whether or not a command indicating the end of this process is input from the user by the pointing device 150 or the keyboard 140. A determination is made (step S310). If the result of this determination is that this processing has ended, this processing ends. If not, processing returns to step S303 and returns to the abnormality detection loop processing again.

As a result of the determination in step S304, the camera 1
When an abnormality occurs in the monitoring area of No. 20, the CPU 131 also executes the processing in the same manner as in steps S307 and S308.
Gives predetermined instructions (parameters (p _c , t _c , z)) as control signals to the cameras 120 and 110 (steps S307 and S308). Then, after a predetermined measure is taken for the above-described abnormality (step S30)
9) If there is an instruction from the user to end the present process, the present process ends. Otherwise, the process returns to step S303 and returns to the abnormality detection loop process again (step S310). Steps S307 and S3
Each process of 08 is a process in which the camera 110 and the camera 120 are replaced in each of the processes of steps S305 and S306, and a detailed description thereof will be omitted.

(Second Embodiment) The present invention provides, for example,
It is applied to a monitoring system 400 as shown in FIG.

The surveillance system 400 has the same configuration as the surveillance system 100 shown in FIG. 1. In addition to this configuration, a fixed camera 401 (hereinafter, referred to as a “wide-field fixed camera”) 401 capable of photographing a wide area. Is provided. The wide-view fixed camera 401 is a camera such as the camera 110 and the camera 120 that does not necessarily need to change the shooting direction and the zoom. For this reason, the wide-view fixed camera 401 includes only the camera unit 402 that captures an object by an imaging system and outputs a captured image (video signal).
A monitor 403 for displaying a video signal output from the wide-view fixed camera 401 on a screen is provided by the camera control device 130.
The bus 137 is connected to the

As described above, in this embodiment, a plurality of cameras (here, two cameras 110 and 120) capable of externally referencing and externally controlling the optical parameters and geometric parameters at the time of photographing are used. ), Their camera control devices 130, and one wide-view fixed camera 401.
This is a configuration including: Then, one wide-view fixed camera 401 is used as a camera for always monitoring (photographing) the entire imaging area, and two cameras 110 and 12 are used.
0 is used to collect more detailed information on a region of interest in the imaging region when an abnormality occurs. Further, as in the first embodiment, the camera control device 13
By 0, control is performed so as to monitor other areas so that each monitoring range does not overlap. At this time, the coordinate system as shown in FIG. 3, the correspondence table between the zoom value and the viewable angle of view as shown in FIG. 4, and the position and range of each monitoring point as shown in FIG. Is used.

In the monitoring system 400 shown in FIG. 9, the parts operating in the same manner as the monitoring system 100 shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted. Here, only the differences from the first embodiment will be specifically described. Similarly to the first embodiment, for the sake of simplicity, the optical parameters are “zoom value”, and the geometric parameters are “pan value” and “tilt value” representing the imaging direction. Again, the present invention is not limited to “zoom value”, “pan value”, and “tilt value”, and other camera parameters may be used or may be used in combination.

Hereinafter, the principle and the processing procedure of the monitoring system 400 will be specifically described.

1. Basic Principle Hereinafter, the basic principle of the present embodiment will be described with reference to FIGS.

FIG. 10 shows one wide view fixed camera 4.
01 and two cameras 110 and 120 are monitoring (photographing) the imaging area. “411” in the figure indicates an area monitored by the two cameras 110 and 120 (the areas 208 and 209 in FIG. 6 described above).
, “A ₁ , A ₂ , A ₃ ,...” Indicate points of interest (monitoring points) scattered in the monitored imaging area 411. “413” indicates an area (field of view) constantly monitored by the wide-field fixed camera 401.

FIG. 11 shows a state in which an abnormality is detected in a certain area (here, monitoring point A ₁ ) while the wide-field fixed camera 401 is monitoring the imaging area 411. As described above, when an abnormality is detected, _one of the two cameras 110 and 120 (here, for example, the camera 110) is used to check the detailed status of the monitoring point A1. in contrast, since the control signal from the camera controller 130 (the above equation for monitoring point a ₁ (1) and (parameters obtained by the execution of 2)) is given, the monitoring points a ₁ is a zoom-up display You. The shooting area of the camera 110 at this time is indicated by “415” in the figure.

[0083] FIG 12 is (as an example here, the monitoring point A ₁ of FIG. 11) previously abnormality detected region of interest before the predetermined treatment is completed for the abnormalities, new abnormal (here, the abnormality of the monitoring points a ₃₎ indicates the condition detected. In this case, in order to investigate the detailed status of the monitoring points A _3, a state where the control is not performed for collecting detailed information by the camera control unit 130 (hereinafter, referred to as "standby state") longer in For one camera 120, the camera control device 1
By given control signal 30 (parameters obtained by the execution of the above formula for the monitoring point A ₃ (1) and (2)), the monitoring points A ₃ is zoom-displayed. The shooting area of the camera 120 at this time is indicated by “
417 ".

FIG. 13 shows a case where both the two cameras 110 and 120 are not in a standby state (here, as an example, the case of FIG. 12), another attention area (here, new anomaly monitoring point to a ₆₎ indicates a state of being detected. In this case, when a new abnormal monitoring point A ₆ is detected, the camera 110
And in any of the camera of the camera 120 relates Should shoot monitoring point A _6, is determined by the camera control unit 130. This determination, for example, if the camera 120 photographs the monitoring points A _6, the camera control unit 13
0, with respect to the optical parameter control unit 122 and the geometric parameter control unit 123 of the camera 120, new abnormality detected watchpoints A ₆ including imaging range such shooting direction (pan, tilt) and zoom (To provide optical and geometric parameters). That is, the camera controller 130 monitors point A ₃ to A ₇ above formula (1) and the camera 120 a parameter obtained by performing a (2) for
Give to. Thereby, the shooting range of the camera 120 is changed. “419” in the figure indicates the shooting area of the camera 120 updated under the control of the camera control device 130.

FIG. 14 shows an area where an abnormality is detected.
Here, as an example, the monitoring point A in FIG._ThreeToss
A) predetermined action is taken for the abnormality of
The state when the state is released is shown. In this case, the supervisor
Viewing point A_ThreeProvided in the camera 120 that was shooting the
Optical parameter control unit 122 and geometric parameters
The camera control unit 123 sends
Still abnormal in the area that Mera 120 was shooting
Shooting method that includes all areas that have not been released
Instructions to change to orientation (pan, tilt) and zoom
Given (optical and geometric parameters are
Given). That is, the camera control device 130
Viewing point A_ThreeMonitoring point A excluding _Five~ A₇about
Parameters obtained by executing the above equations (1) and (2)
Data to the camera 120. Thereby, the camera 12
The shooting range of 0 is changed. “420” in the figure is the turtle
Camera 12 updated under the control of the camera control device 130
0 shows a shooting area.

As described above, the surveillance system 400 is a single wide-field fixed camera 4 that constantly monitors the entire imaging area.
By further providing 01, it is possible to avoid omission of detection of abnormality occurrence in other areas that occurs while examining the detailed situation of abnormality with the two cameras 110 and 120 that can control the shooting direction and zoom. Is configured. Thereby, a more efficient monitoring system can be constructed.

2. Basic Processing Hereinafter, the flow of the basic processing according to the present embodiment will be described with reference to FIGS. FIG. 15 is a flowchart showing a process when an abnormality is detected in the imaging region. FIG. 16 is a flowchart showing a process in which a predetermined process is performed on the abnormality in the attention region in which the abnormality has been detected. 9 is a flowchart illustrating a process when an abnormal state is released. These processes take the form of parallel execution.
As in the first embodiment, in the camera control device 130, for example, the ROM
5 and a processing program according to the flowchart shown in FIG. 16 are stored in advance.
The operation described below is performed by being read and executed by U131.

2-1. Processing when an abnormality is detected in the imaging area (see FIG. 15 above)

When this processing program is executed, the wide-view fixed camera 401 monitors (photographs) the entire imaging area as described above. The video signal obtained by this shooting is supplied to the monitor 403, where it is displayed on the screen.
The user observes the display screen of the monitor 403 (Step S431). Also in the cameras 110 and 120, monitoring (photographing) is performed according to the control of the camera control device 130 as described above, and the video signals obtained thereby are displayed on the monitors 160 and 170 on the screen. The user observes the display screens of these monitors 160 and 170. Then, steps S431, S432, and S4
Iterative loop processing by 40 (error detection loop processing)
Thus, the observation is performed in this state until an abnormality is found, or until a command indicating the end of the present process is input from the user using the pointing device 150 or the keyboard 140.

Therefore, when the user discovers an abnormality, for example, the user inputs a command to that effect using the pointing device 150 or the keyboard 140.
Note that, similarly to the first embodiment, there are various methods for finding an abnormality other than observation by a human, such as automatic moving object detection using an image processing technique. Do not.

The CPU 131 determines whether an error has occurred based on a command input from the pointing device 150 or the keyboard 140 (or by the above-described image processing or the like) (step S432).

If the result of determination in step S 432 is that no abnormality has occurred, the CPU 131 determines whether or not a command indicating the end of this process has been input by the user using the pointing device 150 or the keyboard 140 (step S 440). . If the result of this determination is that this processing has ended, this processing ends.
The process returns to step S331, and returns to the abnormality detection loop processing again.

On the other hand, if the result of the determination in step S303 indicates that an abnormality has occurred, the CPU 131 is in a standby state (a state in which control for collecting detailed information is not performed by the camera controller 130). It is determined whether or not this is the case (step S433).

As a result of the determination in step S433, the camera 1
In a case where the CPU 10 is in the standby state, the CPU 131 controls the optical parameter control unit 112 provided in the camera 110.
And a predetermined instruction (parameter (p _c , t _c , z)) as a control signal to the geometric parameter control unit 113 via the I / O circuit 135. The control signal is a signal indicating a photographing direction and a zoom value for photographing details of the area where the abnormality has occurred. The camera 110 receiving this changes the shooting direction and the zoom value, and shoots details of the attention area where the abnormality is detected. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S434). afterwards,
The CPU 131 determines whether or not a command indicating the end of this process has been input by the user using the pointing device 150 or the keyboard 140 (step S44).
0). If the result of this determination is that this processing has ended, this processing ends; otherwise, step S431
And returns to the abnormality detection loop processing again.

As a result of the determination in step S433, the camera 1
If the camera 10 is not in the standby state, the CPU 131 determines whether the camera 120 is in the standby state (step S435).

As a result of the determination in step S435, the camera 1
If the CPU 20 is in the standby state, the CPU 131 instructs the optical parameter control unit 122 and the geometric parameter control unit 123 included in the camera 120 to perform a predetermined instruction (parameter (p _c , t _c , z)) are given as control signals (step S436). Thereafter, if there is an instruction from the user to end the present process, the present process ends, otherwise, the process returns to step S431 and returns to the abnormality detection loop process again (step S440). Step S4
The processing in step S434 is performed by the camera 11 in step S434.
Since the process is performed by replacing 0 with the camera 120, detailed description thereof is omitted.

As a result of the determination in step S435, the camera 1
20 is not in the standby state, that is, the camera 110
If neither the camera nor the camera 120 is in the standby state, the CP
U131 uses, for example, a predetermined evaluation function with parameters of the positions of the photographing regions of the cameras 110 and 120, the number of abnormal portions during photographing, and the like, and extracts the region of interest in which the newly detected abnormality is detected from the camera. It is evaluated which of the cameras 110 and 120 should be used for shooting (step S437).

As a result of the evaluation in step S437, camera 1
If it is determined that shooting at 10 is optimal,
U131 includes an optical parameter control unit 112 and a geometric parameter control unit 113 provided in the camera 110.
In response to a predetermined instruction (parameters (p _c , t _c ,
z)) is given as a control signal via the I / O circuit 135. This control signal indicates a photographing direction and a zoom value for adding a newly detected region of interest to the region previously photographed by the camera 110 and including all of them in the photographing range of the camera 110. Signal. The camera 110 receiving this changes the shooting direction and the zoom value, and shoots details of the area. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S438). After that, CP
The U131 determines whether or not a command indicating the end of this process has been input by the user using the pointing device 150 or the keyboard 140 (step S440).
If the result of this determination is that this processing has ended, this processing ends. Otherwise, processing returns to step S431 and returns to the abnormality detection loop processing again.

As a result of the evaluation in step S437, camera 1
If it is determined that shooting is more optimal in step S <b> 20, the CPU 131 proceeds to step S <b> 438.
A predetermined instruction (parameters (p _c , t _c , z)) is given as a control signal to the optical parameter control unit 122 and the geometric parameter control unit 123 provided in 20 (step S439). Thereafter, if there is an instruction from the user to end the present process, the present process ends, otherwise, the process returns to step S431 and returns to the abnormality detection loop process again (step S440). still,
The process in step S439 is a process in which the camera 110 and the camera 120 are replaced in the process in step S438, and thus a detailed description thereof will be omitted.

Steps S434, S436, S43
8, and the specific camera control method in S439 is as described in detail in the first embodiment. In step S437, several specific methods for determining a camera to be used for photographing are conceivable. Here, a simple example will be described. For example, the shooting direction of the camera 110 is “p = p _A , t = t _A ”, the shooting direction of the camera 120 is “p = p _B , t = t _B ”, and the newly detected attention area It is assumed that the direction of the (monitoring point) is “p = p _N , t = t _N ”. Then, (p _A -p _N ) ² + (t _A -t _N ) ² ≤ (p _B -p _N ) ² + (t _B -t
_N ) Instruct the camera 110 to be used when the relationship ² is satisfied, and to use the camera 120 otherwise. Of course, the determination method described here is an example of an extremely simple method, and an optimum determination can be expected by another method.

2-2. Processing in a case where an abnormal state is canceled by performing a predetermined treatment on a region of interest in which an abnormality has already been detected (see FIG. 16 above)

When this processing program is executed, predetermined processing is performed on the abnormality in the attention area where an abnormality is detected by executing the processing program of FIG. 15 (step S451). Then, steps S451, S452,
Until the problem for the above abnormality is solved by the repetitive loop processing in S460 and S460, or until a command indicating the end of the present processing is input from the user through the pointing device 150 or the keyboard 140, the above-mentioned state for solving the problem is maintained. A predetermined process is performed.

Therefore, when the problem is solved for a certain abnormality, the CPU 131 deletes the problem-resolved region of interest from the photographing range of the camera that photographed the region of interest in which the abnormality was detected. I do.

For example, the CPU 131 determines whether or not the camera that has photographed the attention-resolved area is the camera 110 (step S453).

As a result of the determination in step S453, the camera 1
If it is 10, the CPU 131 determines that the camera 110
It is determined whether or not the attention area in which an unsolved abnormality still exists other than the attention area in which the problem has been solved is photographed (step S453).

If the result of determination in step S453 is that an attention area in which an unresolved abnormality still exists is captured, the CP
U131 includes an optical parameter control unit 112 and a geometric parameter control unit 113 provided in the camera 110.
In response to a predetermined instruction (parameters (p _c , t _c ,
z)) is given as a control signal via the I / O circuit 135. This control signal is a signal indicating a shooting direction and a zoom value for reducing the shooting area of the camera 110 so as to include all the attention areas where unresolved abnormalities exist. The camera 110 receiving this changes the shooting direction and the zoom value, and shoots details of the area. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S455). Then, C
The PU 131 determines whether or not a command indicating the end of this process has been input by the user using the pointing device 150 or the keyboard 140 (step S46).
0). If the result of this determination is that this processing has ended, this processing ends, otherwise, step S451
To return to the loop processing.

As a result of the determination in step S453, the camera 1
When there is no attention area where an abnormality exists in the 10 imaging areas,
The CPU 131 controls the optical parameter control unit 112 and the geometric parameter control unit 1 provided in the camera 110.
13, a predetermined instruction (parameters (p _c , t _c ,
z)) is given as a control signal via the I / O circuit 135. This control signal is a signal indicating a shooting direction and a zoom value (a shooting direction and a zoom value in an initial state) for returning the camera 110 to the standby state. Camera 11 receiving this
0 changes the shooting direction and the zoom value to shoot details of the area. The video signal thus obtained is supplied to the monitor 160, where it is displayed on the screen (step S456). After that, the CPU 131 uses the pointing device 150 and the keyboard 140 to
It is determined whether or not a command indicating the end of this process has been input from the user (step S460). As a result of this determination,
If the processing has ended, the processing ends. Otherwise, the processing returns to step S451 and returns to the loop processing.

As a result of the determination in step S453, the camera 1
If it is not 10, that is, if it is the camera 120, as in steps S455 and S456, the CP
U131 is an optical parameter control unit 122 and a geometric parameter control unit 123 provided in the camera 120.
In response to a predetermined instruction (parameters (p _c , t _c ,
z)) is given as a control signal (step S458, step S459). Then, after the processing in step S458 or step S459, if there is an instruction from the user to end this processing, this processing ends, otherwise, the processing returns to step S451 and returns to the abnormality detection loop processing again. (Step S460). Step S4
58 and S459 are performed in steps S455 and S455.
Since the camera 110 and the camera 120 are replaced in each process of 456, detailed description thereof is omitted.

(Third Embodiment) The present invention provides, for example,
It is applied to a monitoring system 500 as shown in FIG.

The monitoring system 500 has the same configuration as the monitoring system 400 shown in FIG. 9, but has only one camera that can externally reference and externally control the optical parameters and geometric parameters at the time of photographing. Is different. In the monitoring system 500, the three monitors 160 and 1 are connected to the I / O circuit 135 of the camera control device 130.
70 and 403 are connected, and these monitors 160, 170 and 403 are connected to the camera 11
The video signals output from the 0 and wide-view fixed cameras 401 are not supplied directly, but are supplied via an I / O circuit 135. For this reason, the CPU 131 of the camera control device 130 edits each video signal output by the camera 110 and the wide-view fixed camera 401 to perform I / O.
It also has a function of outputting from the circuit 135.

In the monitoring system 500 shown in FIG. 17, parts operating in the same manner as in the monitoring system 400 shown in FIG. 9 are denoted by the same reference numerals, and detailed description thereof will be omitted.
Here, only the differences from the second embodiment are described.
This will be specifically described. Also, as in the first and second embodiments, for the sake of simplicity, the optical parameters are “zoom value”, and the geometric parameters are “pan value” and “tilt value” representing the imaging direction. And even here,"
It is not limited to the “zoom value”, “pan value”, and “tilt value”, and other camera parameters may be used or may be used in combination.
Although three monitors 160, 170, and 403 are connected to the I / O circuit 135 of the camera control device 130, the number of monitors is not limited.

Hereinafter, the principle and the processing procedure of the monitoring system 500 will be specifically described.

1. Basic Principle Hereinafter, the basic principle of the present embodiment will be described with reference to FIGS.

FIG. 18 shows one camera 110 and one camera 110.
This figure shows a state in which the imaging area is monitored (photographed) with two wide-field fixed cameras 401. "50" in the figure
1 ”indicates an imaging area monitored by the camera 110, and an area indicated by an alphabet such as“ A to G ”is a point of interest (the above-described“
(Corresponding to the monitoring points indicated by A _{1 to} A ₈ ). Further, “503” indicates an imaging area (field of view) constantly monitored by the wide-view fixed camera 401.

As in the first embodiment, for controlling the camera 110, it is possible to take an image with a coordinate system as shown in FIG. 3 and a zoom value as shown in FIG. A correspondence table relating to the angle of view and a table holding information on the position and range of each monitoring point as shown in FIG. 5 are used. Here, since one camera 110 is used, only one type of each of the correspondence table of FIG. 4 and the table of FIG. 5 is prepared.

FIG. 19A shows the wide-view fixed camera 4.
01 indicates a state in which an abnormality is detected in a certain attention area while the entire imaging area 503 is monitored. "5" in the figure
04 ”(= A) indicates a region of interest in which an abnormality has been detected. When an abnormality is detected in this way, the camera is operated as described above to check the detailed status of the region of interest 504. By controlling the camera 110 from the camera control device 130, the area of interest 504 is displayed in a zoom-in display, and the shooting area of the camera 110 at this time is indicated by “505” in the figure. ) Shows a screen display state of the monitor (for example, the monitor 160) at this time.

FIG. 20A shows a state where a new abnormality is detected. “505” (= B) and “506” (= C) in the figure each indicate a region of interest in which a new abnormality is detected. When a new abnormality is detected in this way, the attention area in which the abnormality still exists (here, for example, the attention area 504 in FIG. 19A).
At the same time, attention areas 505 and 50 where a new abnormality exists
6 is included in the shooting range of the camera 110, the camera 110 is controlled by the camera control device 130,
The shooting range of the camera 110 is changed. The photographing range of the camera 110 after this change is indicated by “507” in the figure. FIG. 20B shows a state of screen display on the monitor at this time. Here, three attention areas 50
Since it is assumed that an abnormality is detected in 4, 505, and 506, the attention areas 504, 505, and 506 are zoomed up on the three monitors 160, 170, and 403, respectively. As described above, when a plurality of abnormalities are detected, the video signal obtained by photographing with the camera 110 (that is, the video signal of the photographing area 507) is not output as it is to the monitor. Predetermined signal processing such as enlargement processing is performed on the detected video signal of the region of interest and output to a monitor. Such an operation is performed by causing the camera control device 130 to transmit the video signal of the camera 110 supplied through the I / O circuit 135 to the CPU 131.
And supplies this to each of the monitors 160, 170, and 403 via the I / O circuit 135.

FIG. 21A shows a case where a predetermined treatment is performed for the abnormality in the region of interest in which an abnormality is detected (here, for example, the region of interest 504 in FIG. 20A). This indicates a state in which the abnormal state has been released. In this case, the camera 110 is controlled by the camera control device 13 so that the photographing region of the camera 110 includes only all the attention regions 505 and 506 for which the abnormal state has not been released.
By controlling from 0, the shooting range of the camera 110 is changed. The photographing range of the camera 110 after this change is indicated by “508” in the figure. In addition, FIG.
Is the monitor at this time (for example, monitors 160 and 17).
0) shows a state of screen display.

As described above, the configuration using one wide-field fixed camera 401 that constantly monitors the entire imaging area and one camera 110 that can control the imaging direction and zoom can be used. As in the embodiment, the abnormality occurrence area can be observed in detail, and other areas can be observed reliably, so that an efficient monitoring system can be constructed. Also, the monitoring system 50
In the case of 0, since the video signal of the attention area where the abnormality exists is enlarged and displayed on the screen by the monitors 160, 170 and 403, even if a plurality of abnormalities occur, they can be observed in detail. it can.

[0120] 2. Basic Processing Hereinafter, the flow of the basic processing in the present embodiment will be described with reference to FIGS. FIG. 22 is a flowchart showing a process when an abnormality is detected in the imaging region. FIG. 23 is a flowchart showing a process in which a predetermined process is performed on the abnormality in the region of interest in which the abnormality has been detected. 9 is a flowchart illustrating a process when an abnormal state is released. These processes take the form of parallel execution. still,
In the camera control device 130, for example, the ROM 133
Has stored therein in advance a processing program according to the flowcharts shown in FIGS. 22 and 23. The processing program is read out and executed by the CPU 131 to perform the operation described below.

2-1. When an abnormality is detected in the imaging area (see FIG. 22 above)

When this processing program is executed, first,
The camera control device 130 sets a point (monitoring point) to be monitored in the imaging area as an initial setting.
As the setting method, various methods such as automatic setting, manual setting, and presetting can be considered, but are not particularly limited. The set monitoring points are processed by the CPU 131, and their position and range information are stored in the RAM 132 or the like as a table as shown in FIG. Further, the camera control device 130 initializes a counter variable N of the number of abnormal regions (the number of monitoring points) and an abnormal region list for holding information on the region (monitoring points) in which an abnormality has occurred ( Step S511). The abnormal area list is used to store information such as at which monitoring point an abnormality is currently occurring.
132 and the like are used. The counter variable N indicates the number of monitoring points registered in the abnormal area list, and the RAM 132 or the like is used for this.

Next, the wide-view fixed camera 401 monitors (photographs) the entire image pickup area as described above. The video signal obtained by this photographing is supplied to the camera control device 130, and a predetermined signal processing is performed by the CPU 131, and any one of the monitors 160, 170, and 403 (here, the monitor 160 is used). ) Is displayed on the screen. The user observes the display screen of the monitor 160 (Step S512).

Then, a command indicating the end of this processing is input by the repetition loop processing (abnormality detection loop processing) in steps S512, S513, and S520 until an abnormality is found or from the user using the pointing device 150 or the keyboard 140. Until the observation is performed, observation is performed in this state.

Therefore, when the user finds an abnormality, for example, the user inputs a command to that effect using the pointing device 150 or the keyboard 140.
Note that, similarly to the first and second embodiments, there are various methods for finding an abnormality other than human observation, such as automatic moving object detection using image processing technology. There is no particular limitation.

The CPU 131 determines whether an error has occurred based on a command input from the pointing device 150 or the keyboard 140 (or by the above-described image processing or the like) (step S513).

As a result of the determination in step S513, if no abnormality has occurred, the CPU 131 determines whether or not a command indicating the end of this process has been input from the user by the pointing device 150 or the keyboard 140 (step S520). . If the result of this determination is that this processing has ended, this processing ends.
It returns to step S512 and returns to the abnormality detection loop processing again.

On the other hand, if the result of determination in step S513 indicates that an abnormality has occurred, the CPU 131 adds (registers) the monitoring point where the abnormality has occurred to the abnormal area list and increases the counter variable N by 1 (step S514). .

Then, the CPU 131 sends a predetermined instruction (parameters (p _c , t _c , z)) to the optical parameter controller 112 and the geometric parameter controller 113 provided in the camera 110 by a control signal. As I / O
Provided via circuit 135. This control signal is a signal indicating a shooting direction and a zoom value for including in the imaging range all areas where an abnormality registered based on the abnormal area list is detected. The camera 110 receiving this changes the shooting direction and the zoom value, and shoots details of the attention area where the abnormality is detected. The video signal thus obtained is supplied to the camera control device 130 (step S5).
15).

In the camera control device 130, the CPU 1
Reference numeral 31 performs predetermined signal processing on the video signal from the camera 110, and generates an enlarged video signal of each area where an abnormality has occurred. The CPU 131 also controls the monitor 16 on which the video signal of the wide-view fixed camera 402 is displayed on the screen.
The monitor numbers 1 and 2 are defined for the other monitors 170 and 403 except for 0, and the monitor counter variable n is initialized (step S516). The RAM 132 and the like are also used as the monitor counter variable n.

Then, the enlarged video signals corresponding to the areas registered in the abnormal area list are all output to different monitors by the repetitive loop processing in steps S517 to S519. That is, the CPU 131 supplies the enlarged video signal of the area indicated by the counter variable N registered in the abnormal area list to the monitor indicated by the monitor counter n. As a result, the enlarged video signal in the area indicated by the counter variable N is displayed on the screen on the n-th monitor (step S517). After that, the CPU 13
1 increases the monitor counter n by 1 (step S51).
8) It is determined whether or not the monitor counter n has exceeded the counter N (step S519), and steps S517 and S519 are executed until the monitor counter n exceeds the counter N. Note that the predetermined signal processing performed by the CPU 131 includes digital image enlargement processing such that the size of each area displayed on the screen is the same as the size of the screen of the monitor. As the digital image enlargement processing, for example, an image processing technique called a linear interpolation method or a cubic interpolation method can be used.

After the screen display of the area where all the abnormalities have occurred is completed, the CPU 131 uses the pointing device 150 or the keyboard 140 to display the screen.
It is determined whether or not a command indicating the end of this process has been input from the user (step S520). As a result of this determination,
If the process has ended, the process ends. Otherwise, the process returns to step S512 and returns to the abnormality detection loop process again.

2-2. Processing in a case where an abnormal state is canceled by performing a predetermined treatment on a region of interest in which an abnormality has already been detected (see FIG. 23).

When this processing program is executed, predetermined processing is performed on the abnormality in the attention area where an abnormality is detected by executing the processing program of FIG. 22 (step S521). Then, steps S521, S522,
Until the problem for the above-mentioned abnormality is solved by the repetitive loop processing in S529 and S529, or until a command indicating the end of the present processing is input from the user through the pointing device 150 or the keyboard 140, the above-mentioned state for solving the problem remains unchanged. A predetermined process is performed.

Therefore, when the problem is solved for a certain abnormality, the CPU 131 deletes the solved region from the abnormal region list and decreases the counter variable N by 1 (step S523). This step S52
As a result of No. 3, when nothing is registered in the abnormal area list, that is, when no abnormal area exists, the CPU 131 sets the camera 110 (not shown).
Is set to the standby state and nothing is output to the monitor.
Then, the present processing ends.

Next, the CPU 131 controls a predetermined instruction (parameters (p _c , t _c , z)) to the optical parameter controller 112 and the geometric parameter controller 113 provided in the camera 110. The signal is given via the I / O circuit 135. This control signal is a signal indicating a shooting direction and a zoom value for including in the imaging range all the remaining areas in which an abnormality detected based on the abnormal area list is detected. The camera 110 receiving this,
By changing the shooting direction and the zoom value, the details of the attention area where the abnormality is detected are shot. The video signal thus obtained is supplied to the camera control device 130 (Step S
524).

In the camera control device 130, the CPU 1
Reference numeral 31 performs predetermined signal processing on the video signal from the camera 110, and generates an enlarged video signal of each of the remaining areas where an abnormality has occurred. Further, the CPU 131 initializes a monitor counter variable n (step S525).

Then, steps S517 to S517 in FIG.
As in step S519, all the enlarged video signals corresponding to the remaining areas registered in the abnormal area list are output to separate monitors by the repetitive loop processing in steps S526 to S528.

After the screen display of all the remaining areas where the abnormality has occurred is completed, the CPU 131
Using the pointing device 150 or the keyboard 140, it is determined whether or not a command indicating the end of this process has been input by the user (step S529). If the result of this determination is that this processing has ended, this processing ends. Otherwise, processing returns to step S521 and returns to loop processing again.

In the third embodiment described above, only one camera 110 having a variable imaging range is used. However, the present invention is not limited to this, and a plurality of cameras 110 as in the second embodiment may be used. It is also applicable to surveillance systems using cameras.
In this case, by increasing the number of cameras, the number of areas where an abnormality to be imaged by each camera has occurred decreases and the imaging range becomes narrower. As a result, it becomes possible to output a higher resolution video to the monitor. Also,
The object of the present invention can be achieved even with a simplified configuration in which only the wide-field fixed camera 401 is used without providing the camera 110 having a variable imaging range. In this case, for example, in an image obtained by imaging with the wide-field fixed camera 401, digital image enlargement processing is executed on a portion where it is detected that an abnormality has occurred.

In the above-described second and third embodiments, one wide-field fixed camera 401 is used. However, the wide-field fixed camera 401 is not always required. For example, a camera similar to the camera 110 may be provided instead of the wide-view fixed camera 401, and only one camera that constantly monitors the entire imaging area may be determined from a plurality of cameras. Alternatively, instead of constantly monitoring the entire imaging area and fixing the camera to a certain camera, the camera may be selected dynamically according to the situation at that time.

An object of the present invention is to supply a storage medium storing program codes of software for realizing the functions of the host and the terminal of each of the above-described embodiments to a system or an apparatus, and to provide a computer for the system or the apparatus. Needless to say, this can also be achieved by a program (or CPU or MPU) reading and executing a program code stored in a storage medium. In this case, the program code itself read from the storage medium realizes the function of each embodiment, and the storage medium storing the program code constitutes the present invention.

As storage media for supplying the program code, ROM, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, C
DR, a magnetic tape, a nonvolatile memory card, or the like can be used.

By executing the program code read by the computer, not only the functions of the respective embodiments are realized, but also an OS or the like running on the computer is executed based on the instructions of the program code. It goes without saying that a part or all of the actual processing is performed, and the function of each embodiment is realized by the processing.

Further, after the program code read from the storage medium is written into a memory provided in an extension function board inserted into the computer or a function extension unit connected to the computer, based on an instruction of the program code, The CPU provided in the function expansion board or function expansion unit performs part or all of the actual processing,
It goes without saying that the functions of the embodiments are realized by the processing.

[0146]

As described above, according to the present invention, an image pickup area in which a plurality of points of interest are set is individually imaged in each image pickup area by each image pickup means, and a change among the plurality of points of interest is performed. Is configured to control the imaging range of the imaging means whose imaging range is changeable so as to include the attention point in which is detected. Thus, for a changed portion, it is possible to perform zoom-up and follow-up on the screen separately from the entire imaging area, and to perform efficient monitoring work or the like while constantly controlling the imaging range. .

More specifically, for example, optical parameters at the time of shooting (parameters related to the inside of the camera such as focus, zoom, shutter speed, iris, and gain) and geometric parameters (parameters related to the shooting direction such as pan and tilt) are set. As an interface, two cameras (imaging means capable of controlling an imaging range) which can be referred to and controlled from the outside (control means) are used, and a wide imaging area in which a plurality of points of interest (monitoring points) are set is used. , Each of which is photographed (monitored) in an individual imaging range. At this time, the imaging range of each camera is changed to the position of the point of interest (pan angle, tilt angle,
Based on the zoom value (maximum value or minimum value), external control is performed so as not to capture the same region as much as possible and to capture all the points of interest. Then, a change (movement of a subject, an abnormality) in the imaging region is automatically detected by an image processing technique such as a background difference method or an inter-frame difference method. When a change is detected, the change of the two cameras is determined using a predetermined evaluation function (a function using the position of the imaging range of each camera, the number of points of interest at which the change is detected, and the like as parameters). The camera that captures the detected point of interest is determined, and the image capturing range of the other camera is determined to be an image capturing range including the point of interest whose change has been detected. External control is performed so that the imaging range includes the point of interest. At this time, if a new point of interest where a change has occurred is detected, control is performed to change the imaging range so as to include both points of interest together with the point of interest where the change has been detected earlier. As a result, from each camera, an image of the area where the change has occurred and an image of another area where the change has not occurred are obtained. As a result, the area where the change has occurred is zoomed up and tracked. In addition to the screen display, other areas can be displayed on the screen.

When a camera (wide-field fixed camera) that constantly captures the entire image capturing area and a camera whose image capturing range is controlled as described above are used, the entire image capturing area can be displayed on the screen. The area on which the change has occurred can be displayed on the screen while zooming up and tracking.
At this time, if there are a plurality of points of interest that have changed, in the video signal obtained by controlling the imaging range, the video signal in the area of the point of interest is subjected to enlargement processing, and the screen It may be displayed.

Therefore, according to the present invention, it is possible to perform efficient monitoring work while always controlling the imaging area of the camera system (monitoring system) for monitoring. In other words, the problems that arise when trying to build a surveillance system using conventional techniques, such as the problem of increasing the economic burden due to the increase in the number of cameras and the details of abnormalities due to the use of a wide-field camera are confirmed. In such a case, it is possible to solve the problem of inconvenience, and the problem of not being able to monitor another area during zoom-up, and to perform efficient and accurate monitoring work.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a monitoring system to which the present invention is applied in a first embodiment.

FIG. 2 is a diagram illustrating a state in which an imaging area is monitored by two cameras in the monitoring system.

FIG. 3 is a diagram for explaining a coordinate system corresponding to a shooting direction (pan direction and tilt direction) of each camera with respect to the imaging region.

FIG. 4 is a diagram for explaining a correspondence table in which a zoom value that can be taken by each camera and a viewable angle of view corresponding to the zoom value are indicated by a horizontal angle of view and a vertical angle of view.

FIG. 5 is a diagram for explaining a table in which attributes (parameters) of positions and ranges of monitoring points scattered in the imaging region are set (held).

FIG. 6 is a diagram for explaining what area each camera is actually monitoring (photographing) in the imaging area.

FIG. 7 is a diagram for explaining a monitoring state of each camera at a different time point;

FIG. 8 is a flowchart illustrating a basic process of the monitoring system.

FIG. 9 is a block diagram showing a configuration of a monitoring system to which the present invention is applied in the second embodiment.

FIG. 10 is a diagram for explaining a state in which an imaging area is monitored by one wide-field fixed camera and two cameras in the monitoring system.

FIG. 11 is a diagram for explaining a state when an abnormality is detected in a certain area during monitoring by the wide-field fixed camera.

FIG. 12 is a diagram for explaining a state in which a new abnormality is detected before a predetermined process for the abnormality in the attention area in which the abnormality has been detected previously ends.

FIG. 13 is a diagram illustrating a state where a new abnormality is detected in another region of interest when both cameras are not in a standby state.

FIG. 14 is a diagram illustrating a state in which a predetermined treatment is performed on the abnormality in an area where the abnormality is detected, and the abnormal state is released.

FIG. 15 is a flowchart illustrating a process when an abnormality is detected in an imaging region in the basic processing of the monitoring system.

FIG. 16 is a flowchart illustrating a process in a case where an abnormal state is canceled by performing a predetermined process for the abnormality in the attention area in which the abnormality has been detected in the basic processing of the monitoring system. .

FIG. 17 is a block diagram showing a configuration of a monitoring system to which the present invention is applied in the third embodiment.

FIG. 18 is a diagram for explaining a state in which an imaging area is monitored by one wide-field fixed camera and one camera in the monitoring system.

FIG. 19 is a diagram for explaining a state when an abnormality is detected in a certain area during monitoring of the wide-field fixed camera.

FIG. 20 is a diagram for explaining a state in which a new abnormality is detected before a predetermined process for the abnormality in the attention area in which the abnormality has been detected previously ends.

FIG. 21 is a diagram for explaining a state in which a predetermined treatment is performed on the abnormality in the attention area in which the abnormality is detected, and the abnormal state is released.

FIG. 22 is a flowchart illustrating a process performed when an abnormality is detected in an imaging region in the basic processing of the monitoring system.

FIG. 23 is a flowchart for explaining a process in a case where an abnormal state is canceled by performing a predetermined process for the abnormality in the attention area in which the abnormality has been detected in the basic processing of the monitoring system. .

[Explanation of symbols]

 Reference Signs List 100 monitoring system 110, 120 camera 111, 121 camera unit (imaging system) 112, 122 optical parameter control unit 113, 123 geometric parameter control unit 130 camera control unit 131 CPU 132 RAM 133 ROM 134 secondary storage unit 135 I / O circuit 136 Communication I / F circuit 137 Bus 140 Keyboard 150 Pointing device (mouse) 160, 170 Monitor 180 Network

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Tomoyoshi Takagi 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. F-term (reference) 5C054 CF06 CG05 FC01 FC13 FD07 GB01 HA18

Claims (31)

[Claims] 1. A plurality of image pickup means including at least one image pickup means capable of changing an image pickup range; a setting means for setting a plurality of points of interest in an image picked up by the plurality of image pickup means; Detecting means for detecting changes at a plurality of points of interest set by the following; and control means for controlling the imaging range of the imaging means capable of changing the imaging range based on the detection result of the detecting means. An imaging device characterized by the above-mentioned. 2. An imaging range of all of the plurality of imaging units is changeable, and the control unit is configured to perform a detection based on a detection result of the detection unit.
2. The imaging apparatus according to claim 1, wherein each imaging range of the plurality of imaging units is controlled. 3. The imaging apparatus according to claim 1, wherein the plurality of imaging units include an imaging unit having a fixed imaging range capable of imaging all of the plurality of points of interest. 4. The imaging apparatus according to claim 1, further comprising a signal processing unit that performs an enlargement process on a video signal obtained by the imaging unit whose imaging range is controlled by the control unit. 5. The imaging apparatus according to claim 4, wherein said signal processing means performs enlargement processing on a part of said video signal. 6. The imaging apparatus according to claim 1, wherein said control means has a predetermined interface for controlling said imaging means capable of changing said imaging range. 7. The method according to claim 1, wherein the detecting unit performs predetermined signal processing on the video signals obtained by the plurality of image capturing units to detect a motion in the image, thereby detecting a plurality of attentions set in the image. 2. The imaging apparatus according to claim 1, wherein a change at a point is automatically detected. 8. The imaging apparatus according to claim 7, wherein the predetermined signal processing includes at least image processing using a background difference method or an inter-frame difference method. 9. The imaging apparatus according to claim 1, wherein said control means determines an imaging means for controlling an imaging range using a predetermined evaluation function. 10. The imaging apparatus according to claim 1, wherein the control means determines the imaging range of the imaging means for controlling the imaging range based on the position information of the plurality of points of interest. 11. An imaging system comprising an imaging device, a control device for controlling an imaging operation of the imaging device, and a display device for displaying a video signal obtained by imaging with the imaging device on a screen. The imaging device has at least one imaging unit capable of changing an imaging range, the control device is a setting unit that sets a plurality of points of interest in an image captured by the imaging device, Detecting means for detecting changes at a plurality of points of interest set by the setting means; and control means for controlling the imaging range of the imaging means capable of changing the imaging range based on the detection result of the detecting means. An image pickup system, wherein the display device individually displays each video signal obtained by imaging with the image pickup device on a screen. 12. An imaging range of all of a plurality of imaging units included in the imaging device can be changed, and the control unit, based on a detection result of the detection unit,
The imaging system according to claim 11, wherein each imaging range of said plurality of imaging means is controlled. 13. The imaging system according to claim 11, wherein the imaging device includes an imaging unit having a fixed imaging range capable of imaging all of the plurality of points of interest. 14. The control device according to claim 11, wherein the control device further includes a signal processing unit that performs an enlargement process on a video signal obtained by the imaging unit whose imaging range is controlled by the control unit. Imaging system. 15. The signal processing device according to claim 1, wherein the signal processing unit performs an enlargement process on a part of the video signal.
5. The imaging system according to 4. 16. The imaging system according to claim 11, wherein said control means has a predetermined interface for controlling said imaging means capable of changing said imaging range. 17. The method according to claim 17, wherein the detecting unit performs predetermined signal processing on the video signal obtained by the imaging device to detect a motion in the image, thereby detecting a plurality of points of interest set in the image. 12. The imaging system according to claim 11, wherein a change in the image is automatically detected. 18. The imaging system according to claim 17, wherein the predetermined signal processing includes at least image processing using a background difference method or an inter-frame difference method. 19. The imaging system according to claim 11, wherein said control means determines an imaging means for controlling an imaging range by using a predetermined evaluation function. 20. The imaging system according to claim 11, wherein said control means determines an imaging range of an imaging means for controlling an imaging range based on position information of said plurality of points of interest. 21. An imaging control method for capturing an arbitrary image by a plurality of cameras including at least one camera whose imaging range is changeable, comprising: a setting step of setting a plurality of points of interest in the image. A detection step of detecting a change at a plurality of points of interest set by the setting step; and a control step of controlling the imaging range of the camera capable of changing the imaging range based on a detection result in the detection step. And an imaging control method. 22. An imaging range of each of the plurality of cameras can be changed, and the control step controls each imaging range of the plurality of cameras based on a detection result in the detection step. 22. The imaging control method according to claim 21, wherein: 23. The imaging control method according to claim 21, wherein the plurality of cameras include a camera having a fixed imaging range capable of imaging all of a plurality of points of interest. 24. The imaging control method according to claim 21, further comprising a signal processing step of performing enlargement processing on a video signal obtained by a camera whose imaging range is controlled by said control step. 25. The imaging control method according to claim 24, wherein said signal processing step includes a step of performing enlargement processing on a part of said video signal. 26. The imaging method according to claim 21, wherein the control step includes a step of controlling the imaging range using a predetermined interface for controlling the camera whose imaging range is changeable. Control method. 27. The method according to claim 27, wherein the detecting step performs a predetermined signal processing on the video signals obtained by the plurality of cameras to detect a motion in the image, thereby detecting a plurality of points of interest set in the image. 22. The imaging control method according to claim 21, further comprising a step of automatically detecting a change in the image. 28. The imaging control method according to claim 27, wherein said predetermined signal processing includes image processing using at least a background difference method or an inter-frame difference method. 29. The imaging control method according to claim 21, wherein said control step includes a step of determining a camera for controlling an imaging range by using a predetermined evaluation function. 30. The imaging control according to claim 21, wherein the control step includes a step of determining an imaging range of a camera that controls the imaging range based on positional information of the plurality of points of interest. Method. 31. A storage medium characterized by storing the processing steps of the imaging control method according to claim 21 in a computer readable manner.