JP4812099B2 - Camera position detection method - Google Patents

Description

  The present invention relates to a camera position detection method capable of automatically detecting the positions of a plurality of cameras without involving human hands.

  In a monitoring apparatus that performs monitoring by coordinating a plurality of cameras, it is necessary to accurately adjust the installation position and direction of the camera (camera position calibration). For example, when the person tracked by the camera A goes out of the defense range of the camera A, it is considered that the tracking is taken over by the camera B having the defense range in an adjacent area. At this time, if the positional relationship between the cameras A and B (and their installation directions) is known, each has a common coordinate system, and information can be easily exchanged between the cameras.

  On the other hand, when the positional relationship between the two is unknown, information cannot be exchanged because they do not have a common coordinate system. For this reason, when operating a plurality of cameras in cooperation, it is necessary to calibrate the camera position in order to provide a common coordinate system.

  However, calibration of the camera position requires specialized knowledge and a lot of time. In particular, for example, when operating to improve security at event venues, installation / removal is frequently performed, and the camera position is calibrated at a high cost for each installation. Become.

  Reducing the labor of camera position calibration is one of the most important issues in a monitoring apparatus that performs monitoring by coordinating a plurality of cameras.

  The following patent document 1 and patent document 2 can be referred to as publicly known documents relating to camera position calibration. For example, Patent Document 1 proposes a calibration method between stereo cameras of a stereo camera system. In this calibration method, when the number of persons captured by the first and second stereo cameras is the same, the position of the projected person area correlated with the (each) person captured by one stereo camera and the other stereo The position of the projected person area correlated with the (each) person captured by the camera is compared, and if the position is shifted, calibration correction is performed so that, for example, the position of the center of gravity of the projected person area coincides.

Further, Patent Document 2 proposes a face information measurement system that performs camera calibration by measuring a face marker from an image output of a camera that captures a user's face and simultaneously measuring a background marker. According to this face information measurement system, it is possible to cope with a change in the position of the stereo camera, a change in zoom, and the like by performing camera calibration at the same time while measuring the movement of the face.
Japanese Patent Laying-Open No. 2005-233639 JP 2005-267258 A

  By the way, there are several methods for camera position calibration. For example, when camera position calibration of camera A and camera B is performed, camera B (or camera B) captures images captured by camera A. When the camera A is captured in the image, this is a big clue for obtaining the mutual positional relationship. At this time, it is necessary to know the position of the camera B in the image photographed by the camera A, that is, to find the appearance of the camera B from the image photographed by the camera A, but in general, automatic detection is difficult.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a camera position detection method capable of automatically detecting the positions of a plurality of cameras without manual intervention. It is to provide.

  In order to achieve the above object, a camera position detection method according to the present invention includes a plurality of cameras each having a drive mechanism for controlling the operation of a camera, a panoramic camera for photographing the plurality of cameras, and the plurality of the plurality of cameras. A camera position for controlling the camera and the panoramic camera and having a data processing device for inputting data captured by each camera and performing data processing, and obtaining a positional relationship of the cameras based on the images captured from the plurality of cameras and the panoramic camera A detection method comprising: a first step of photographing the plurality of cameras with the panoramic camera; a second step of operating the plurality of cameras by controlling the drive mechanism; and the panoramic camera to detect the plurality of cameras. A third step of photographing a plurality of cameras after the operation, and a camera before and after the operations of the plurality of cameras. The fourth step of performing data processing for specifying a plurality of cameras in the captured image based on the difference image data obtained by imaging the camera, and calculating the positions of the plurality of cameras from the images captured by the specified plurality of cameras And a fifth step for performing data processing to be performed.

  In this case, the drive mechanism for controlling the operation of the camera is a drive mechanism for the pan / tilt / zoom function of the camera, and a mark is attached to the camera. In the fourth step, the mark added to the camera is attached. It may be configured to detect and identify a plurality of cameras. Further, as the panoramic camera here, one of a plurality of cameras each having a drive mechanism for controlling the operation of the camera may be used.

  According to the camera position detection method of the present invention, for example, a physical movement of the camera is performed by a driving mechanism of the camera pan / tilt / zoom function, and the difference between the images before and after the movement is obtained. A certain camera can be identified from the image and the identified camera can be detected. Accordingly, it is possible to automatically detect the position of the camera that appears in the image without any manual intervention.

  Specifically, a panning / tilting / zooming camera (PTZ camera), such as a pan / tilt / zoom camera (PTZ camera), controls a driving mechanism for controlling the operation of the camera, and the camera is photographed before and after the movement, thereby moving The camera is identified in the image, and the position of the identified camera is calculated from a plurality of images photographed by cameras having different photographing directions. Thereby, the camera position in the image is automatically detected. Conventionally, the specification of the camera position, which has been done manually, is automated, which greatly contributes to the reduction of the trouble of camera position calibration.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating the configuration of a system for carrying out the camera position detection method according to the present invention in one aspect. In FIG. 1, reference numeral 11 denotes a first camera that is a camera position detection target, and reference numeral 12 denotes a second camera that is also a camera position detection target. Each of the first camera 11 and the second camera 12 has a drive mechanism for pan / tilt / zoom functions, and its operation is controlled by a control device (not shown). Reference numeral 13 denotes a panoramic camera having a wide viewing angle. The operation of the panoramic camera 13 is also controlled by a control device (not shown).

  In FIG. 1, reference numeral 100 denotes an input unit that receives an input operation from a keyboard and mouse and has an input port for inputting measurement data from the outside. Reference numeral 101 denotes a keyboard for input operation such as numerical data. Is a pointing device mouse for input operation to operate the system. Reference numeral 103 denotes an arithmetic unit composed of a system device such as a microprocessor (CPU) and a memory, 104 denotes an output unit composed of, for example, a display device, and 105 denotes a data storage device composed of a hard disk device or the like.

  The data storage device 105 temporarily stores the image data captured by the first camera 11, the second camera 12, and the panoramic camera 13 via the input unit 100 and the calculation unit 103, and the calculation unit. 103 reads out the image data stored in the data storage unit 105, and the calculation unit 103 performs data processing on the image data to perform calculation for detecting the camera position.

  In the system configuration, the panoramic camera 13 uses, for example, “Ladybug 2” manufactured by Point Gray Research, and the first camera 11 and the second camera 12 include, for example, “EVI-” manufactured by SONY Corporation. D100 "is used. As the input unit 100, the keyboard 101, the mouse 102, the arithmetic unit 103, the output unit 104, and the data storage unit 105, a personal computer system having these functions can be used. In addition, the personal computer system may function as a control device for controlling the first camera 11, the second camera 12, and the panoramic camera 13.

  In order to detect the positions of the PTZ cameras which are the first camera 11 and the second camera 12, the basic operations are as follows: (1) The physical motion of the PTZ camera is determined by the background subtraction method, the interframe difference method, the optical Detection is performed using any of the difference detection techniques such as flow and pattern matching.

Here, in order to improve the accuracy of detecting the camera position, the following may be devised.
(1) The NOT of the difference image before the PTZ camera motion includes a camera and a still area.
(2) The difference image after the PTZ camera motion includes a camera and a moving area / noise.
(3) By taking the AND of (1) and (2), a camera area including a moving object and noise is obtained.
(4) Repeating (3) a plurality of times and taking all ANDs to detect only the camera.

(5) Mark the camera for easy detection.
(6) The processing results obtained in various camera postures are integrated to perform more accurate position detection.
It may be made like.

  In the above system configuration, one panoramic camera and a plurality of PTZ cameras are used to automatically obtain the positional relationship between the panoramic camera (13) and the PTZ cameras (11, 12). Although a panoramic camera is used to make the object wide, this panoramic camera can be replaced with, for example, a normal fixed camera or a PTZ camera depending on the purpose.

In order to facilitate the installation of the system, simple calibration is automatically performed by the camera position detection method according to the present invention. In this case, there are the following two restrictions when installing the camera.
(1) Install the camera where each other's cameras can be seen.
(2) In order to detect the direction of the panoramic camera from the PTZ camera, the camera is installed so as to be horizontal with the ground.

  FIG. 2 is a flowchart showing a process flow of the simple calibration process by the camera position detection method according to the present invention. The data processing of simple calibration here is performed according to the processing flow as shown in FIG. 2, and an outline of this will be described.

  In simple calibration, first, the position of the PTZ camera viewed from the panoramic camera is found. Basically, the PTZ camera is detected by detecting the movement of the PTZ camera in a panoramic image. However, since the panoramic image captures a wide range, it includes a moving object other than the PTZ camera, noise, and the like. Therefore, the position of the PTZ camera is detected robustly by synchronizing the difference detection technique and the swinging motion of the PTZ camera.

A difference detection result at a pan / tilt angle different from that at the time of background photographing at an arbitrary position p in the image is I _t (p), and a difference detection result at the same pan / tilt angle is J _t (p). t is the number of image acquisitions, and t = 1, 2, 3,. I _t (p) and J _t (p) are binary images, and are 0 when there is no difference and 1 when there is a difference.

Naturally region of PTZ cameras in _I t (p) becomes _1, becomes 0 in _J t (p). Using this fact, the PTZ camera candidate region K _t (p)
Is required. Here, · represents a logical product (and), and ￣ represents bit inversion (not). However, in addition to the PTZ camera, this K _t (p) includes a moving body region in J _t (p), a stationary region in I _t (p), noise due to flickering of illumination, and the like. Therefore, the PTZ camera is repeatedly driven, a plurality of K _t (p) is acquired, and the logical product is taken to remove the area other than the PTZ camera. Specifically, assuming L ₀ (p) = 1,
And labeling is repeated until L _t (p) has only one connected region having a certain size range. As a result, only the PTZ camera is detected.

  FIG. 3 is a diagram for explaining the data processing in which the camera position is actually detected by the camera position detecting method with the image illustrated. FIG. 3A shows an image at a pan / tilt angle different from that at the time of background photographing, and FIG. 3B shows a difference image at this time. The PTZ camera is installed on a tripod near the center of the image. As described above, the PTZ camera is obtained as the difference area, but a lot of moving people and noise are seen. FIG. 3C shows an image at the same pan / tilt angle as in the background photographing, and FIG. 3D shows the difference image at this time. At this time, since the PTZ camera does not appear in the difference area, the bit is inverted and the logical product with the image of FIG. 3B is taken to leave the PTZ camera area as shown in FIG. Noise and people can be erased.

  However, even in this case, there remains a region that is moving at the time of FIG. 3A and determined to be stationary at the time of FIG. 3C. By acquiring the image once again and taking the logical product of the PTZ camera candidate areas, most of the noise can be eliminated as shown in FIG. By repeating this until there is one connected area, only the PTZ camera area can be obtained as shown in FIG.

  Next, the position of the panoramic camera is searched from the PTZ camera, and the pan / tilt posture and the distance between the cameras are obtained. Since a panoramic camera cannot be moved mechanically like a PTZ camera, it is searched from image features such as the shape and color of the panoramic camera in the image. However, since the direction from the panoramic camera to the PTZ camera is already known, the tilt angle is fixed and the line search for the pan angle is sufficient. Here, Ladybug 2 of the used panoramic camera has a red cylindrical shape, and the panoramic camera is detected by detecting this characteristic color and shape. When using a panoramic camera having no shape or color feature, a marker that is easy to find is added to the camera body. The distance between the cameras is obtained using the relationship between the zoom value of the PTZ camera and the size of the panoramic camera that has been shot, and the autofocus value.

  Due to the heightened social security awareness, the demand for surveillance camera systems is rapidly increasing. In the past, products with a level where an image was recorded by a single camera were the mainstream, but in the future, there will be an increasing need for systems for coordinating multiple cameras to monitor a wide area simultaneously and cooperatively. Is expected.

  The biggest difficulty in such a system is the cost of its installation. It takes a lot of time for the specialists to adjust the camera installation precisely over time for each installation. The present invention is an elemental technology necessary for automating this work, and its industrial utility value is extremely high.

It is a figure explaining the system configuration which implements the camera position detection method concerning the present invention by one mode. It is a flowchart which shows the process flow of the simple calibration process by the camera position detection method concerning this invention. It is a figure explaining the data processing which actually detected the camera position by the camera position detection method, exemplifying an image.

Explanation of symbols

11 PTZ camera 12 PTZ camera 13 Panorama camera 100 Input unit 101 Keyboard 102 Mouse 103 Arithmetic unit 104 Output unit 105 Data storage unit

Claims (4)

A plurality of cameras each having a drive mechanism for controlling the operation of the camera, a panorama camera for photographing the plurality of cameras, the plurality of cameras and the panorama camera, and input images taken by the cameras and data Having a data processing device for processing,
In the camera position detection system for obtaining the positional relationship of the cameras based on the captured images from the plurality of cameras and the panoramic camera,
As a control procedure of the data processing device,
A first step of photographing the plurality of cameras with the panoramic camera;
A second step of operating the plurality of cameras by controlling the drive mechanism;
Performing a third step of photographing the plurality of cameras after operating the plurality of cameras by the panoramic camera;
As a data processing procedure of the data processing device,
A fourth step of performing data processing for identifying a plurality of cameras in a captured image based on difference image data obtained by imaging the cameras before and after the operations of the plurality of cameras;
Run a fifth step of performing data processing for calculating the position of said plurality of cameras by image the identified plurality of cameras is captured,
The fourth step is performed by the data processing device.
When the difference detection result at a pan / tilt angle different from that at the time of background photographing is It (p) and the difference detection result at the same pan / tilt angle is Jt (p),
The pan / tilt angle of the camera is repeatedly driven, and a plurality of the camera candidate regions Kt (p) are obtained by the following equation (3):
As L ₀ (p) = 0, labeling is performed using the following equation (4),
It is a step of detecting only the camera by removing a region other than the camera by repeatedly calculating the logical product of Lt (p) until there is only one connected region having a certain range of sizes. A camera position detection method.
However, - the logical product (and), ^- represents bit inversion of (not). The camera position detection method according to claim 1,
One of a plurality of cameras each having a drive mechanism for controlling the operation of the camera is used as the panoramic camera. A plurality of cameras each having a drive mechanism for controlling the operation of the camera, a panorama camera for photographing the plurality of cameras, the plurality of cameras and the panorama camera, and input images taken by the cameras and data Having a data processing device for processing,
The data processing device is
A camera position detection system for obtaining a positional relationship of cameras from captured images from the plurality of cameras and the panoramic camera,
The data processing apparatus, as a control procedure,
A first step of photographing the plurality of cameras with the panoramic camera;
A second step of operating the plurality of cameras by controlling the drive mechanism;
Performing a third step of photographing the plurality of cameras after operating the plurality of cameras by the panoramic camera;
As the data processing procedure, the data processing device
A fourth step of performing data processing for identifying a plurality of cameras in a captured image based on difference image data obtained by imaging the cameras before and after the operations of the plurality of cameras;
And a fifth step of performing data processing for calculating the positions of the plurality of cameras based on images obtained by capturing the plurality of identified cameras.
The fourth step is performed by the data processing device.
When the difference detection result at a pan / tilt angle different from that at the time of background photographing is It (p) and the difference detection result at the same pan / tilt angle is Jt (p),
The pan / tilt angle of the camera is repeatedly driven, and a plurality of the camera candidate regions Kt (p) are obtained by the following equation (5):
L ₀ (P) = 0, and labeling is performed using the following equation (6).
It is a step of detecting only the camera by removing a region other than the camera by repeatedly calculating the logical product of Lt (p) until there is only one connected region having a certain range of sizes. Camera position detection system.
Where • is the logical product (and), ⁻ Represents bit inversion (not). In the camera position detection system according to claim 3,
One of a plurality of cameras each having a driving mechanism for controlling the operation of the camera is used as the panoramic camera.