KR20100103109A - Apparatus and method for unmanned surveillance using omni-directional camera and pan/tilt/zoom camera - Google Patents

Abstract

PURPOSE: An unmanned surveillance apparatus using an omni-directional camera and a pan/tilt/zoom camera and a method thereof are provided to assign each intrinsic identification code to object images to continuously photograph the same surveillance target object, thereby increasing performance of finding detailed information of an object. CONSTITUTION: The first image obtaining unit(110) obtains all image frames photographed by an omni-directional camera for the entire surveillance target area at a fixed time interval. An object extracting unit(120) extracts an object image from a current image frame based on differences among values of pixels configuring a previous image frame. The second image obtaining unit(130) obtains an object image frame photographed by a pan/tilt/zoom camera for a surveillance target object corresponding to the object image from the surveillance target area. A controller(140) controls rotation and zooming operations of a pan/tilt/zoom camera by tracking movement of the surveillance target object based on a location of the object image in the entire image frame.

Description

Apparatus and method for unmanned surveillance using omni-directional camera and pan / tilt / zoom camera}

The present invention relates to an unmanned surveillance apparatus and method using an omnidirectional camera and a pan / tilt / zoom camera, and more particularly, to an apparatus and method capable of detecting an object entering an unmanned surveillance region and grasping detailed information of the object. It is about.

Recently, the automatic object recognition and tracking technology used in the video field is applied in many fields, the development of related technologies is increasing significantly at home and abroad. The most common application is an unmanned surveillance system using video, which is used for security or crime prevention in underground parking lots, bank automation counters, and large factories. The unmanned surveillance system receives an image of a surveillance camera to identify an external intruder without security personnel and continuously calculates the direction of movement to continuously track an intruder. In the unmanned surveillance system, when an unauthorized intruder approaches a forbidden area, a crime can be prevented by automatically taking an action such as issuing an alarm.

An automatic video recorder is used to record video footage of areas requiring security. Instead of unconditionally recording video in areas that need to be monitored for 24 hours, the unattended surveillance system can automatically monitor and track unauthorized intruders, recording only the video when the alarm is issued, thereby increasing the amount of memory used for video storage. Can be effectively reduced.

The omni-directional camera used in the existing unmanned surveillance system has the advantage of monitoring a large area with one camera, but it is difficult to know the detailed information of the object in case of unauthorized intrusion. Meanwhile, a pan / tilt / zoom (PTZ) camera that captures the detailed view of an object through rotation or zooming along the detected object path distinguishes the bait object, which is not an actual intruder. The disadvantage is that you can't. Therefore, the necessity of developing an unmanned surveillance system using only the advantages of omnidirectional camera and PTZ camera is increasing.

The technical problem to be achieved by the present invention is to capture the entire area to be monitored to detect all the objects entered into the area to be monitored and at the same time to obtain a detailed information of each of the entered objects and the pan / tilt / zoom camera It is to provide an unmanned monitoring apparatus and method used.

Another technical problem to be achieved by the present invention is to capture the entire area to be monitored to detect all the objects entering the monitoring area and at the same time to obtain the detailed information of each of the entered object and the pan / tilt / zoom camera The present invention provides a computer-readable recording medium that records a program for executing an unmanned surveillance method using a computer.

In order to achieve the above technical problem, an unmanned surveillance apparatus using an omnidirectional camera and a pan / tilt / zoom camera according to the present invention includes a plurality of entire image frames photographed by an omnidirectional camera over an entire surveillance region. A first image acquisition unit obtained by obtaining; An object that extracts an object image from the current image frame based on a difference between pixel values of pixels of the entire image frame and pixels of the previous image frame temporally preceding the current image frame. Extraction unit; A second image acquisition unit for acquiring an object image frame photographed by a pan / tilt / zoom camera with respect to the object to be monitored corresponding to the object image in the area to be monitored; And a controller configured to control the rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the object to be monitored based on the position of the object image within the entire image frame.

In order to achieve the above technical problem, the unattended monitoring method using the omnidirectional camera and the pan / tilt / zoom camera according to the present invention, the entire image frame photographed by the omnidirectional camera over the entire area to be monitored for a predetermined time Obtaining a first image at intervals; An object that extracts an object image from the current image frame based on a difference between pixel values of pixels of the entire image frame and pixels of the previous image frame temporally preceding the current image frame. Extraction step; A second image acquisition step of acquiring an object image frame photographed by a pan / tilt / zoom camera with respect to the object to be monitored corresponding to the object image in the area to be monitored; And controlling a rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the object to be monitored based on the position of the object image within the entire image frame.

According to the unattended monitoring apparatus and method using the omni-directional camera and the pan / tilt / zoom camera according to the present invention, for the entire image frame photographed by the omnidirectional camera and the monitored object entering the surveillance area. By using all of the object image frames photographed by the pan / tilt / zoom camera, even when a plurality of monitored objects exist, all of them can be detected, and detailed information of the monitored objects can be obtained. In addition, by assigning unique identification codes to the plurality of object images, it is possible to improve the performance of grasping the detailed information of the object by continuously photographing the same monitored object. Furthermore, when there are a plurality of monitored objects to be photographed by one pan / tilt / zoom camera, all the monitored objects can be photographed by setting priorities based on identification codes assigned to the respective object images.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the unattended monitoring device and method using an omnidirectional camera and a pan / tilt / zoom camera according to the present invention.

1 is a block diagram showing the configuration of a preferred embodiment of the unmanned monitoring apparatus according to the present invention.

Referring to FIG. 1, the unmanned monitoring apparatus according to the present invention includes a first image acquisition unit 110, an object extractor 120, a second image acquisition unit 130, and a controller 140.

The first image acquisition unit 110 acquires a plurality of all image frames photographed by the omnidirectional camera over the entire surveillance region at regular time intervals.

The omni-directional camera is a camera that can take a picture of surrounding objects in a single video frame over 360 ° without a blind spot, and is used to simultaneously observe a large surveillance area. 2 is a diagram illustrating an example of an image frame photographed by the omnidirectional camera. Referring to FIG. 2, it can be seen that all the shapes of 360 ° around the omnidirectional camera appear in the image frame.

The first image acquisition unit 110 receives a plurality of temporally continuous whole image frames photographed by the omnidirectional camera at regular time intervals with respect to the surveillance target area. As a reference image for monitoring an object entering the surveillance region from a plurality of whole image frames photographed by the omnidirectional camera, a background image without an object in the surveillance region may be modeled.

The object extractor 120 may generate an object image from the current image frame based on a difference between the pixels constituting the current image frame and the pixels constituting the previous image frame temporally preceding the current image frame. Extract

As a method of extracting an object image by the object extractor 120, there is a method of using only a change of a pixel value between a continuous previous image frame and a current image frame. When an object enters a surveillance region where an object does not exist, a change occurs in the entire image frame photographed by the omnidirectional camera, which is a pixel in the region where the object entering the surveillance region on the entire image frame is located. It appears as a change in value. Therefore, the object extractor 120 may compare the pixel values of the pixels constituting the previous image frame and the current image frame, respectively, and determine the region where the change in the pixel value occurs as the object image.

As another method of extracting the object image, a criterion for extracting the object image from the current image frame may be set using a plurality of previous image frames. That is, the object extractor 120 generates a background image for extracting the object image based on a plurality of previous image frames that temporally precede the current image frame, and configures the current image frame and the pixels constituting the background image. An object image may be extracted based on a difference between pixel values of the pixels. The region where the pixel value does not change over a plurality of previous image frames is regarded as a background where no object exists, and the object extractor 120 generates a background image using the same. When the area of the pixel value changed between the background image and the current image frame is determined as the object image, the object image can be extracted more accurately than when comparing the previous image frame with the current image frame.

When the object displayed in the surveillance region is not one but a plurality of objects, the plurality of object images are extracted by the object extractor 120. 3 is a diagram illustrating an example of a plurality of object images extracted by the object extractor 120 from an entire image frame. In FIG. 3, a rectangular area represents a plurality of object images. In this case, the object extractor 120 assigns a unique identification code to each of the plurality of object images, and allows the corresponding object image to maintain the same identification code even in a video frame that is temporally continuous to the current video frame. Whether the object is the same can be determined by checking whether the object images extracted from a plurality of consecutive whole image frames maintain substantially the same pixel value.

The second image acquisition unit 130 acquires the object image frame photographed by the pan / tilt / zoom camera for the object to be monitored corresponding to the object image in the area to be monitored.

When the object image is extracted by the object extractor 120 from the entire image frame obtained by the omnidirectional camera, it means that there is an object entering the surveillance region. Therefore, it is necessary to grasp detailed information of the object, and a pan / tilt / zoom camera is used for this. The pan / tilt / zoom camera is a camera that can rotate and zoom in up, down, left and right directions, and is used to continuously track a specific object. The pan / tilt / zoom camera is interlocked with the omnidirectional camera. When the object extractor 120 extracts the object image from the entire image frame photographed by the omnidirectional camera, the pan / tilt / zoom camera captures a monitoring object corresponding to the object image.

4 is a diagram illustrating an example of an object image frame photographed by a pan / tilt / zoom camera. Referring to FIG. 4, the pan / tilt / zoom camera captures a monitoring object shown in one object image among the object images extracted from the entire image frame shown in FIG. 3. As such, the image captured by the pan / tilt / zoom camera includes detailed information of the object to be monitored, and the second image acquisition unit 130 is an object image obtained by photographing the object to be monitored by the pan / tilt / zoom camera. Take a frame. The acquired object image frame is stored for a certain time and is used to identify the photographed object to be monitored or to detect the movement of the object within the area to be monitored. At this time, the pan / tilt / zoom camera should not photograph the fixed point where the monitored object exists, but when the monitored object moves, it must continuously photograph the monitored object along the moving path. To this end, the unmanned monitoring device according to the present invention includes a control unit 140.

The controller 140 controls the rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the monitored object based on the position of the object image in the entire image frame.

As described above, the pan / tilt / zoom camera captures a monitored object corresponding to the object image extracted from the current image frame by the object extractor 120, and when the monitored object moves, the monitored object is moved along the moving path. The object must be tracked and photographed continuously. At this time, the controller 140 tracks the movement of the monitored object and performs a function of allowing the pan / tilt / zoom camera to continuously photograph one monitored object. The motion tracking of the monitored object is performed based on the position of the object image within the entire image frame. That is, the controller 140 controls the pan / tilt / zoom camera according to the movement of the object to be monitored by the object extractor 120 to detect the position of the object image extracted from the entire image frame and the pan / tilt within the surveillance region. The rotation angle of the pan / tilt / zoom camera is calculated based on the position of the tilt / zoom camera. Also, when the distance between the monitored object and the pan / tilt / zoom camera changes, the zooming ratio of the pan / tilt / zoom camera is calculated accordingly. In this case, the zooming ratio may be calculated according to the size change of the object image within the entire image frame, not the distance between the monitored object and the pan / tilt / zoom camera.

The rotation operation of the pan / tilt / zoom camera is divided into up and down directions and left and right directions. Therefore, the controller 140 must calculate both the rotation angle in the vertical direction and the rotation angle in the left and right directions. The angle of rotation in the vertical direction and the left and right directions depends on the installation position of the pan / tilt / zoom camera installed in the area to be monitored. For example, when the pan / tilt / zoom camera is installed on the wall of the area to be monitored, the rotation in the vertical direction is mainly adjusted according to the height of the object to be monitored, and the rotation in the left and right directions is the movement of the object to be monitored. Is adjusted accordingly. In addition, when the pan / tilt / zoom camera is installed on the ceiling of the area to be monitored like the omnidirectional camera, the angle of rotation in the vertical direction and the horizontal direction is determined according to the moving direction of the monitored object. One or more rotation and zooming operations of the pan / tilt / zoom camera in the up, down, left and right directions may occur simultaneously.

The surveillance object area may be represented by one coordinate space, and the omnidirectional camera, pan / tilt / zoom camera and the monitoring object positioned in the surveillance area may be represented by coordinate information in the coordinate space. In this case, the controller 140 may calculate the rotation angle and the zooming ratio of the pan / tilt / zoom camera from the coordinate information of the pan / tilt / zoom camera and the monitoring object.

When there are a plurality of object images extracted from one whole image frame by the object extractor 120, the pan / tilt / zoom camera should photograph each object to be monitored. Accordingly, the second image acquisition unit 130 acquires the same number of object image frames as the number of object images corresponding to the identification codes of the respective object images. If there are multiple pan / tilt / zoom cameras, it is possible for each pan / tilt / zoom camera to continuously capture one monitored object to acquire multiple object image frames at the same time. In order to photograph a plurality of monitored objects by the camera, it is impossible to continuously photograph only one monitored object.

Therefore, the controller 140 sets the priority based on the identification code assigned to each object image so that the pan / tilt / zoom camera sequentially photographs the monitored object corresponding to the object image. In this case, since it is preferable to first capture the new object to be monitored in the area to be monitored to obtain detailed information, the identification code assigned to the object image extracted by the object extraction unit 120 is already received by the second image acquisition unit. In the case of the identification code corresponding to the acquired object image frame, the pan / tilt / zoom camera may be set to a lower priority in the shooting target. This is because the same identification code is assigned to the object image of the same monitored object.

In addition, the entire image frame and the object image frame acquired by the first image acquisition unit and the second image acquisition unit may be stored for a predetermined time and then deleted from a storage device (not shown) or a memory. This is because it is preferable to delete the entire image frame and the object image frame that are no longer used after extracting the object image or grasping the motion and detailed information of the object to be monitored.

As described above, when a plurality of objects enter the surveillance region by using both the entire image frame photographed by the omnidirectional camera and the object image frame photographed by the pan / tilt / zoom camera in the unattended surveillance of the surveillance region. All of them can be detected and detailed information about each object can be obtained.

5 is a flowchart illustrating a process of performing a preferred embodiment of the unattended monitoring method according to the present invention.

Referring to FIG. 5, the first image acquisition unit 110 acquires a plurality of all image frames photographed by the omnidirectional camera over the entire surveillance target area at regular time intervals (S510). Next, the object extractor 120 is configured to determine from the current image frame based on a difference between the pixels constituting the current image frame and the pixels constituting the previous image frame temporally preceding the current image frame. Extract the object image (S520). At this time, a unique identification code is assigned to the extracted object image, and the same identification code is assigned to the same object image even when located in different full image frames.

The second image acquisition unit 130 obtains an object image frame photographed by a pan / tilt / zoom camera with respect to the object to be monitored corresponding to the object image in the area to be monitored (S530). The controller 140 controls the rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the monitored object based on the position of the object image in the entire image frame (S540). In this case, when a plurality of object images are extracted, priority is set according to an identification code assigned to the object image, and the pan / tilt / zoom camera captures a monitoring object corresponding to the object image according to the set priority. In addition, when the object to be monitored moves, the rotation angle of the pan / tilt / zoom camera is calculated along the moving path, and thus the same object to be monitored may be continuously photographed to obtain detailed information of the object.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific preferred embodiments described above, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

1 is a block diagram showing the configuration of a preferred embodiment of the unmanned monitoring apparatus according to the present invention,

2 is a diagram illustrating an example of an image frame photographed by an omnidirectional camera;

3 is a diagram illustrating an example of a plurality of object images extracted by an object extracting unit from an entire image frame;

4 is a diagram illustrating an example of an object image frame photographed by a pan / tilt / zoom camera, and

5 is a flowchart illustrating a process of performing a preferred embodiment of the unattended monitoring method according to the present invention.

Claims (13)

A first image acquisition unit for acquiring a plurality of all image frames photographed by the omnidirectional camera over the entire surveillance region at regular time intervals; An object that extracts an object image from the current image frame based on a difference between pixel values of pixels of the entire image frame and pixels of the previous image frame temporally preceding the current image frame. Extraction unit; A second image acquisition unit for acquiring an object image frame photographed by a pan / tilt / zoom camera with respect to the object to be monitored corresponding to the object image in the area to be monitored; And And a controller configured to control the rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the object to be monitored based on the position of the object image in the whole image frame. Unmanned Surveillance Device. The method of claim 1, And the object extracting unit assigns a unique identification code to the object image extracted from the current image frame. 3. The method of claim 2, And the control unit sets priority to a plurality of monitored objects photographed by the pan / tilt / zoom camera based on identification codes assigned to a plurality of object images, respectively. The method according to any one of claims 1 to 3, And the control unit calculates a rotation angle of the pan / tilt / zoom camera based on the position of the object image and the position of the pan / tilt / zoom camera in the area to be monitored. The method of claim 4, wherein The control unit controls the pan / tilt / zoom camera based on the size of the object image in the entire image frame or the installation position of the pan / tilt / zoom camera and the position of the object image in the surveillance region. Unattended monitoring device for calculating the zooming ratio. The method according to any one of claims 1 to 3, The object extracting unit generates a background image for extracting the object image based on a plurality of previous image frames that temporally precede the current image frame, and includes pixels constituting the background image and pixels constituting the current image frame. And extracting the object image based on the difference between pixel values. A first image acquisition step of acquiring a plurality of all image frames photographed by the omnidirectional camera over the entire surveillance region at regular time intervals; An object that extracts an object image from the current image frame based on a difference between pixel values of pixels of the entire image frame and pixels of the previous image frame temporally preceding the current image frame. Extraction step; A second image acquisition step of acquiring an object image frame photographed by a pan / tilt / zoom camera with respect to the object to be monitored corresponding to the object image in the area to be monitored; And And controlling the rotation and zooming operation of the pan / tilt / zoom camera by tracking the movement of the object to be monitored based on the position of the object image within the entire image frame. Unmanned surveillance method. The method of claim 7, wherein And in the object extraction step, assigning a unique identification code to the object image extracted from the current image frame. The method of claim 8, And in the controlling step, setting priorities for a plurality of monitored objects photographed by the pan / tilt / zoom camera based on identification codes assigned to a plurality of object images, respectively. The method according to any one of claims 7 to 9, In the controlling step, the rotation angle of the pan / tilt / zoom camera is calculated based on the position of the object image and the position of the pan / tilt / zoom camera in the area to be monitored. . The method of claim 10, In the controlling step, the pan / tilt / zoom is based on the size of the object image in the entire image frame or the installation position of the pan / tilt / zoom camera and the position of the object image in the surveillance target area. Unattended monitoring method characterized by calculating the zooming ratio of the camera. The method according to any one of claims 7 to 9, In the object extraction step, a background image for extracting the object image is generated based on a plurality of previous image frames that temporally precede the current image frame, and the pixels constituting the background image and the current image frame are configured. And extracting the object image based on a difference between pixel values of pixels. A computer-readable recording medium having recorded thereon a program for executing the unattended monitoring method according to any one of claims 7 to 9.

Images