KR101578030B1 - Apparatus and method for generating event - Google Patents

Abstract

The present invention relates to an image monitoring apparatus and method, and more particularly, to an apparatus and method for generating an event that accurately detects a set area even when a camera is moved in a pan / tilt / will be. The method of generating an event includes estimating a position of a second camera relative to a first camera through a feature point matching of a first image captured by a first camera and a second image captured by a second camera, Estimating a horizon plane using a three-dimensional absolute coordinate system of minutiae for two images, calculating three-dimensional absolute coordinates for a user area set by a user in one of the displayed first or second images, Tilt / zoom movement of the first camera or the second camera through three-dimensional absolute coordinates and pan / tilt / zoom information, and generating an event when an object enters the converted user area The method comprising the steps of:

Description

[0001] Apparatus and method for generating event [0002]

The present invention relates to an image monitoring apparatus and method, and more particularly, to an apparatus and method for generating an event that accurately detects a set area even when a camera is moved in a pan / tilt / will be.

PTZ (pan, tilt zoom) camera is a structure in which a video camera equipped with a motorized zoom lens is combined with a pan / tilt driving device capable of rotating up and down and left and right. The camera is rotated and zoom in ) And zoom out are possible. Thus, compared to fixed cameras, it has the advantage of monitoring a large area around the camera through rotation of the camera, though it is expensive, and allowing zoom / distance monitoring and an enlargement of interest through zoom lens.

In the wide area monitoring system using PTZ camera, event area detection is a technique for detecting how the event area exists in the corresponding camera image according to the PTZ position of the camera. It is a technique for detecting area based event in the PTZ camera, It is an indispensable technology.

Conventional event area detection is a concept of storing and retrieving, rather than actually detecting, because it uses a method of assuming an event area at a previously designated position when the camera reaches a pre-designated PTZ position in advance. This technique has a problem in that the camera can not detect an event while moving to a designated position, resulting in a monitoring blank.

Japanese Patent Laid-Open No. 2010-193170

SUMMARY OF THE INVENTION It is an object of the present invention to provide an apparatus and method for generating an event that precisely detects a set area even when a camera is panning, tilting, or zooming,

According to an aspect of the present invention, there is provided an event generating method comprising: a first step of generating a first feature point of a first image captured by a first camera and a second feature point of a second image captured by a second camera, Estimating a position of the second camera with respect to the first camera; Estimating a horizon plane using a three-dimensional absolute coordinate system of minutiae for the first and second images; Calculating three-dimensional absolute coordinates of a user area set by a user in one of the displayed first or second images; Detecting transformation of the user area when pan / tilt / zoom movement of the first camera or the second camera is performed through three-dimensional absolute coordinates of the user area and pan / tilt / zoom information; And generating an event when an object enters the converted user area.

In the present invention, the step of estimating the position of the camera may include: obtaining a homography matrix between the first camera and the second camera through feature point matching of a portion overlapping the second image with reference to the first image; And detecting rotational information of the second camera relative to the first camera through disassembly of the homography matrix.

In the present invention, the step of estimating the horizon plane may include classifying a screen region through an MSER from a three-dimensional absolute coordinate system of minutiae points; Selecting a main area below a half of the screen area and designating the main area as a horizon area; Designating minutiae included in the horizon area as minutiae minutiae; And estimating a horizon plane by performing a flat fitting of the 3D absolute coordinates of the zenith surface feature points by a PCA method.

In the present invention, the step of calculating the three-dimensional absolute coordinates may include: receiving edge information of the region set by the user on one of the displayed first or second images; Estimating a corner position of the user area in another image in which the user area is not set using a homography matrix for the horizon plane; And estimating a three-dimensional absolute coordinate of the user area by applying a triangular method to the received corner information and the estimated corner information as a homography matrix for the horizon plane.

In the present invention, the step of detecting the conversion of the user area may include detecting pan / tilt / zoom information from the first and second cameras, a preset zoom scale function, rotating the first camera or the second camera Calculating a position of the first camera or the second camera with respect to a position of the first camera or the second camera before moving the PTZ using the distance between the center and the center of the camera; And generating a projection image of the camera with respect to the three-dimensional absolute coordinate system based on the calculated position.

According to an aspect of the present invention, there is provided an event generating method comprising: a first step of generating a first feature point of a first image captured by a first camera and a second feature point of a second image captured by a second camera, A position information estimating unit estimating a position of the second camera with respect to the first camera; A horizontal plane estimating unit estimating a horizontal plane using a three-dimensional absolute coordinate system of minutiae points of the first and second images; An area setting unit for calculating three-dimensional absolute coordinates of a user area set by a user in any one of the displayed first or second images; An area detecting unit for detecting a change of the user area when the first camera or the second camera performs the pan / tilt / zoom movement through the three-dimensional absolute coordinates of the user area and the pan / tilt / zoom information; And an event occurrence detection unit for detecting an occurrence of an event when an object enters the converted user area.

As described above, according to the present invention, even when the camera is panning, tilting, or zooming, it is possible to accurately detect a set area and accurately generate an event when an object is touched in the setting area.

1 is a block diagram illustrating a configuration of an event generator according to an embodiment of the present invention.
FIG. 2 is a view for explaining a disassembling process of the homography matrix in FIG.
FIG. 3 is a view showing a result of detecting a horizontal plane using the MSER in FIG. 1. FIG.
FIG. 4 is a view showing generation of a horizontal plane according to a result of a horizontal plane fitting using a feature point in FIG.
FIG. 5 is a diagram for explaining the conversion of a user area when the PTZ camera is moved in FIG. 1;
FIG. 6 is a view showing the conversion of the user area when the PTZ of the camera is moved in FIG.
FIG. 7 is a diagram showing an event occurrence in the user area in FIG.
8 is a flowchart illustrating an operation of an event generating method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Referring to the accompanying drawings, the same or corresponding components are denoted by the same reference numerals, do.

1 is a block diagram illustrating a configuration of an event generator according to an embodiment of the present invention.

Referring to FIG. 1, the event generating apparatus 10 includes a first camera 100, a second camera 200, an image processing unit 300, a position information estimating unit 400, a horizontal plane estimating unit 500, An area detection unit 700, and an event occurrence detection unit 800. [0033]

The first camera 100 and the second camera 200 refer to a left camera and a right camera that photograph a subject using, for example, a COMS (Complementary Metal-Oxide Semiconductor) module or a CCD (charge coupled device) , The input first and second images (or moving images) are provided to the COMS module or the CCD module through the lens, and the COMS module or the CCD module converts the optical signal of the subject passing through the lens into an electrical signal Output.

In this embodiment, the first camera 100 and the second camera 200 may be a PTZ (pan, tilt zoom) camera equipped with a motorized zoom lens in a pan / tilt driving device capable of rotating up and down and left and right. Also, any one of the first camera 100 and the second camera 200 may be a PTZ camera.

The image processor 300 reduces noise of the first and second image signals output from the first camera 100 and the second camera 200 and generates gamma correction and color filter array filter and performs image signal processing for image quality improvement such as array interpolation, color matrix, color correction, and color enhancement.

In addition, the image processing unit 300 may generate image files by compressing the image data generated by the image signal processing for improving image quality, or may restore the image data from the image files. The compression format of the image includes reversible format or irreversible format.

Also, the image processing unit 300 can functionally perform color processing, blur processing, edge emphasis processing, image analysis processing, image recognition processing, image effect processing, and the like. The face recognizing process, the scene recognizing process, and the like can be performed by the image recognizing process. For example, brightness level adjustment, color correction, contrast adjustment, contour enhancement adjustment, screen division processing, character image generation, and image synthesis processing can be performed.

The position information estimating unit 400 estimates the position of the first camera 100 based on the feature point of the first image captured by the first camera 100 and the feature point of the second image captured by the second camera 200 2 The position of the camera 200 is estimated.

Here, the position of the camera is a concept that collectively refers to the rotation information about the position of the center of the camera on the reference coordinate system and the direction in which the camera is rotated. In general, the rotation angle with respect to the direction in which the camera is viewed, Quot; refers to rotation from the Z axis of the camera to the camera viewing direction. In this embodiment, the reference coordinate system is set as a three-dimensional absolute coordinate system for convenience.

In the present embodiment, the reference of the three-dimensional absolute coordinate system will be referred to as the first camera 100. That is, the center of the first camera 100 is the origin of the three-dimensional absolute coordinate system, and the direction of the first camera 100 is the Z axis of the three-dimensional absolute coordinate system. The X and Y axes (i.e., the horizontal axis and the vertical axis) of the image plane of the three-dimensional absolute coordinate system were taken as the X and Y axes. In this embodiment, the first camera 100 is set as a reference camera.

The position estimation of the position information estimation unit 400 estimates the position of the second camera 200 excluding the first camera 100 on the three-dimensional absolute coordinate system. To this end, the second camera 200 must have a field of view (FOV) that overlaps the first camera 100.

The position information estimating unit 400 estimates the position of the first camera 100 by comparing the first and second images captured by the first camera 100 and the second image captured by the camera 200 with a homography matrix and finds the center position and field of view of the second camera 200 through the disassembly process of the homography matrix.

In this embodiment, for example, a speed up robust features (SURF) algorithm is used to obtain a homography matrix. The SURF algorithm represents an algorithm that finds features that are invariant to environmental changes, taking into account environmental changes such as scale, illumination, and viewpoint from multiple images. The SURF algorithm consists of the feature extraction of the input image, the setting of the reference direction and the size of the feature, and the generation of the descriptor. The contents related to the SURF algorithm are already known, and thus the detailed description will be omitted.

Next, the process of disassembling the homography matrix will be described with reference to FIG. Referring to FIG. 2, C1 is the center position of the first camera 100 and C2 is the center position of the second camera 200. FIG. z1 represents the view direction of the first camera 100, and z2 represents the view direction of the second camera 200. [ R represents the rotation from z1 to z2, and t represents the parallel movement from C1 to C2. The goal of the homography matrix disassembly process is to extract R and t from the homography matrix. Assuming that the homography matrix is H, an equation such as Equation 1 can be obtained by singular value decomposition (SVD).

D, U ', V' and defines it as shown in Equation (2).

Using the equation (2), the candidates of R and t can be obtained as in equation (3).

In Equation (3), s is a scale factor. A total of four position candidates of the second camera 200, which can be obtained using the obtained R and t candidates, are shown in Equation (4).

In order to select the position corresponding to the position of the second camera 200, depth information is obtained through triangulation of the feature points at each position, and the depth information of the first camera 100 and the second camera 200 depth) are both positive.

The horizontal plane estimating unit 500 estimates the horizontal plane using the three-dimensional absolute coordinate system of the minutiae for the first image and the second image.

To find the plane, you must find the characteristic points of the plane. Therefore, the horizon plane information generating unit 500 classifies the screen area through maximally stable extremal regions (MSER) using the three-dimensional absolute coordinates of the minutiae obtained from the position information estimator 400, Select the area to be the main area and designate it as the horizon area. The minutiae points included in the designated surface area are designated as minutiae minutiae points. Thereafter, the three-dimensional absolute coordinates of the zenith surface feature points are subjected to plane fitting using, for example, a principle component analysis (PCA) method to finally estimate the horizon plane. Then, a homography matrix (H _GP ) is generated for the estimated horizontal plane.

In the present embodiment, MSER is preferentially used to find the horizon. The MSER technique specifies a distinguished region called the extremal region, where the segment region is a closed region. If the segmentation region has a local minimum, the segmentation region is maximally stable, and the segmentation region may be a Maximal Stable Extremal Region (MSER). For the MSER technique, see 'J. The detailed description will be omitted because it is disclosed in detail in Matas, O. Chun, M. Urban, T. Pajdla, "Robust Wide Baseline Stereo from Maximally Stable Extremal Regions," British Machine Vision Conference, BMVC 2002 ". FIG. 3 is a view showing a result of detecting a horizontal plane using the MSER in FIG. 1. FIG. 3A to 3F show MSER results for an input image. As shown in FIG. 3C and FIG. 3F, the area indicated in red indicates the horizon plane relatively accurately. If the feature points are classified based on the extracted region and the classified feature points are flat-fitted to the PCA, the result of the desired plane can be obtained.

FIG. 4 shows that the feature points are located in the three-dimensional absolute coordinate system, the feature points corresponding to the horizon planes are classified, and the planes corresponding to the feature points are fitted to generate the horizon planes.

The area setting unit 600 specifies a user area set through the user from any one of the displayed first and second images, and calculates the three-dimensional absolute coordinates for the user area. The setting of the user area is basically based on polygons. That is, the user can set a user area by taking a point on a screen on which one of the first and second images is displayed.

The area setting unit 600 designates a user area on the screen on which one of the first and second images is displayed as a user area, and the horizon plane generated by the horizon plane estimating unit 500 includes a homography matrix (H _GP ) is used to estimate the edge position of the user area in another image in which the user area is not set. Then, the three-dimensional absolute coordinates of the user area are estimated by applying the triangular method to the corner information and the edge information mapped to the homography matrix for the horizon plane.

If the coordinates of the feature point designated by the user on the image screen captured by any one of the first camera 100 and the second camera 200 is x, the position of the corresponding point on another camera screen is represented by x ' Can be expressed as

In this embodiment, the trigonometry is applied using x and x ', and finally the three-dimensional absolute coordinate X is obtained. This process is applied to all the corners of the user area.

The area detecting unit 700 detects the conversion of the user area when the first camera 100 or the second camera 200 pan / tilt / zoom moves through the three-dimensional absolute coordinates of the user area and the pan / tilt / zoom information do.

The area detecting unit 700 detects the pan / tilt / zoom information of the first camera 100 and the second camera 200 and the zoom scale function of the first camera 100 or the second camera 200, The position of the first camera 100 or the second camera 200 is calculated with respect to the position of the first camera 100 or the second camera 200 before the PTZ movement using the distance between the center of rotation and the center of the camera, And generates and displays a projection image of the camera with respect to the three-dimensional absolute coordinate system based on the calculated position.

The coordinate X _img at which an arbitrary point X _WC on the three-dimensional absolute coordinate is projected onto the image of the first camera 100 or the second camera 200 can be expressed as Equation (6).

Here, R _WC and t _WC are vectors and matrices representing the camera line and the camera center on the three-dimensional absolute coordinate system, respectively. Where K is a scalar value according to the zoom scale, R is a matrix representing the rotation of the camera line of sight, t is a parallel movement of the camera center, K, R, and t are obtained through pan / tilt / &Quot;

FIG. 5 is a diagram for explaining the conversion of a user area when the PTZ camera is moved in FIG. 1; The zoom scale interpolation disclosed in FIG. 5A represents K, the rotation in FIG. 5B represents R, and the translation represents t.

The area detecting unit 700 performs the mapping on all user area corners and displays the positions of corresponding corners in each camera.

FIG. 6 is a view showing the conversion of the user area when the PTZ of the camera is moved in FIG. Referring to FIG. 6, FIG. 6A shows a set user area (white box). 6B shows the detected user area when the first camera 100 or the second camera 200 is moved to the right by 15 degrees. FIG. 6C shows the detected user area from the state in which the first camera 100 or the second camera 200 is further moved by 15 degrees to the right from FIG. 6B. FIG. 6D shows the detected user area from the state in which the first camera 100 or the second camera 200 is further moved by 15 degrees to the right from FIG. 6C. FIG. 6E shows the detected user area from the state in which the first camera 100 or the second camera 200 zooms in from FIG. 6D. FIG. 6F shows the detected user area in a state where the first camera 100 or the second camera 200 is moved to the left by 15 degrees from FIG. 6E.

The event occurrence detection unit 800 detects occurrence of an event when an object enters the converted user area. FIG. 7 shows that an object moving in the user area is intruded and an alarm (translucent red box) is displayed on the object to inform the occurrence of an event.

The event generator 10 maps the user area previously designated by the user when the PTZ camera moves the PTZ. That is, even if the camera moves the PTZ, the user area designated by the user in the surveillance area can be judged whether or not the moving object has invaded the corresponding area even when the camera PTZ is moved .

8 is a flowchart illustrating an operation of an event generating method according to an embodiment of the present invention. In the following description, the description of the parts overlapping with those of FIGS. 1 to 8 will be omitted.

The event generating apparatus 10 estimates the position of the second camera with respect to the first camera through the feature point matching of the first image captured by the first camera and the second image captured by the second camera S10 ). The event generator 10 obtains a homography matrix between the first camera and the second camera through the feature point matching of a portion overlapping the second image with reference to the first image, And detects rotation information of the second contrast camera.

When the estimation of the position of the second camera relative to the first camera is completed, the event generator 10 performs step S20 of estimating the horizon plane using the three-dimensional absolute coordinate system of the minutiae for the first and second images . The event generating apparatus 10 classifies the screen area through the MSER from the three-dimensional absolute coordinate system of the minutiae, and selects a main area below the screen half of the classified screen area to designate it as a horizontal plane area. Then, the minutiae points included in the horizon plane area are designated as zenith surface feature points, and the 3D plane absolute coordinates of the zenith surface feature points are flattened by the PCA method to estimate the horizon plane.

When the horizon plane estimation is completed, the event generator 10 calculates a three-dimensional absolute coordinate for the user area set by the user in any one of the first image or the second image displayed (S30). The event generating apparatus 10 receives the edge information of the area set by the user from one of the displayed first or second images and displays the edge information of the user on the other image in which the user area is not set using the homography matrix for the ground plane And estimates the corner position of the user area. Then, the event generating device 10 maps the received edge information and the estimated edge information to a homography matrix for the terrain surface, and estimates the three-dimensional absolute coordinates of the user area by applying the trigonometry.

When the calculation of the three-dimensional absolute coordinates of the user area is completed, the event generating device 10 generates the three-dimensional absolute coordinates of the user area, and the pan / tilt / zoom information of the first camera or the second camera A step S40 of detecting the conversion of the user area is performed. The event generating apparatus 10 uses the pan / tilt / zoom information from the first and second cameras, a preset zoom scale function, the distance between the rotation center of the first camera or the second camera moving in PTZ and the center of the camera The position of the first camera or the second camera is calculated with respect to the position of the first camera or the second camera before the PTZ movement. Next, the event generating apparatus 10 generates a projection image of the camera with respect to the three-dimensional absolute coordinate system based on the calculated position.

Thereafter, the event generator 10 performs a step S50 of generating an event when an object enters the converted user area.

Meanwhile, the present invention can be embodied in computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to preferred embodiments. It will be understood by those skilled in the art that the present invention may be embodied in various other forms without departing from the spirit or essential characteristics thereof. Therefore, the above-described embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

100: First camera
200: Second camera
300:
400:
500: Horizontal plane information generating unit
600: area setting unit
700:
800: Event occurrence detection unit

Claims (6)

Estimating a position of the second camera with respect to the first camera through a feature point matching of a first image captured by the first camera and a second image captured by the second camera;
Estimating a horizon plane using a three-dimensional absolute coordinate system of minutiae for the first and second images;
Calculating three-dimensional absolute coordinates of a user area set by a user in one of the displayed first or second images;
Detecting transformation of the user area when pan / tilt / zoom movement of the first camera or the second camera is performed through three-dimensional absolute coordinates of the user area and pan / tilt / zoom information; And
And generating an event when an object is invaded in the converted user area,
Wherein the step of detecting the conversion of the user area comprises:
The first camera and the second camera are located on the same plane as the first camera and the second camera, respectively, using the pan / tilt / zoom information from the first and second cameras, a preset zoom scale function, Or calculating the position of the first camera or the second camera with respect to the position of the second camera; And
And generating a projection image of the camera with respect to the three-dimensional absolute coordinate system based on the calculated position. delete delete Claim 4 has been abandoned due to the setting registration fee. 2. The method of claim 1, wherein calculating the three-
Receiving corner information of the user area set in any one of the displayed first or second images;
Estimating a corner position of the user area in another image in which the user area is not set using a homography matrix for the horizon plane; And
And estimating a three-dimensional absolute coordinate of the user area by applying a triangular method to the corner area information of the user area and the edge position of the user area as a homography matrix for the horizon plane. Generation method. delete delete

Images